You can translate the question and the replies:

Job schedule with date and time stamp and load to cache.

Hi, I have a requirement as the job need to run every 2 hours daily with date and time stamp consideration and load to cache.As an example consider the scenario as follows: 1st run at 05-11-2015 02:00:00-Job executed sucessfully and load to cache with condition as >05-11-2015 02:00:00 2nd run at 05-11-2015 04:00:00 executed sucessfully same as 1st run with condition as >05-11-2015 04:00:00 3rd run at 05-11-2015 06:00:00 failed In 4th run automatically the job starts at 05-11-2015 08:00:00 but I need to run from condition as >05-11-2015 06:00:00 to 05-11-2015 08:00:00 since 3rd job got failed. Can you please let me know how to set this condition and that the job need to run from the failed job time. Other query is cache_invalidate='Matching_rows" This scenario is whenever any record comes from the source it will match with the cache and if any match found it will delete the record and insert the same record to the cache.if this process is correct my concern is, it will definitely impact the performance since deleting and inserting the same record again.Is there a way that duplicate records cannot be inserted to the cache. Awaiting for your reply.
user
11-05-2015 08:09:05 -0400

10 Answers

Hi, You can use the 'Enable retries' option in the 'Retry section' tab. Using this option you can configure the scheduler to retry in case of an error. You can get more information about retry in the 'Retry Section' of the Scheduler Administration Guide. When you use 'cache_invalidate'='matching_rows' it will invalidate the rows that match the query along with any conditions you specify in the 'Query conditions:' text area and not the exact value of the rows returned. You can find more information about this in the 'VDPCache Extraction Section' of the 'Schedular Administration Guide'. Hope this helps.
Denodo Team
12-05-2015 05:14:32 -0400
Hi, I tried with the option giving an incorrect query in the Extraction section and enabled the retries section with All queries whan an error is returned. Now I try to execute the job, the job keep on running and even if I try to stop the job keep on running only. Let me know how to set up enable retires considering the requirement: 1st run at 05-11-2015 02:00:00-Job executed sucessfully and load to cache with condition as >05-11-2015 02:00:00 2nd run at 05-11-2015 04:00:00 executed sucessfully same as 1st run with condition as >05-11-2015 04:00:00 3rd run at 05-11-2015 06:00:00 failed In 4th run automatically the job starts at 05-11-2015 08:00:00 but I need to run from condition as >05-11-2015 06:00:00 to 05-11-2015 08:00:00 since 3rd job got failed. Can you please let me know how to set this condition and that the job need to run from the failed job time.
user
 Edited on: 04-05-2018 08:05:45 -0400
Hi, You can create a new VDPCache job and in the extraction section you can specify the condition with a interpolation variable and enter the values of the variable as a list. For example you can specify a query like below, Select * from view where shippeddate > @shippeddate Once you click save, you can add a 'New Source' that could be a list, where you can specify the values like '05-11-2015 02:00:00,05-11-2015 04:00:00,05-11-2015 06:00:00'. In the 'Retry section' tab, under the 'What to retry' section, you can select the option 'Only queries which return an error'. So when a job fails at condition '05-11-2015 02:00:00' that job will be re run. You can use the 'Triggers section' to run job at a specific interval. Hope this helps.
Denodo Team
27-05-2015 05:39:22 -0400
Thanks its working I have the same scenario but with between the dates. Example: 1st run between 05-11-2015 02:00:00 and 05-11-2015 04:00:00-Job executed successfully and loaded to cache. 2nd run between 05-11-2015 04:00:00 and 05-11-2015 06:00:00 executed successfully same as 1st run and loaded to cache. 3rd run between 05-11-2015 06:00:00 and 05-11-2015 08:00:0005-11-2015 06:00:00 failed and not loaded to cache. In 4th run the job start should be between 05-11-2015 06:00:00 and 05-11-2015 10:00:00 etc... How to capture the 3rd Jon run failed start time in case of 4th run.
user
 Edited on: 04-05-2018 08:05:45 -0400
Corrected the scenario. 1st run between 05-11-2015 02:00:00 and 05-11-2015 04:00:00-Job executed successfully and loaded to cache. 2nd run between 05-11-2015 04:00:00 and 05-11-2015 06:00:00 executed successfully same as 1st run and loaded to cache. 3rd run between 05-11-2015 06:00:00 and 05-11-2015 08:00:00 failed and not loaded to cache. In 4th run the job start should be between 05-11-2015 06:00:00 and 05-11-2015 10:00:00 etc... How to capture the 3rd Jon run failed start time in case of 4th run.
user
 Edited on: 04-05-2018 08:05:45 -0400
can you please provide solution to the scenario: 1st run between 05-11-2015 02:00:00 and 05-11-2015 04:00:00-Job executed successfully and loaded to cache. 2nd run between 05-11-2015 04:00:00 and 05-11-2015 06:00:00 executed successfully same as 1st run and loaded to cache. 3rd run between 05-11-2015 06:00:00 and 05-11-2015 08:00:00 failed and not loaded to cache. In 4th run the job start should be between 05-11-2015 06:00:00 and 05-11-2015 10:00:00 etc... How to capture the 3rd Jon run failed start time in case of 4th run.
user
 Edited on: 04-05-2018 08:05:45 -0400
Hi, You can use two interpolation variables in the same query and use two lists, for example you can use a query like below, Select * from view where shippeddate between @shippeddate1 and @shippeddate2 Can you also tell me more about the scenario and exactly what you are trying to do with this jobs in the scheduler?
Denodo Team
02-06-2015 14:11:41 -0400
Hi, My requirement is every 2 hrs so totally on a day the job will run for 12 hrs and on every run source data load to cache.Consider today's date 03-06-2015(dd-mm-yyyy) the job is running and the requirement scenario as follows: 1st run between 03-06-2015 00:00:00 and 03-06-2015 02:00:00-Job executed successfully and the source date between the dates loaded to cache. 2nd run between 03-06-2015 02:00:00 and 03-06-2015 04:00:00 executed successfully same as 1st run and loaded to cache between the dates 3rd run between 03-06-2015 04:00:00 and 03-06-2015 06:00:00 failed. 4th run should have to load between the dates 03-06-2015 04:00:00 and 03-06-2015 08:00:00. Ideally I set the interpolation variables between @shippeddate1 and @shippeddate2 according to now() function. So in 4th run dynamically the job will start between 03-06-2015 06:00:00 and 03-06-2015 08:00:00 but here the 3rd job got failed so in the 4th job run I need to set @shippeddate1 to 03-06-2015 04:00:00 which is job failed start time. Hope you understood my requirement.So how to set @ shippeddate1 to job failed start time
user
 Edited on: 04-05-2018 08:05:45 -0400
Hi, If you use the now() then when a job fails and a retry is made, the time at that moment is used for now() and hence it won't be possible to re-run the job with the same time it has failed. That is why I recommended you to use a list of values for your first scenario. You can do the same for the second scenario too. If you specify a query like below, Select * from view where shippeddate between @shippeddate1 and @shippeddate2 You will have the option of specifying 'New Source'. Click on the 'New Source' and select the 'list' option. Now in the 'Query parameter' select 'shippeddate1' and enter the values. You can again click on the 'New Source' and select the 'list' option, but this time select the 'shippeddate2' and enter the values for that. Hope this helps.
Denodo Team
03-06-2015 12:58:31 -0400
Rather than use the Denodo Scheduler, another option that we do is to manage cache loads through our company's existing job scheduler and python scripts. If you use your own scripts (python, java, or whatever), you can build your own logic into it along with any workflow you want. Once we have the logic right in the script right (e.g., when THIS fails then do THAT, or whatever else you want), we just run this VQL statement to refresh the cache: CONNECT DATABASE PUT_DATABASE_NAME_HERE; select * from PUT_VIEW_NAME_HERE CONTEXT('cache_preload'='true', 'cache_invalidate'='true', 'returnQueryResults'='false', 'WaitForCacheLoad'='true'); You can modify the VQL as necessary to get exactly what you want (the VQL above blows away the existing cache and reloads all of it). The Advanced VQL guide explains all the CONTEXT options regarding cache loads. Once you have the right VQL statement(s), you just use the import.sh/.bat scripts to execute it against your Denodo server. I know this doesn't use the same capabilities of Denodo scheduler, but we found it easier to just write our own VQL statements and run them using the import.sh/.bat scripts. Thanks~!
user
03-06-2015 13:08:16 -0400
You must sign in to add an answer. If you do not have an account, you can register here