You can translate the question and the replies:

Incremental cache implementation in denodo

Hi Team, Our source is a custom java API using custom java wrapper, we have made couple of views in denodo. These views are cache'd everyday using scheduler jobs. After vdb cache job ran, we have set of vdb scheduler jobs which will export the cached data to SQL server. This entire scheduling activity is running at different time. After all done, some stored procedure will run and create some set of reports. We have a sql to compare th API count and SQL server count for each table or view. Most of the time this count will be more on API as after the SQL server load also, in API data will be coming in. In order to solve this problem we have created a view which will give count against each table: table name and API count. We need to cache this view in such a way that within each vdb cache job for each table, we will try to cache this view too only for that table. Like wise we need to cache this API count view incremently just prior to the vdb-cache job for that view/table. Please help me with how to do cache incremently? I tried partial cache with cache pre load = true and explicit_load = true too. But 'in' clause and union SQL is not working to get the resut from cache at one shot. Query is still hitting the data source not cache table. Thanks a lot, Manju
Incremental Cache
user
28-05-2020 09:51:08 -0400
code

1 Answer

Hi, I understand from your question that you would like to match the record count of the API source against the cache table and based on the count difference, you would like to cache a view. If so, you could first schedule the job which will check the count differences and then another job to cache the view, here the view name needs to be interpolated based on the output of the previous job. This view name will be the ones that show count differences. If you would like to create dependencies among the jobs, you could take a look at the section [Dependencies Among Jobs](https://community.denodo.com/docs/html/browse/latest/scheduler/administration/creating_and_scheduling_jobs/configuring_new_jobs/dependencies_among_jobs#dependencies-among-jobs). Alternatively, if you have a field that indicates when the row was added or updated in your API, then you could use the incremental mode of Full cache in your base view. Please refer to the section [Incremental Mode](https://community.denodo.com/docs/html/browse/7.0/vdp/administration/cache_module/cache_modes/incremental_mode#incremental-mode) of Virtual DataPort Administration Guide, for more information. I believe that when you query a view that is configured with [partial cache mode](https://community.denodo.com/docs/html/browse/7.0/vdp/administration/cache_module/cache_modes/partial_mode#partial-mode), the Server checks if the cache contains the data required to answer the query. If not, it retrieves the data from the data source and stores it in the cache. Hope this helps!
Denodo Team
01-06-2020 04:23:27 -0400
code
You must sign in to add an answer. If you do not have an account, you can register here