Cache Module¶

Virtual DataPort incorporates a Cache Engine that can store a local copy of the data retrieved from the data sources, in a JDBC database. This may reduce the impact of repeated queries hitting the data source and speed up data retrieval, especially with certain type of sources.

This is the list of databases that you can use for this:

The drivers that are not included in the Denodo Platform have to be downloaded from the vendor’s website.

In addition, you can configure the cache module to store the cache data in the Apache Derby database embedded in Virtual DataPort. However, we strongly recommend using an external Database Management System (DBMSs), especially in production environments.

In addition to the adapters listed above, Virtual DataPort provides a generic adapter that can be used with any other JDBC database to store cached data. The section Generic Support for Other Databases explains how to configure it. However, we strongly recommend that you use one of the databases of the list above because the performance when querying the cache database will be better. The reason is that Virtual DataPort will push down more operations to the supported databases than the generic one.

To use the cache module you have to enable it in the Server (see section Configuring the Cache). Alternatively, you can enable the cache module only for some databases (see section Configuring and Deleting Databases).

After enabling the cache module in the Server or in a database, enable the cache in each view that requires it (see section Configuring the Cache of a View).

When the Server initializes the cache, it creates a set of tables in the database that will have information about the data stored in the cache. For each query whose data is cached, the Server stores a Query Pattern that later is used to know if a query can be “solved” with the data stored in the cache. A Query Pattern is formed by:

• An id that uniquely identifies every Query Pattern.

• The name of the view that these data belongs to.

• The conditions of the WHERE clause of the query used to retrieve the data from the data source.

• The expiration time of the data set that the Query Pattern represents. Once this time is reached, the data set associated with this Query Pattern is marked as invalid and no longer used. At regular intervals (Maintenance Period), the Server deletes the data sets that are marked as invalid. Note that this applies to cache partial only. For cache full, the expiration date represents the time of the latest cache refresh.

• Status of the Query Pattern. This can be valid, processing (while the data is being stored in the cache) or invalid (when the expiration time of the Query Pattern has been reached).

When enabling the cache on a view, the cache module creates a table in the cache’s database that will store the cached data of this view. The data types of the fields of this table are equivalent to the data types of the fields of the view in Virtual DataPort.

When a user executes a query that involves a view with cache enabled, the Server checks if it can retrieve the data from the cache. If not, it retrieves the data from the data source, stores it in the cache and creates a Query Pattern that represents these data.

The cache can also be by-passed for specific queries if you need to access the actual data.

When querying a cache with required fields (e.g. a Web service base view with mandatory input parameters) and the view is cached, you do not have to provide a value for the input parameters. However, you have to do it if the data has to be retrieved from the data source.

You can see information about the Query Pattern of the database by executing the Denodo stored procedure CACHE_CONTENT. The section Displaying the Contents of the Cache of the VQL Guide explains how to invoke this procedure.