Availability of full cache

Applies to: Denodo 8.0 , Denodo 7.0 , Denodo 6.0
Last modified on: 03 Jun 2020
Tags: Cache Optimization Performance

Download document

You can translate the document:

Content

When “cache_preload” = “true” is added to the CONTEXT clause of a query, the result of the query is stored in the cache.   This clause allows to load the data in the cache when using partial cache with explicit loads or when using full cache.  (See Virtual DataPort Administration Guide, Section  “Cache Modes”)

To avoid storing duplicate data, while loading the cache the existing data should also be invalidated.  To do this, the parameters ("cache_preload"="true", "cache_invalidate"="true") can be used in the CONTEXT clause of the query.

For instance:

SELECT * FROM phone_inc WHERE pinc_id > 0

CONTEXT ("cache_preload"="true", "cache_invalidate"="true")


When receiving this query, VDP is going to:

  1. retrieve the results of the query from the data source
  2. results are inserted into the cache as hidden
  3. in one single transaction the old rows are marked as “invalid” and the new rows as “valid”

By default, step three is transactional; this means that queries will only see the new data when the whole load process is finished. This ensures client applications always have a consistent view of the data in the cache. All the queries will receive the old results from the cache until the fresh data finishes loading, and from that point new queries will receive the new results also from the cache.

Although the behavior described above provides the best data consistency level, it can overflow the transaction log of the cache database when the volume of data to load/invalidate is large and the size of the transaction log is small.

By adding the 'cache_atomic_operation'='false' property to the CONTEXT clause, the load and invalidation process will be executed in batches of rows (the token 'invalidate_block_size' can be used to specify the batch size). With this additional clause, the process will not be atomic but much less space will be required in the database transaction log (see section “Caching Very Large Data Sets” of the Virtual DataPort Administration Guide for details).

References

Virtual DataPort Administration Guide: Cache Modes

Virtual DataPort Administration Guide: Caching Very Large Data Sets

Questions

Ask a question
You must sign in to ask a question. If you do not have an account, you can register here

Featured content

DENODO TRAINING

Ready for more? Great! We offer a comprehensive set of training courses, taught by our technical instructors in small, private groups for getting a full, in-depth guided training in the usage of the Denodo Platform. Check out our training courses.

Training