1: Pre-Migration Preparation & Backup¶

1. Create a backup following the steps in Exporting the Server Metadata. Can be a backup of all the server metadata or just the databases that use cache or contain the cache data sources. It is not necessary to include extensions in the backup. This operation might take some time.

2. Disable any jobs in Denodo Scheduler that preload cached data. See details Jobs.

3. If you have a job in Denodo Scheduler to execute periodically procedure COMPACT_CACHE, disable this job as the new Denodo Lakehouse accelerator does not requiere it.

4. Disable cache. If cache was enabled at server level, it can be disabled at Administration > Server configuration, Cache tab by switching off the Cache status. If it was enabled at database level, it can be disabled at Administration > Database management, select the database and click on Cache and switch off the Cache status.

2: New Schema Configuration¶

1. Create a new SCHEMA in an Iceberg catalog of the existing Denodo Lakehouse Accelerator using procedure CREATE_SCHEMA_ON_SOURCE. Ensure that this new schema targets to a new location in your object storage. This schema alongside the object storage location will be used for the internal control tables and for the cache data tables. As explained in next steps, for cache at database level, a different schema can be configured just for the cache data tables. In example, execute the following to create an schema named new_schema_for_cache in catalog named iceberg with location s3a://some-bucket/lakehouse/java/iceberg_cache.db/ for data source embedded_mpp from database admin_denodo_mpp:

   CALL CREATE_SCHEMA_ON_SOURCE('admin_denodo_mpp', 'embedded_mpp',
       'iceberg', 'new_schema_for_cache',
       's3a://some-bucket/lakehouse/java/iceberg_cache.db/');
   

2. In Read & Write section of the cache data source, configure the new URI for the object storage and the new schema created in the previous step as explained in Configuring the Denodo Lakehouse Accelerator to use Iceberg Tables. The image below shows how to do it with the parameters used in the previous example.

   

3. If data tables were in a different catalog and/or schema than the control tables, create another new schema also in an Iceberg catalog for data tables using procedure CREATE_SCHEMA_ON_SOURCE. Once again, ensure that this new schema targets to a new location in your object storage.

3: Updating existing data sources to the new Denodo Lakehouse Accelerator¶

To benefit of the new adapter in charge of new optimizations to cache with Iceberg tables, the data sources must updated with the internal name used for such adapter. For each Denodo Lakehouse Accelerator data source used as a cache data source, execute in a VQL Shell the following replacing <database_name> and <data_source_name> with the database name and the name of the data source respectively:

CONNECT DATABASE <database_name>;

ALTER DATASOURCE JDBC <data_source_name>
    DATABASENAME = 'lakehouseaccelerator'
    DATABASEVERSION = '2_0_0_on_java';

CLOSE;

4: Re-enabling Cache & Scheduler Jobs¶

1. Enable cache. If cache was disabled at server level, it can be enabled at Administration > Server configuration, Cache tab by switching on the Cache status. If it was disabled at database level, it can be enabled at Administration > Database management, select the database and click on Cache and switch on the Cache status. In both cases, select the data source to be used for cache. For more details, see Configuring the Cache. For cache at database level, if data tables were in a different catalog and/or schema, Configure it in section Specify catalog and schema for data tables (see image below). When all parameters are configured, click OK. This operation might take some time, as it will create the internal control tables for cache as well as all required data tables for cached views.

   

2. Enable jobs to preload cached data for the views in Denodo Scheduler. When enabled, they can be started to immediately preload cache data or wait until they are automatically started by their triggers. See details Jobs. Take into account considerations in Concurrent preload of cached views with the Denodo Lakehouse Accelerator.

3. It is recommended to configure jobs in Denodo Scheduler to execute periodically stored procedures OPTIMIZE_LAKEHOUSE_ACCELERATOR_CACHE_TABLES and OPTIMIZE_LAKEHOUSE_ACCELERATOR_ICEBERG_VIEWS See: Cache Maintenance for the Denodo Lakehouse Accelerator.

5: Cleanup of old data and jobs¶

After checking that migration was successful, the following can be deleted:

• The old schema (or schemas) used for cache.

• Data in the old object storage location used for cache.

• The job(s) in Denodo Scheduler used to execute periodically procedure COMPACT_CACHE, if you had one.