Upgrading from Denodo 8.0: Export the Settings and Metadata of the Current Installation¶
This section explains how to export the settings and metadata of each module of the Denodo Platform 8.0, including the Solution Manager.
Note
If you are upgrading from Denodo 6.0, go to Upgrading from Denodo 6.0: Export the Settings and Metadata of the Current Installation.
Note
If you are upgrading from Denodo 7.0, go to Upgrading from Denodo 7.0: Export the Settings and Metadata of the Current Installation.
Important
Before proceeding with the upgrade, install the latest update of the Denodo Platform 8.0. The only supported configuration for an upgrade is with the latest update. If you export from previous updates of 8.0, the settings and metadata may have a format that Denodo Platform 9 cannot import.
If you cannot install the latest update on the current installation, see below Alternatives for When You Cannot Install the Latest Update of Denodo 8.0.
If you do not use a module, you do not need to export its metadata and settings.
Virtual DataPort
When exporting the metadata and settings of Virtual DataPort, you are going to obtain two files:
The metadata file. This file contains the
CREATE
commands for the data sources, views, web services, etc.; the commands to change the settings, etc.The resources file. This is a file that contains the “resources” of Virtual DataPort. That is:
The DenodoConnect components, JDBC Drivers and other extensions you imported using the menu File > Extension management of Design Studio.
If you enabled Kerberos, the keytab file and the krb5 file.
If you imported the SAP Java Connector (JcO), it will also be included within this file.
Other resources like these.
This file includes these resources encoded in Base64.
To upgrade to previous versions of Denodo, you had to manually copy the drivers that were not included with Denodo, to the new installation. By generating the resources file, the process of upgrading to Denodo 9 is easier.
Important
During this process, it is mandatory to set the configuration property “com.denodo.exportMigrationCompatibility” to “true” in several Denodo components (read more about this at the end of this page). Without this property, the files obtained may not be compatible with Denodo 9. In the case of Virtual DataPort, some VQL commands will fail when executed in Virtual DataPort 9.
Log in to Design Studio 8.0 with an administrator account.
Execute this from the VQL Shell:
SET 'com.denodo.exportMigrationCompatibility' = 'true'; SET 'com.denodo.exportOnlyResourcesAndJars' = 'true';
You do not need to restart to apply this change.
Connect to the computer where the Denodo Platform 8.0 is installed, and execute this from the command line:
On Windows:
cd <DENODO_HOME_8_0> cd bin .\export.bat --login "<user name>" --metadatapassword "<metadata_password>" --server "localhost:9999" --file resources_denodo.vql -P includeJars=yes -P includeResources=yes -P includeScanners=yes -P includeCustomComponents=yes -P replaceExistingElements=yes
On Linux:
cd <DENODO_HOME_8_0> cd bin ./export.sh --login '<user name>' --metadatapassword '<metadata_password>' --server 'localhost:9999' --file resources_denodo.vql -P includeJars=yes -P includeResources=yes -P includeScanners=yes -P includeCustomComponents=yes -P replaceExistingElements=yes
In this command, replace this:
<user name>
with your user name (an administrator account). This command will prompt for your password.<metadata_password>
with a password to encrypt the sensitive values. You can indicate the password encrypted to avoid entering it in plain text. To do this, use the scriptencrypt_password
to obtain the encrypted password and prefix the value withencrypted
.
Important
You must use the parameter
--metadatapassword <metadata_password>
to export. Without this parameter, the files obtained may not be compatible with Denodo 9.Write down this password; you will need it later, when importing this file in to Denodo 9.
This script will generate the resources file (
resources_denodo.vql
) in the folder<DENODO_HOME>/bin
. Copy this file to your computer.Go back to the VQL Shell and execute this:
SET 'com.denodo.exportOnlyResourcesAndJars' = NULL; SET 'com.denodo.vdb.catalog.exportMigrationCompatibilityIncludeResources' = 'false';
Execute this from the command line (note that the value of the parameter
--file
is different than in step #3):On Windows:
cd <DENODO_HOME_8_0> cd bin .\export.bat --login "<user name>" --metadatapassword "<metadata_password>" --server "localhost:9999" --file metadata_denodo.vql -P cluster=yes -P includeScanners=yes -P includeCustomComponents=yes -P includeStatistics=yes -P includeDeployments=yes -P replaceExistingElements=yes
On Linux:
cd <DENODO_HOME_8_0> cd bin ./export.sh --login '<user name>' --metadatapassword '<metadata_password>' --server 'localhost:9999' --file metadata_denodo.vql -P cluster=yes -P includeScanners=yes -P includeCustomComponents=yes -P includeStatistics=yes -P includeDeployments=yes -P replaceExistingElements=yes
Important
You must use the parameter
--metadatapassword
to export. Without it, the files obtained may not be compatible with Denodo 9. You will need to enter this password later, when importing this file in Denodo 9.
This script will generate the metadata file (
metadata_denodo.vql
) in the folder<DENODO_HOME>/bin
. Copy this file to your computer.If, after executing the script “export”, you see this message:
MIGRATION TO 9 - WARNINGS SUMMARY
Copy the messages that appear right below. You will need this information in the next step of the upgrade process.
If you do not get this warning, this VQL file can be imported into Virtual DataPort 9 without modifying it.
Execute this from the VQL Shell:
SET 'com.denodo.exportMigrationCompatibility' = NULL; SET 'com.denodo.exportOnlyResourcesAndJars' = NULL; SET 'com.denodo.vdb.catalog.exportMigrationCompatibilityIncludeResources' = NULL;
Data Catalog
Log in to Data Catalog 8.0 with an administrator account and export all. To do this, click the menu Administration > Import/Export > Export. In this dialog, select all the options.
Scheduler and Scheduler Index
Log in to Scheduler 8.0 with an administrator account and export all. To do this, go to Configuration > Server configuration and then, click Export. In this page, do this:
Select all projects
Select all the options.
Connect to the computer where the Denodo Platform 8.0 is installed, and execute this:
On Windows:
cd <DENODO_HOME_8_0> cd tools\arn-index .\export.bat -h localhost -p 9000 -l "<user name>" -P "<password>" -f scheduler-index_upgrade.zip
On Linux:
cd <DENODO_HOME_8_0> cd tools/arn-index ./export.sh -h localhost -p 9000 -l '<user name>' -P '<password>' -f scheduler-index_upgrade.zip
In this command, replace
<user name>
and<password>
with the credentials of an administrator user.This script will generate the file
scheduler-index_upgrade.zip
(folder<DENODO_HOME>/tools/arn-index
). This file will contain the configuration and indexes of Scheduler Index. Copy this file to the computer where you installed Denodo Platform 9.
Note
If you have just upgraded from update 8.0u20220815 or earlier, to the latest 8.0 update, you have to upgrade the Lucene indexes, as neither Data Catalog nor Scheduler can use the indexes generated by Scheduler-Index with the update 8.0u20220815 or previous updates.
Solution Manager
Log in to embedded VDP server using Design Studio 8.0 with an administrator account.
Execute this from the VQL Shell:
SET 'com.denodo.exportMigrationCompatibility' = 'true';
You do not need to restart to apply this change.
Log in to Solution Manager with an administrator account. Then, right-click on the tree, on the left side, and click Export.
Go back to the VQL Shell and execute this:
SET 'com.denodo.exportMigrationCompatibility' = NULL;
You will obtain a file that you will import later into Denodo 9.
Note
Solution Manager 9 will not import all the elements included in that file see Import from Solution Manager 8.0.
Configuration Properties of Virtual DataPort for Upgrades: “exportMigrationCompatibility” and “exportOnlyResources”¶
During the process of exporting the settings and metadata of each Denodo component, you have to set the property “exportMigrationCompatibility” to “true”. Without this property, the files obtained may not be compatible with Denodo 9.
During the time the property “exportMigrationCompatibility” is “true”, consider this for Virtual DataPort:
You can continue running queries and creating/modifying/deleting views.
Do not perform any task that involve generating VQL statements. For example, do not create a revision with Solution Manager or export the VQL of any element. This is because some of the VQL statements generated when this property is “true”, are only compatible with version 9.
Do not change the global settings of Virtual DataPort.
Do not modify data sources.
In the process of exporting the metadata of Virtual DataPort, we also instruct the administrator to do this:
Set the property “com.denodo.vdb.catalog.exportOnlyResources” to “true”.
Execute the script “export.bat/sh” to obtain the “resources file”.
Set this property to “false”.
Execute the script “export.bat/sh” again, to obtain the “metadata file”.
We recommend generating two files (the resources file and the metadata file) so you can modify the metadata file easily. Generally, you will not need to modify the metadata file before importing it into Denodo 9. However, if you do need to modify it and this file also included the resources, many text editors would not be able to open it because the resources use a lot of space (consider that the resources file includes the DenodoConnect components you imported, the JDBC drivers, etc.).
In basic scenarios, you can create only one file that includes both the resources and the metadata. To do this, do not set the property “exportOnlyResources” and only execute the script “export” once.
Alternatives for When You Cannot Install the Latest Update of Denodo 8.0¶
If you cannot install permanently the latest update on the current installation, there are two alternatives but they are more cumbersome:
If you configured any module of the Denodo Platform to store metadata on an external database, follow these steps:
Do a temporary installation of Denodo 8.0 and install the latest update there (you do not need to set up an external database in this new installation).
Export the metadata of 8.0 from the existing installation and import into this temporary installation.
Do the steps of this page (this is to export the metadata from the temporary installation).
Stop the servers of this temporary installation and delete the folder.
If you did not configure any of the modules to store metadata on an external database, follow these steps:
Stop all the servers of the current installation.
Make a copy of the folder of the current installation.
Install the latest update of Denodo 8.0.
Perform the steps explained in this section.
Delete the folder of this installation and put back in place the folder copied in step #2.
The same applies for the installation of Solution Manager.