USER MANUALS


Upgrading from Denodo 7.0: Export the Settings and Metadata of the Current Installation

This section explains how to export the settings and metadata of each module of the Denodo Platform 7.0, including the Solution Manager.

Important

Before proceeding with the upgrade, install the latest update of the Denodo Platform 7.0. The only supported configuration for an upgrade is with the latest update. If you export from previous updates of 7.0, the settings and metadata may have a format that Denodo Platform 9 cannot import.

If you cannot install the latest update on the current installation, see below Alternatives for When You Cannot Install the Latest Update of Denodo 7.0.

To simplify the upgrade from Denodo 7.0 to Denodo 9, consider this:

  • If you do not use a module, you do not need to export its metadata and settings.

  • Install Denodo 8.0 in the same computer where you are going to run Denodo 9. Once you complete the upgrade to 9, remove the installation of Denodo 8.0 because you are not going to use it; it is only necessary to complete the upgrade. Note that you cannot start both versions at the same time but you do not need to for the upgrade.

Virtual DataPort

When exporting the metadata and settings of Virtual DataPort, you are going to obtain two files:

  1. The metadata file. This file contains the CREATE commands for the data sources, views, web services, etc.; the commands to change the settings, etc.

  2. The resources file. This is a file that contains the “resources” of Virtual DataPort. That is:

    • The JDBC drivers used by data sources and that are not included in the Denodo Platform. For example, if you have a Teradata data source, which requires the Teradata driver, the VQL file will include the Teradata driver itself so you do not have to copy it to the new installation.

    • The DenodoConnect components and other extensions you imported using the menu File > Extension management of the Administration Tool.

    • If you enabled Kerberos, the keytab file and the krb5 file.

    • If you imported the SAP Java Connector (JcO), it will also be included within this file.

    • Other resources like these.

    This file includes these resources encoded in Base64.

    To upgrade to previous versions of Denodo, you had to manually copy the drivers that were not included with Denodo, to the new installation. By generating the resources file, the process of upgrading to Denodo 9 is easier.

Important

During this process, it is mandatory to set the configuration property “com.denodo.exportMigrationCompatibility” to “true” in several Denodo components (read more about this at the end of this page). Without this property, the files obtained may not be compatible with Denodo 9. In the case of Virtual DataPort, some VQL commands will fail when executed in Virtual DataPort 9.

  1. Open the Administration Tool of Virtual DataPort 7.0 and log in with an administrator account.

  2. Execute this from the VQL Shell:

    SET 'com.denodo.exportMigrationCompatibility' = 'true';
    SET 'com.denodo.exportOnlyResourcesAndJars' = 'true';
    

    You do not need to restart to apply this change.

  3. Connect to the computer where the Denodo Platform 7.0 is installed, and execute this from the command line:

    • On Windows:

      cd <DENODO_HOME_7_0>
      cd bin
      export.bat --login "<user name>" --file resources_denodo.vql --server "localhost:9999" -P includeJars=yes -P replaceExistingElements=yes
      
    • On Linux:

      cd <DENODO_HOME_7_0>
      cd bin
      ./export.sh --login '<user name>' --file resources_denodo.vql --server 'localhost:9999' -P includeJars=yes -P replaceExistingElements=yes
      

    In this command, replace <user name> with the user name of an administrator user. This command will prompt for your password.

    This script will generate the resources file (resources_denodo.vql) in the folder <DENODO_HOME/bin. Copy this file to your computer.

  4. Go back to the VQL Shell and execute this:

    SET 'com.denodo.exportOnlyResourcesAndJars' = NULL;
    SET 'com.denodo.vdb.catalog.exportMigrationCompatibilityIncludeResources' = 'false';
    
  5. Execute this from the command line (note that the value of the parameter --file is different than in step #3):

    • On Windows:

      cd <DENODO_HOME_7_0>
      cd bin
      .\export.bat --login "<user name>" --file metadata_denodo.vql --server "localhost:9999" -P cluster=yes -P includeStatistics=yes -P includeDeployments=yes -P replaceExistingElements=yes
      
    • On Linux:

      cd <DENODO_HOME_7_0>
      cd bin
      ./export.sh --login '<user name>' --file metadata_denodo.vql --server 'localhost:9999' -P cluster=yes -P includeStatistics=yes -P includeDeployments=yes -P replaceExistingElements=yes
      

    This script will generate the metadata file (metadata_denodo.vql) in the folder <DENODO_HOME/bin. Copy this file to your computer.

  6. If, after executing the script “export”, you see this message:

    MIGRATION TO 8.0 - WARNINGS SUMMARY
    

    Copy the messages that appear right below. You will need this information in the next step of the upgrade process.

    If you do not get this warning, this VQL file can be imported into Virtual DataPort 7.0 without modifying it.

  7. Execute this from the VQL Shell:

    SET 'com.denodo.exportMigrationCompatibility' = NULL;
    SET 'com.denodo.exportOnlyResourcesAndJars' = NULL;
    SET 'com.denodo.vdb.catalog.exportMigrationCompatibilityIncludeResources' = NULL;
    

Data Catalog

  1. Edit the file <DENODO_HOME_7_0>/resources/apache-tomcat/webapps/denodo-data-catalog/WEB-INF/classes/ConfigurationParameters.properties and add the following line:

    com.denodo.exportMigrationCompatibility=true
    
  2. Restart Data Catalog

  3. Log in to Data Catalog 7.0 with an administrator account and export all. To do this, click the menu Administration > Import/Export > Export. In this dialog, select all the options.

  4. Edit the file <DENODO_HOME_7_0>/resources/apache-tomcat/webapps/denodo-data-catalog/WEB-INF/classes/ConfigurationParameters.properties and remove the line

    com.denodo.exportMigrationCompatibility=true
    
  5. Restart Data Catalog

Scheduler and Scheduler Index

  1. Edit the file <DENODO_HOME_7_0>/conf/scheduler/ConfigurationParameters.properties and add the following line:

    com.denodo.exportMigrationCompatibility=true
    
  2. Edit the file <DENODO_HOME_7_0>/conf/arn-index/ConfigurationParameters.properties and add the following line:

    com.denodo.exportMigrationCompatibility=true
    
  3. Restart Scheduler and Scheduler Index.

  4. Log in to Scheduler 7.0 with an administrator account and export all. To do this, go to Configuration > Server configuration and then, click Export. In this page, do this:

    1. Select all projects

    2. Select all the options.

  5. Connect to the computer where the Denodo Platform 7.0 is installed, and execute this:

    • On Windows:

      cd <DENODO_HOME_7_0>
      cd tools\arn-index
      
      export.bat -h localhost -p 9000 -l "<user name>" -P "<password>" -f scheduler-index_upgrade.zip
      
    • On Linux:

      cd <DENODO_HOME_7_0>
      cd tools/arn-index
      
      ./export.sh -h localhost -p 9000 -l '<user name>' -P '<password>' -f scheduler-index_upgrade.zip
      

    In this command, replace <user name> and <password> with the credentials of an administrator user.

    This script will generate the file scheduler-index_upgrade.zip (folder <DENODO_HOME/tools/arn-index). This file will contain the configuration and indexes of Scheduler Index. Copy this file to the computer where you installed Denodo Platform 7.0.

  6. Stop Scheduler and Scheduler Index.

  7. Remove the line

    com.denodo.exportMigrationCompatibility=true
    

    from these files:

    • <DENODO_HOME_7_0>/conf/scheduler/ConfigurationParameters.properties

    • <DENODO_HOME_7_0>/conf/arn-index/ConfigurationParameters.properties.

  8. Start Scheduler and Scheduler Index.

Solution Manager

  1. Log in to Solution Manager with an administrator account. Then, right-click on the tree, on the left side, and click Export.

  2. Select Generate the export file to migrate to the Denodo 8.0 and click Ok.

    You will obtain a file that you will import later into Denodo 8.0.

  3. Follow the steps of these pages of the Upgrade Guide of Denodo 8.0: Review the Virtual DataPort Metadata and Import Settings and Metadata in to the New Installation.

  4. Once you do this, continue with the following stage of the upgrade: Upgrading from Denodo 8.0: Export the Settings and Metadata of the Current Installation.

Configuration Properties of Virtual DataPort for Upgrades: “exportMigrationCompatibility” and “exportOnlyResources”

During the process of exporting the settings and metadata of each Denodo component, you have to set the property “exportMigrationCompatibility” to “true”. Without this property, the files obtained may not be compatible with Denodo 9.

During the time the property “exportMigrationCompatibility” is “true”, consider this for Virtual DataPort:

  • You can continue running queries and creating/modifying/deleting views.

  • Do not perform any task that involve generating VQL statements. For example, do not create a revision with Solution Manager or export the VQL of any element. This is because some of the VQL statements generated when this property is “true”, are only compatible with version 9.

  • Do not change the global settings of Virtual DataPort.

  • Do not modify data sources.

In the process of exporting the metadata of Virtual DataPort, we also instruct the administrator to do this:

  1. Set the property “com.denodo.vdb.catalog.exportOnlyResources” to “true”.

  2. Execute the script “export.bat/sh” to obtain the “resources file”.

  3. Set this property to “false”.

  4. Execute the script “export.bat/sh” again, to obtain the “metadata file”.

We recommend generating two files (the resources file and the metadata file) so you can modify the metadata file easily. Generally, you will not need to modify the metadata file before importing it into Denodo 9. However, if you do need to modify it and this file also included the resources, many text editors would not be able to open it because the resources use a lot of space (consider that the resources file includes the DenodoConnect components you imported, the JDBC drivers, etc.).

In basic scenarios, you can create only one file that includes both the resources and the metadata. To do this, do not set the property “exportOnlyResources” and only execute the script “export” once.

Alternatives for When You Cannot Install the Latest Update of Denodo 7.0

If you cannot install permanently the latest update on the current installation, there are two alternatives but they are more cumbersome:

  1. If you configured any module of the Denodo Platform to store metadata on an external database, follow these steps:

    1. Do a temporary installation of Denodo 8.0 and install the latest update there (you do not need to set up an external database in this new installation).

    2. Export the metadata of 7.0 from the existing installation and import into this temporary installation.

    3. Do the steps of this page (this is to export the metadata from the temporary installation).

    4. Stop the servers of this temporary installation and delete the folder.

  2. If you did not configure any of the modules to store metadata on an external database, follow these steps:

    1. Stop all the servers of the current installation.

    2. Make a copy of the folder of the current installation.

    3. Install the latest update of Denodo 8.0.

    4. Perform the steps explained in this section.

    5. Delete the folder of this installation and put back in place the folder copied in step #2.

The same applies for the installation of Solution Manager.

Add feedback