Denodo Replicator - User Manual

Last modified on: 04 Jan 2019

Download original document


Preface

Scope

Denodo Replicator provides an interface for import/export operations between several Denodo Platform servers so that you can synchronize them. This document introduces the reader to the tool.

Who should use this document

This document is aimed at system administrators that require performing synchronization operations between different servers of Denodo Platform.

Summary of contents

This document describes:

  • How to install Denodo Replicator

  • Limitations

  • How to add Denodo Platform servers in order to be able to synchronize them

  • How to define an environment where you include all the servers to be synchronized with the master one

  • How to synchronize an environment

Installation

Software requirements

This software requires, at least, JRE1.7 installed. The environment variable JAVA_HOME must be pointing to the folder containing the JRE installation.

As the import/export functionalities make use of several files in your Denodo Platform installation, you will need to have the environment variable DENODO_HOME set and pointing to your Denodo Platform installation folder.

Installation

This software is distributed in a zip file. To install it, extract its content to a folder in the file system.

Execution

To launch the application, execute one of the scripts included in the bin folder:

  • For Windows systems, execute the script denodo-replicator.bat

  • In Unix-like systems or Cygwin, execute the shell script denodo-replicator.sh

These scripts use the JAVA_OPTS environment variable to configure the JVM. If it does not exist, the variable will be set to -Xmx1024m, usually enough for the import/export features. Please, make sure you use enough memory for the import/export processes to work.

Once the tool is launched, the initial empty screen will be displayed. Below there is an image.

Enable SSL connections

Denodo Replicator can connect with a Denodo server that has SSL enabled.

Some configuration might be needed at the launcher scripts:

 

  • If you use the trust store by default and if the server's public key is signed by a well known international CA no configuration is required.

  • Otherwise, you should configure the trust store adding the system property

-Djavax.net.ssl.trustStore=<path_to_truststore>.

Note that, depending on the operating system you are using, you might need to modify denodo-replicator.bat or denodo-replicator.sh.

You could adapt your denodo-replicator.bat script by replacing this line:

%JAVA_BIN% %JAVA_OPTS% -classpath "%REPLICATOR_CLASSPATH%"

  com.denodo.connect.monitoring.replicator.ReplicatorTool %*

with:

%JAVA_BIN% %JAVA_OPTS% -Djavax.net.ssl.trustStore=

    "DENODO_HOME/jre/lib/security/cacerts"

  -classpath "%REPLICATOR_CLASSPATH%"

    com.denodo.connect.monitoring.replicator.ReplicatorTool %*

In addition, all the certificates of the servers secured with SSL taking part in Replicator’s synchronization operations have to be imported into the java cacerts store file of the installation of the platform pointed by the DENODO_HOME environment variable.

There is al alternative option instead of modifying the execution scripts: to modify the JAVA_HOME environment variable. Setting the virtual machine of the DENODO_HOME installation as JAVA_HOME. If you execute Denodo Replicator in a Windows system, you can do this:

> set JAVA_HOME=%DENODO_HOME%\jre

> denodo_replicator.bat

Limitations

IMPORTANT: The Denodo Platform version and patch of the servers involved in the process should be the same in order for the import/export to work. Please, prior to synchronizing using Replicator make sure:

  • Your Denodo Platform local installation: the one defined by DENODO_HOME environment variable.

  • The master node server and all the non-master node servers in the environment have the same Denodo Platform version and the same update installed.

If the previous requirements are not met, please, perform the synchronization processes manually by using the import/export scripts as explained at the “Virtual DataPort Administration Guide”.

Overview

The Denodo Replicator has three main areas: the menu bar, the side tree, and the workspace.

  1. Menu bar

It has the following menu entries:

  • File:
  • New Server: opens the New Server screen in the workspace area.
  • New Environment: opens the New Environment screen in the workspace area.
  • Exit: exits the tool.

  • Help:
  • About: displays the version of the Denodo Replicator and copyright information.

  1. Side tree

It shows a list of the servers and environments that have been added to the tool.

  • Servers are individual Denodo Platform servers.

  • Environments are groups of servers. These groups can be created for the different environments present in an enterprise. For example, an environment could be created for the Development servers, another for the QA servers, another for the Production cluster, etc.

The following options are available for each type of element and they are accessible using the right click popup menu.

  1. Servers section

  • New server: opens the New Server screen in the workspace area.

The right click menu options for each server are:

  • Ping: launches a ping query to the server. The ping time, in milliseconds, is displayed in a dialog.
  • Edit: opens the Edit Server dialog in the workspace area.
  • Delete: removes the server from the list and from all the environments it belongs to. A confirmation popup will be displayed prior to this operation.
  • Add to environment: adds the server to the environment selected from the submenu.

  1. Environments section

  • New environment: opens the New Environment dialog in the workspace area.

The right click menu options for each environment are:

Edit: opens the edit environment dialog in the workspace area

Delete: removes the environment from the list. A confirmation popup will be displayed prior to this operation.

Synchronize Environment: imports metadata from the Master node to the other servers in the environment. When this action is used, a pop-up window appears, where you can choose the options of the synchronization:

  • Include server configuration in synchronization: By default is unchecked. If you check this option the following elements will be synchronized additionally:

  • JMS listeners
  • Configuration of the “Cache maintenance task” of each database.
  • Server connectivity settings (Server port number, Shutdown port number, Auxiliary port number, ODBC port number, Web container HTTP port number, Web container shutdown port number, Auxiliary Web container port number)
  • JVM settings of the Server and the embedded Web container

  • Include binary jars in the synchronization: by default is checked, when you know that the jars are already up-to-date in all the servers, you can do the synchronization without the binary jars, potentially saving many megabytes of transferred files. Note that, if the jars are not updated, the synchronization will fail.

For each server that belongs to the environment, these are the right click menu options:

  • Ping: launches a ping query to the server. The ping time, in milliseconds, is displayed in a dialog.

  • Remove from this Environment: removes the server from the list of servers that belong to this environment. The server is not deleted from the servers section.

  • Set as Master: sets the server as the master node of the environment. The master node of an environment holds the metadata that will be propagated to the other servers in the synchronization operation.

  • Clear Master Status: removes the master status if the server is already the master node.

  • Import Metadata from Master: imports the metadata from the Master to this server. This is useful in case you don’t want to synchronize all the nodes of the environment.
    This option is enabled only in case the environment contains more than one server and one of them is set as the master node.

        When this action is used, a pop-up window appears, where you can choose the options of the import. These options are the same that in the option Synchronice environment, you can learn more about these options above these lines.

        

  1. Workspace

It is the main screen of the tool and it is where all the dialogs are displayed.

Servers section

This section provides a detailed overview on the servers section of the tool.

Create a new server

Before synchronizing any set of servers, those servers must be added to Denodo Replicator. The screen where a new server is defined can be accessed in three ways:

  • From the main menu, File > New Server

  • From the right click popup menu on the Server

  • Using the keyboard shortcut Ctrl+Shift+S

The New Server form looks like this:

The following parameters must be filled:

  • Name: name that will be displayed in the server list in the side tree.

  • Host: host where Denodo Platform server is running. It can be a machine name or an IP address.

  • Port: port where the server is listening to connections Default port is 9999.

  • Login: login to access Denodo server. The user name should have admin privileges as import operations in VDP are performed in single user mode.

  • Password: password to access the server.

All the parameters are required. Once the form is filled, click Ok to create the server, or Cancel to leave the tool with no changes.

Edit a server

An existing server can be edited by clicking on the Edit option of the popup menu of the server node. The form is identical to the New Server form, but the inputs are filled in with the current parameters.

Once the modifications are done, click Ok to modify the server, or Cancel to leave the server with no changes.

Environments section

This section provides a detailed overview on the environments section of the tool. An environment is just a set of servers that can be synchronized between them: import master server metadata into every other server in the environment.

Create a new environment

The screen for defining a new environment can be accessed in three ways:

  • From the main menu, File > New Environment

  • From the right click popup menu on the Environment

  • Using the keyboard shortcut Ctrl+Shift+E

Below there is a screenshot of the New Environment form,  displayed in the workspace area:

The following parameters must be filled:

  • Name: name that will be displayed in the environment list in the side tree

  • Servers in the environment: a list of the servers currently defined in the Servers section. Select on this list all the servers in the environment being created (to select multiple servers use CTRL+click on the name of the server).

Name is required and, at least, one server must be selected. Once the form is filled, click Ok to create the environment, or Cancel to leave the dialog, with no changes.

Edit an environment

An existing environment can be edited by clicking on the Edit option of the popup menu of the environment. The form is almost identical to the New Environment one, but the inputs are filled with the current parameters and there is one additional button to clear the master node of the environment.

In case you wish to clear the master node, click Clear Master Node. Otherwise, once the modifications are done, click Ok to modify the environment, or Cancel to leave the environment with no changes.

Synchronize an environment

Once you have an environment defined and its master node configured, you can synchronize that environment by right clicking on the environment name and selecting the option Synchronize Environment.

The synchronization operation consists of exporting the metadata of the master node to a single file for re-creating the same metadata in the rest of the servers of the environment.

The steps performed by the synchronization process are:

  1. Exports master node metadata: exports the metadata of the master node to a single file, which will contain the VQL statements to recreate the metadata.

An exception at this point will abort the whole process and an error icon will appear next to the master node name.

  1. For each server in the environment, except the master one:

  1. Saves a backup of the server metadata so that in case any exception arises during the synchronization of this server, previous status can be recovered.

If an exception occurs during this step, the synchronization of this server will stop although, the other servers will not be affected and the whole process will go on. In this case the progress bar will turn red to let you know there has been an exception.

  1. Synchronizes the environment: imports the master node metadata into this server. If an exception occurs, the backup from step a) will be imported and the process will go on with the next server. In this case the progress bar will turn red.

Below there is an image of the synchronization process once finished:

The window above contains three main areas:

  • Progress bar at the top

  • List of tasks on the left

  • Log window on the right

During the execution of the synchronization process there can be both exceptions and errors. There is an important difference between them.

If an exception occurs, the progress bar will turn red and the task throwing the exception will be aborted whereas, if there is an error, the task execution will go on and a warning icon will appear on the left of the window so that you know at what point the error happened and you can go to that part of the log in the window.

An error can be just a problem with the execution of any VQL sentence contained in the master node export when imported into a server whereas an exception may be a connection problem to a server node.

Progress bar

The progress bar at the top of the synchronization window will let you know when the process finishes. If any exceptions arise, the bar will turn red.

List of tasks

All the different tasks of the current synchronization process will be displayed here. Each task contains an icon at the right that lets you know its status:

Waiting status. The task is waiting to be executed

Processing. The task is being executed

Ok. The task has finished correctly

Warning. The task has finished but it has errors. You should check the log of the task

Error. The task has not finished due to an exception. You should check the log of the task. In this case, the progress bar will turn red as a task execution has been aborted

Log window

This window displays the logging information of the tasks performed. Any error messages will be displayed in red so that they can be easily located. Note that every task starts with a bold message, i.e. “Synchronizing serverA with master...”, “Saving serverB backup...”, etc.

TROUBLESHOOTING

Symptom

Denodo Replicator tries to connect to a VDP server and shows the following error: “sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target”

Resolution

VDP Server has SSL enabled, so it is necessary to specify the location of the trust store in the launcher script of the Replicator.

In order to enable SSL connections between VDP Server and the Replicator see section Enable SSL connections.

Symptom

Denodo Replicator tries to make ping in a server and shows the following error:

Resolution

VDP Server and Denodo Replicator are not compatible. You should update the version of VDP server or the version of Denodo Replicator.