Lookout

Expert trails guide Denodo users through all the relevant materials related to a specific topic, including official docs, KB articles, training, Professional Services offerings, and more. The main goal is to give users a single place with references to all the information that they need to become a Denodo expert on any specific topic.

Installing the Denodo Platform and its modules is the first step before starting Data Virtualization. This expert trail provides all the necessary steps to install and maintain the Denodo Platform and its modules including managing updates, deployments, and backup and recovery processes.

The Hike

Stage 1: Installing the Solution Manager

The Denodo setup has two parts:

• Solution Manager: Denodo Solution Manager is a tool which can be used to manage Denodo Platform Licenses, manage large Denodo Platform deployments and take into account requirements of high availability and load balancing.
• Denodo Platform: The Denodo Platform is a global solution for the real-time integration of heterogeneous and dispersed, structured and unstructured data sources.

Before doing the installation of the Solution Manager and Denodo Platform, it is required to define the servers that are going to be included in the organization’s infrastructure. Information about the process of this is listed in the Expert Trail: Deployment Topology.

The first step before installing the Solution Manager is to check out the supported operating systems which includes Hardware and Software requirements.

After reviewing the installation requirements, the software can be downloaded from Denodo Support Site. More detailed information is explained in the Knowledge Base article: Denodo Solution Manager: First Steps.

• Installing Denodo platform and Solution Manager can be done via two ways: Graphical installation and Headless installation.
• After installing the software, it will be necessary to check if the relevant ports needed by the servers to run are open through the firewall for communication. For example, Licence Manager by default runs on port 10091 and is needed by Denodo platform servers (Virtual DataPort server, Scheduler server etc.) to obtain the license. The document Default ports used by Solution Manager modules provides more information. 
• Once the software has been installed and the relevant ports have been opened, it is necessary to install the license file. The section Install the license of Solution Manager Installation Guide explains the steps to install the global license file.
• For complete information on Solution Manager installation components, please read the Knowledge base article Solution Manager Installation Checklist.

Stage 2: Installing the Denodo Platform

After installing the Solution Manager which acts as a single point of contact for licenses and managing the Denodo servers, it is time to install the Denodo Platform. After reviewing the Hardware and Software requirements the software can be downloaded from the Denodo Support site for the intended operating system.

• After installing the Denodo platform, it is necessary to make sure that the ports needed by the Denodo platform servers are available and open through the firewall. The document Ports needed by the VDP server explains more about the same.
• To start the Denodo servers, they need to request the license from the license manager server (which is a part of Solution Manager installation). For this, it is required to:

• create Environments, Clusters and servers for the corresponding Denodo platform servers in Solution Manager.
• register the Denodo servers in the License Manager 

• The KB article Denodo Platform Installation checklist explains about the other tasks which can be done after installing the platform and starting the servers.
• After the installation, the recommendation is to follow the KB Article Denodo Admin and Development best practices for some of the best practices to configure Denodo platform.

Stage 3: Installing the Latest Update

Once the Solution Manager and Denodo platform is installed, it is important to install the latest update release of both the installations to take advantage of latest features in the product and also to get rid of the latest bug fixes and security vulnerabilities. Updates can be downloaded from the Denodo Support Site navigating to Downloads -> Updates section. Updates can be installed in two ways:

• Installing an update or hotfix graphically
• Installing an update or hotfix through the command line

If you are using AWS or Azure Automated mode, you can use the Solution Manager to install the latest updates. For more information, please follow the section Install an update in a cluster section of Solution Manager Guide.

Note: All Denodo Platform and Solution Manager servers should be stopped before installing an update. In other words, all Denodo related Java processes should be stopped completely before you install an update on the Denodo Platform.  

Stage 4: Deployment Process

Within the Solution Manager, a promotion takes under consideration all the tasks that you simply ought to perform to migrate changes within the servers’ metadata from one environment to another. These changes are spoken to by a modification, which could be a collection of Virtual DataPort components and Scheduler jobs to migrate. The Solution Manager Admin Tool helps you make Promotions. You can select the elements you need to promote by creating revisions from particular Virtual DataPort and Scheduler servers and also Data Catalog servers. For a detailed explanation, please follow the Expert Trail: Development Lifecycle.

Based on the environment, deployments can be of two types:

• Standard Deployment
• Cloud Deployment

Stage 5: Upgrade Process

The process of upgrading to a new major version of the Denodo Platform involves installing the new version in a separate computer - or in a different directory of the same computer - and transferring the metadata and settings from the previous version to the new installation. The same applies to the Solution Manager.

Although the Denodo Platform is backward compatible, switching back between the major versions is not supported. So, once the Denodo Platform metadata is transferred to the new version, rolling back the metadata to the previous one is not allowed.

The upgrade process might slightly differ based on the version you are upgrading from. For a detailed explanation and steps involved in upgrading the Denodo platform, please refer to the Denodo Platform Upgrade Guide.

Stage 6: Backup and Recovery Policies

Backup of the metadata of Virtual DataPort and other servers is important if any adverse situation arises when you need to restore the servers to its previous state, which even includes restoring the servers to a previous update release. Periodic backups of Denodo platform installation directories can be helpful and easy to restore the servers to a previous state, if needed. Apart from that, there are other ways to export the metadata from Virtual DataPort, Scheduler and Data Catalog server.

• Export/Import scripts
• Export the server metadata from VDP Admin Tool
• Export the metadata to a VCS repository

In a Denodo production installation, it is recommended to have a backup server that will be connected to the Load Balancer when some of the main servers are not working for an unexpected reason. For those scenarios, if it’s allowed by your license, you can configure Hot Backups or Cold Backup servers in order to maintain its metadata in sync with the Production servers so they can quickly replace the Production ones if they fail.

Similar to the Virtual DataPort module, other modules of the Denodo Platform should also be backed up.

• Scheduler Server: Scheduler supports import server configuration, projects, plugins and JDBC adapters from a file containing the metadata of a server in a zip file. For more information, please follow the section Import/Export backups section of Scheduler Admin Guide.
• Data Catalog: You can also export and import the Data Catalog’s configuration and metadata (categories, tags, saved queries, etc.) as a collection of JSON files packed in a zip file. For more information, please follow the section Import and export Data Catalog metadata of Data Catalog Guide.
• Solution Manager: Similar to Scheduler server and Data Catalog, solution manager server also provides an option to backup the metadata. For more information, please refer to the Import and Export section of Solution Manager Admin Guide.

Stage 7: Monitoring the Denodo Servers

The Expert Trail: Monitoring provides  information on monitoring.

Stage 8: Resource Management

In an environment with multiple concurrent user sessions that run queries, not all user sessions have the same importance. You may want to give more priority to queries executed by transactional applications, which need an immediate response, than to queries executed to generate a daily report.

The Resource Manager allows you to classify sessions into groups based on the attributes of the session, and to allocate resources to those groups in a way that optimizes resource utilization for your application environment.

The Resource Manager introduces two main concepts:

• Plan: It is a set of restrictions assigned to a user session. For example: limit the number of concurrent queries the same user can run, limit the maximum number of rows of the queries’ results, etc.
• Rule: It assigns a “plan” to the session of a user if a certain condition is met. This condition can use any of the attributes of the session. I.e. user name, access interface used by the application (JDBC, ODBC, etc.), IP address, etc. Please follow the section Available fields to evaluate a rule of VDP Admin Guide.

Exploration

Fill up your backpack with additional gear:

Guided Routes

Denodo Training Courses

Denodo training courses provide expert data virtualization training for data professionals, including administrators, architects, and developers.

If you are interested in “Operations Expert Trail” you should enroll the following course/s:

• Denodo Platform Installation: This course provides in-depth knowledge of all activities required to install Denodo Platform. Highlights of these activities are: Installing and configuring the Denodo Platform and Updating best practices. 
• Denodo Configuration and Maintenance: This course covers the usual operations performed by a Denodo Administrator like creation of virtual databases, user management and auditing, configuration of Denodo servers, etc.

Success Services

Denodo Customer Success Services can help you at the start or any part of your Self-Service Analytics trail. You can find information about the Denodo Success Services offering in:

Success Services

Advisory Sessions

Denodo Customers with active subscriptions have access to request Meet a Technical Advisory sessions.

These are the sessions available related to Operations.

Success Accelerators

In addition to Advisory sessions, Success Services includes Success Accelerators that can help you.

• Operations Quick Start

If you are a Denodo customer, you can reach out to your Customer Success Manager for details about any Guided Route that you need.

Big Hike Prep Check

Let’s see if you are ready to start your big trail. Take this 4-question questionnaire to check your readiness for an enjoyable hike.

Read the questions below, think about the solution and check if you got them right by looking at the solution. Have you become an expert?

1. While upgrading the Denodo platform, can you use the same cache database ?

Click here to check if you got it right

2. You have multiple nodes in a cluster of standard environments. You want to minimize the downtime while deployment. How can you do that?

Click here to check if you got it right

3. You want to automate the backup process in case the deployment does not work. Is it possible to do that in Denodo?

Click here to check if you got it right

4. Can the VDP nodes of the same environment share a metadata database so that the metadata synchronization becomes easy?

Click here to check if you got it right