The Denodo Platform has been designed to remain available 24 hours a day, 365 days a year. A standalone Denodo node has its own mechanisms to provide the highest possible availability (limiting the concurrent requests, establishing a memory usage threshold, etc.) but there are also external factors like network failures or general hardware errors that make a single node deployment not enough for a system with high availability requirements.
A properly designed High-Availability configuration will offer no single-point-of-failure. This will require at least two Denodo Platform nodes deployed in an active-active or active-passive configuration:
Specifically for AWS, AWS Elastic Load Balancing is a load balancer which distributes incoming application or network traffic across multiple targets, such as Amazon EC2 instances, containers, and IP addresses, in multiple Availability Zones.
It also allows to configure health checks, which monitor the health of the computing resources, so that the load balancer sends requests only to the healthy ones.
There are different types of load balancers:
Specifically for AWS, Amazon Route 53 service is a Domain Name System (DNS) web service which allows configure a failover with one Primary component (for example, the “Active” Elastic Load Balancer) and one Secondary resource (the “Passive” Elastic Load Balancer).
This guide will list the steps needed in each server for configuring an active-active high availability architecture in AWS.
Note that using Solution Manager Automated Cloud Mode, all these steps can be automated, that is the Solution Manager creates a Network Load Balancer and registers targets for the corresponding ports while deploying Virtual DataPort nodes in AWS. You can refer to Solution Manager Automated Cloud Mode Quick Start Guide for more information
Let’s see a working example of a configuration of a Denodo 8.0 Cluster (2 Virtual DataPort nodes). The idea is to build this architecture in AWS:
Basically it consists of 2 Denodo instances and an Elastic Load Balancer (ELB):
This guide will explain the steps done for configuring the NLB in this environment. The NLB could be used also for autoscaling (see Configuring Autoscaling of Denodo in AWS), but this document will focus on creating the NLB in a static way.
The first thing to do is to start the Denodo instances in Amazon EC2:
Before launching the Denodo Platform 8.0 servers, connect to the instances one-by-one and ensure they are configured in this way:
Note: All these ports can be edited from the <DENODO_HOME>/conf/vdp/VDBConfiguration.properties file. After modifying this file , execute the script <DENODO_HOME>/bin/regenerateFiles.bat|.sh and restart the Virtual DataPort server for the changes to take effect.
Before creating the NLB, creation of some Target Groups is needed.
A target group is used to route requests to one or more registered targets (nodes in the cluster). A Load Balancer rule needs to be associated with a target group (for example, connections to port 9999 have to go to this group of EC2 instances). We recommend having the groups created before creating the load balancer.
Basically we will need a group for each port opened in the load balancer. In our case, we will need one group for port 9999, another for port 9996, etc.
Follow these steps:
With this configuration, the balancing is done at connection level, so all the requests sent to Denodo from the same connection will reach the same Denodo server.
The procedure to create the NLB is accomplished by:
Note for Denodo 7.0, the 9997 port should also be configured.
Now, as the groups have been assigned to the Load Balancer, we can go to the Target Groups list again and check if the instances are healthy (it means the health-check is working fine):
As previously discussed in Step #1, in all the Denodo instances of the cluster need to set the RMI host to the URL of the load balancer.
Go to the Description tab of the Load Balancer and copy the DNS name.
Connect to each Denodo instance, stop the Denodo servers and edit the %DENODO_HOME%/conf/vdp/VDBConfiguration.properties file.
Set the DNS name as the value for the com.denodo.vdb.vdbinterface.server.VDBManagerImpl.registryURL property.
For monitoring, if SSL is going to be enabled add the property com.denodo.vdb.vdbinterface.server.VDBManagerImpl.jmx.port. This port should be:
NOTE: This property was added in the Denodo 8 20210715 update and the Denodo 7 20201116 update.
After these changes, execute the script <DENODO_HOME>/bin/regenerateFiles.bat/.sh and start again the Denodo services.
To verify that the configuration is correct try to connect from an external VDP Administration Tool to the URL of the load balancer.
NOTE: You may need to edit the /etc/hosts file of your Denodo instances to add an entry mapping the Load Balancer DNS name to the IP address of the instance.
Enabling TLS is a matter of following the steps to enable SSL/TLS in Denodo Platform servers. No further configuration is needed in the Network Load Balancer for SSL unless you are using HTTPS and the port changes to 9443 which should be added to the NLB configuration.
This section applies for Denodo 7.0 and Denodo 8.0 as well.
The High Availability Step-by-step section listed all the things that have to be configured to set-up a cluster of Denodo servers using a Network Load Balancer. All the nodes configured in the Target Group are available so the NLB will route user connections to all those nodes.
In some cases, an Active-Passive configuration is needed, for example, for Disaster recovery scenarios with a cluster of Active Denodo servers and another cluster as Passive. In the case that the Active cluster is down, all the connections to the cluster should be routed to the Passive one (which will be converted to Active at that time).
Let’s see a working example of an Active-passive configuration of a Denodo 8.0 Cluster. The idea is to build this architecture in AWS:
Basically it consists of several Denodo instances in Amazon EC2, Elastic Load Balancers on top of each Denodo cluster and a new player Amazon Route 53:
Refer to the High Availability section for the detailed steps to build that.
We have to configure two load balancers: one for the Active cluster and another one for the Passive cluster.
From the AWS console, go to the Route 53 service and go to Hosted zones (this document assumes that the domain is already configured in AWS):
Select your Route 53 domain to see the current Record Sets.
To create an Active-passive (or Disaster Recovery) configuration create a new Record Set associated with a Failover Routing Policy. Follow the steps bellow:
In some non-standard configurations (for example, when single Denodo servers have to be configured as Active-Passive) we can avoid using Load Balancers so the Architecture is simpler:
For this specific case the configuration is quite similar but some changes are needed to the Record Sets and Health Checks. See the steps bellow:
That way if the Active node is down, the Health check will return that the record is Unhealthy, so Route 53 will switch the DNS to the Passive instance, so new connections will be routed to the healthy server.
The Web Applications of Denodo Platform 8.0 should be configured with Session Stickiness in order to be able to route the requests from the same user to the same server, where the session is kept. In order to ensure that the session is not lost, you would need to configure the load balancer to use sticky sessions.
Steps to configure the Session Stickiness
For more information see Sticky Sessions.