The Denodo Platform has been designed to remain available 24 hours a day, 365 days a year. A standalone Denodo node has its own mechanisms to provide the highest possible availability (limiting the concurrent requests, establishing a memory usage threshold, etc.) but there are also external factors like network failures or general hardware errors that make a single node deployment not enough for a system with high availability requirements.
A properly designed High-Availability configuration will offer no single-point-of-failure. This will require at least two Denodo Platform nodes deployed in an active-active or active-passive configuration:
Specifically for AWS, AWS Elastic Load Balancing is a load balancer which distributes incoming application or network traffic across multiple targets, such as Amazon EC2 instances, containers, and IP addresses, in multiple Availability Zones.
It also allows to configure health checks, which monitor the health of the computing resources, so that the load balancer sends requests only to the healthy ones.
There are different types of load balancers:
Specifically for AWS, Amazon Route 53 service is a Domain Name System (DNS) web service which allows configure a failover with one Primary component (for example, the “Active” Elastic Load Balancer) and one Secondary resource (the “Passive” Elastic Load Balancer).
This guide will list the steps needed in each server for configuring an active-active high availability architecture in AWS.
Let’s see a working example of a configuration of a Denodo 7.0 Cluster (2 Virtual DataPort nodes). The idea is to build this architecture in AWS:
Basically it consists of 2 Denodo instances and an Elastic Load Balancer (ELB):
This guide will explain the steps done for configuring the ELB in this environment. The ELB could be used also for autoscaling (see Configuring Autoscaling of Denodo in AWS), but this document will focus on creating the ELB in a static way.
The first thing to do is to start the Denodo instances in Amazon EC2:
Before launching the Denodo Platform 7.0 servers, connect to the instances one-by-one and ensure they are configured in this way:
Note: All these ports can be edited from the <DENODO_HOME>/conf/vdp/VDBConfiguration.properties file. After modifying this file , execute the script <DENODO_HOME>/bin/regenerateFiles.bat/.sh and restart the Virtual DataPort server for the changes to take effect.
Before creating the NLB, creation of some Target Groups is needed.
A target group is used to route requests to one or more registered targets (nodes in the cluster). A Load Balancer rule needs to be associated with a target group (for example, connections to port 9999 have to go to this group of EC2 instances). We recommend having the groups created before creating the load balancer.
Basically we will need a group for each port opened in the load balancer. In our case, we will need one group for port 9999, another for port 9996, etc.
Follow these steps:
Note: The two instances have to be added to all groups but the Auxiliary port groups. These groups will contain only the specific instance configured with that auxiliary port.
Why is this needed?
The auxiliary port (by default 9997) is a RMI Factory port. When a client is trying to connect to Virtual DataPort it connects to the port 9999, then the server responds saying the client that it has to connect to the host configured as RMIHost and the RMI Factory port 9997 for communications with the server.
For that reason, we have to set the IP (or DNS) of the Load Balancer as RMI host in our Denodo instances, and use a different auxiliary port (one per server).
With this configuration, the balancing is done at connection level, so all the requests sent to Denodo from the same connection will reach the same Denodo server.
The procedure to create the NLB is accomplished by:
Now, as the groups have been assigned to the Load Balancer, we can go to the Target Groups list again and check if the instances are healthy (it means the health-check is working fine):
As previously discussed in Step #1, in all the Denodo instances of the cluster need to set the RMI host to the URL of the load balancer.
Go to the Description tab of the Load Balancer and copy the DNS name.
Connect to each Denodo instance, stop the Denodo servers and paste the DNS name into the %DENODO_HOME%/conf/vdp/VDBConfiguration.properties file:
After that execute the script <DENODO_HOME>/bin/regenerateFiles.bat/.sh and to start again the Denodo services.
To verify that the configuration is correct try to connect from an external VDP Administration Tool to the URL of the load balancer
Enabling TLS is a matter of following the steps to enable SSL/TLS in Denodo Platform servers. No further configuration is needed in the Network Load Balancer for SSL unless you are using HTTPS and the port changes to 9443 which should be added to the NLB configuration.
The High Availability Step-by-step section listed all the things that have to be configured to set-up a cluster of Denodo servers using a Network Load Balancer. All the nodes configured in the Target Group are available so the NLB will route user connections to all those nodes.
In some cases, an Active-Passive configuration is needed, for example, for Disaster recovery scenarios with a cluster of Active Denodo servers and another cluster as Passive. In the case that the Active cluster is down, all the connections to the cluster should be routed to the Passive one (which will be converted to Active at that time).
Let’s see a working example of an Active-passive configuration of a Denodo 7.0 Cluster. The idea is to build this architecture in AWS:
Basically it consists of several Denodo instances in Amazon EC2, Elastic Load Balancers on top of each Denodo cluster and a new player Amazon Route 53:
Refer to the High Availability section for the detailed steps to build that.
We have to configure two load balancers: one for the Active cluster and another one for the Passive cluster.
From the AWS console, go to the Route 53 service and go to Hosted zones (this document assumes that the domain is already configured in AWS):
Select your Route 53 domain to see the current Record Sets.
To create an Active-passive (or Disaster Recovery) configuration create a new Record Set associated with a Failover Routing Policy. Follow the steps bellow:
In some non-standard configurations (for example, when single Denodo servers have to be configured as Active-Passive) we can avoid using Load Balancers so the Architecture is simpler:
For this specific case the configuration is quite similar but some changes are needed to the Record Sets and Health Checks. See the steps bellow:
That way if the Active node is down, the Health check will return that the record is Unhealthy, so Route 53 will switch the DNS to the Passive instance, so new connections will be routed to the healthy server.