Introduction

In cloud environments (like AWS), a typical use case is to use auto scaling capabilities in order to allow client applications to increase / decrease servers capacity according to a different set of rules (time intervals, CPU, memory usage…). This ability to automatically adjust the capacity of a Denodo cluster in AWS require taking into account several aspects:

Sine Denodo Platform 8.0 autoscaling is fully managed by the Solution Manager Automated Cloud Mode. This feature provides automatic deployment of Denodo Platform components, including autoscaling.

However, sometimes it might be necessary to have a finer control over the rules that guide the autoscaling to ensure that resources are provided at the right time. To ensure this we can configure autoscaling based on a Denodo custom metrics instead of the default ones.

Auto Scaling based on a Denodo custom metric

CloudWatch allows you to create alarms based on metrics and define actions to be taken when the alarm changes the state. One of these available actions is the Amazon EC2 Auto Scaling action.

Auto Scaling groups are created automatically by Solution Manager Automated Cloud Mode but these can be further edited later. Typically a Auto Scaling group should be created specifying the number of instances you want to run. In our example the desired number of instances is 2 and the minimum and maximum number of instances the Auto Scaling group should have at any time is 2 and 3, respectively.

In this scenario, we need to configure an Auto Scaling action in response to a high workload that will increase the number of instances and also an Auto Scaling action in response to a low workload that will decrease the number of instances. But note that regardless of the actions, the Auto Scaling group will maintain the minimum and maximum number of instances without further configuration. This is why the execution of an action is going to fail when it tries to scale out but the number of running instances in the Auto Scaling group is 3 (the maximum) or when it tries to scale in and the number of instances is 2 (the minimum).

For example, you can develop a Denodo custom metric in CloudWatch for monitoring the number of requests initiated in Denodo per server every minute and add an alarm that goes to ALARM state when the number of queries in the last 3 minutes is greater than 100. Take into account that it is not available for anomaly detection alarms therefore you have to specify a static threshold in order to add this kind of action.

See the ‘Monitoring Denodo Metrics’ and ‘Adding an alarm’ sections of the Monitoring Denodo with Amazon CloudWatch document for more detailed information about creating Denodo custom metrics and adding alarms based on metrics.

When you are creating or editing an alarm, the second step allows you to configure the actions. Since you are going to use our alarm as an indicator of high workload, we want to launch a new instance in our Auto Scaling group, Test-ASG for this document, whenever the alarm enters in the ALARM state. Similarly, you have to define as an indicator of low workload an alarm in which the ALARM state will enter when the number of requests initiated in Denodo per server every minute in the last 3 minutes is lower than 40.

On the other hand, a scaling policy associated with the Auto Scaling group is necessary to define an Auto Scaling action. Therefore, you must create the alarm, called my_alarm, without any action.

To create a scaling policy you should:

1. Open your EC2 console and choose Auto Scaling Groups, on the navigation pane.
2. Select your group, Test-ASG in our example, and go to Scaling Policies tab.
3. Click on the “Add Policy” button and select Create a scaling policy with steps.
4. Give a name to the policy and select the alarm created to warn of a high workload in Executing policy when field. In the Take the action section choose Add 1 instances to increase the capacity of the Auto Scaling group. You should leave the by default configuration for the upper bound for this step adjustment, which is the alarm threshold, and the lower bound is null (negative infinity).

To create a scaling policy in response to the need of decreasing the number of instances, select the alarm created as an indicator of low workload in the Execution policy when field and in  the Take the action section choose Remove 1 instances.

5. Click on the “Create” button.

Now you can add the Auto Scaling action to my_alarm alarm in CloudWatch editing it.  

1. Click on the “Add Auto Scaling action” button in the Configure actions step.

2. Select:

• in Alarm as the state that will trigger the action.
• EC2 Auto Scaling group as resource type and select your group, Test-ASG.
• Choose the policy, my_policy, in the Take the following action section.

3. Click on the “Update alarm” button.

Note that after an alarm invokes an Amazon EC2 Auto Scaling action due to a change in state, the alarm continues to invoke the action for every period that the alarm remains in the new state. Nevertheless, in the history of the alarm the action appears only once, between the two state updates: