Auto Scaling Guide¶
This guide explains how to identify the Auto Scaling groups (ASG) associated with your Execution Plane services and details an example of their configuration in the Solution Manager.
How to deploy a cluster with Auto Scaling¶
There are two modes in which the Agora Execution Plane servers can be deployed using the Solution Manager: automated, using predefined templates or manually, through the Manage Environments option. When deploying with PoC and Production templates, Agora automatically creates an ASG for Virtual DataPort and the Data Catalog. When deploying manually, the Launch instances in Auto Scaling Group check box can be selected in the Load Balancing & Auto Scaling section.
Having elected to deploy the Agora server instances in an ASG, an appropriate auto scaling strategy can be chosen:
Basic: In this case, the Solution Manager will create and manage an auto scaling group defined with a fixed number of instances. This strategy guarantees a new instance will be automatically created and added to the group should an instance go down.
Custom Auto Scaling Group: The default option. In this case, the Solution Manager creates a basic auto scaling group, which can be configured further to meet specific user requirements in the AWS console.

How to Identify the Name of the Auto Scaling Group¶
In the overview section of the Solution Manager, the cluster summary can be viewed by selecting the button.
The summary includes tabs for the Virtual DataPort (VDP) and Data Catalog displaying server details including the name of the assigned ASG, as in the example below.

Example of Automatic Scaling Configuration¶
The following two examples (A. and B.) describe the steps required to configure automatic scaling strategies with the following objectives:
To stop all instances on weekends.
Increase capacity dynamically during peak workloads. Ensure the combined CPU usage for the instances under the Virtual DataPort ASG should be approximately 60%, and the number of active instances should be reduced on weekends.
Auto Scaling groups can be configured in the EC2 Auto Scaling Groups section of the AWS Console. The Virtual DataPort ASG’s name (identified as described in the previous section), can be used to locate and edit its details in the console.
There are different types of scaling policies available in AWS. The AWS documentation provides more detailed explanation of the configuration of automatic scaling policies however, the steps for these examples are summarized below.
A. Stopping all instances on weekends¶
To stop all instances on weekends when no processing is required, we configure a scheduled action to stop the Virtual Data Port (VDP) instances and another to restart them at the appropriate time.
Create a scheduled action to stop the instances
To create a scheduled action to stop the VDP instances on weekends:
Go to automatic scaling.
Create a scheduled action with the following properties:
Set Name to Stop VDP instances on weekends.
Set Desired capacity, Min and Max to 0.
Set Recurrence to Cron and set the appropriate schedule, for instance to stop the servers at 20:00 on Fridays, the value would be 0 20 * * Fri.
Set the specific start time and end time (optionally) as necessary (to activate the policy only on that period).
Select create

Create a scheduled action to restart instances
To create a scheduled action to restart the VDP instances on Monday mornings:
Go to automatic scaling.
Create a scheduled action with the following properties:
Set Name to Start VDP instances on business days.
Set Desired capacity, Min and Max to 1.
Set Recurrence to Cron and set the appropriate schedule, for instance to start the servers at 8:00 on Mondays, the value would be 0 8 * * Mon.
Set the specific start time and end time (optionally) as necessary (to activate the policy only on that period).
Hit create.

B. Increasing capacity dynamically during peak workloads¶
In this example, a dynamic scaling policy is needed to ensure the 60% CPU usage target is met, in addition to two scheduled actions to match capacity to lower weekend processing requirements.
Configure the dynamic scaling policy
To configure a dynamic scaling policy to ensure 60% Average CPU usage, edit the details of the ASG in the EC2 Auto Scaling Groups section of the AWS Console as follows:
Go to automatic scaling.
Create a dynamic scaling policy with the following properties:
Set Policy type to Target tracking scaling.
Set Metric type to Average CPU utilization.
Set Target value to 60.
Select create.

Create the first scheduled action to lower resources
To configure a reduced number of instances during weekends, create a scheduled action to lower the number of instances every Friday at 22:00:
Create a scheduled action.
Set Desired capacity and Min to 3, and Max to 5.
Set Recurrence to Every week.
Choose your time zone.
Set start date on the next Friday of your calendar.
Set the hour to 22:00.

Create the second scheduled action to restore the resource levels
To restore the parameters for the rest of the week, create another scheduled action as follows:
Create another scheduled action.
Set Desired to 6, Min to 4 and Max to 8.
Set Recurrence to Every week.
Choose your time zone.
Set start date on the next Monday of your calendar.
Set the hour to 06:00.
Dynamic scaling policies will always be enabled. This type of policy dynamically modifies the number of desired instances of the ASG within the bounds of the min and max specified. Therefore, when a dynamic scaling policy is used, any scheduled actions need only amend the max and min server instances.
In this example, the scheduled action lowers the number of instances every Friday at 22:00. The dynamic scaling policy manages the number of active instances in order to maintain average CPU Usage at 60%.