Creating Clusters

Note

Only global administrators and Solution Manager administrators can create clusters. More information is available in the Authorization section.

Solution Manager administrators have several ways to create clusters:

  1. Click on the environment (environment_node) and then, click New cluster on the emerging menu.

    Create new cluster from environment

    Create new cluster from environment

  2. When you create an environment, the Solution Manager Administration Tool will suggest you to create a new cluster. Accept the suggestion.

    Suggestion for creating a cluster

    Suggestion for creating a cluster

Note

Only one automated cluster can be created in each environment.


Global Configuration

Whatever path you follow, the following dialog will open to create a new cluster:

Dialog to create a cluster

Dialog to create a cluster

A cluster can be composed of three different types of servers: Virtual DataPort, Data Catalog and Scheduler. Each of them will be configured in its own tab, but common properties are configured in the global configuration tab. This dialog considers the following fields:

  • Name: Descriptive name for the cluster.

  • Description: Extensive description about the cluster. This field is optional.

  • Key Pair: The key pair used to connect to the EC2 instances that are part of the cluster, you must have access to the private key to connect to the instances. Please check AWS key pairs for further details.

    Denodo Solution Manager does not create key pairs, you have to manually create a key pair using AWS console.

  • Subnet: Subnet to use for this cluster. You can select an existing subnet or create a new one providing the CIDR block of the subnet. This is the subnet that will be used for the resources. The instances will be launched with a private or a public IP address according to the subnet auto-assign public IP setting. Check AWS VPC subnets for further details.

  • TLS Configuration: Cluster SSL/TLS configuration.

    Dialog to configure TLS

    Dialog to configure TLS

    You can enable TLS by providing the needed certificates in one of the following formats:

    • Java KeyStore: You have to provide the JKS format file and the public certificate in CER format. You can optionally provide a chain of certificates to trust in CER format.

    • PEM: You have to provide both the private key file and the public certificate in PEM format. You can optionally provide a chain of certificates to trust in PEM format.

    • PKCS #12: You have to provide the PKCS #12 bundle containing the private and public keys and its password.

    Note

    A KeyStore will be created to store the certificates if you have not chosen to provide it directly.

    You always have to provide:

    • KeyStore password: This password will be used in the KeyStore storing the private certificates.

    • TrustStore password: This password will be used in the TrustStore storing the public certificates.

    Note

    By default the TrustStore password used in the JVM is “changeit”.

    When configuring TLS, new AMIs containing the certificates will be created from the AMIs you have chosen, replacing them. Neither the certificates nor the passwords can be retrieved editing the cluster, if you ever replace the AMI manually, you will have to configure TLS again.

  • S3 logging enabled: Use S3 to store logs from all cluster instances.

  • Profile for S3 bucket: this is the profile that will be used for the S3 bucket containing the logs of the cluster. This value overrides the global one defined in Automated Cloud Mode (AWS).

  • S3 base location URL: this is the URL of the bucket and folder where the logs of the cluster will be saved. For instance s3://my-bucket/denodo-logs. This value overrides the global one defined in Automated Cloud Mode (AWS).

  • Design Studio URL: If the Design Studio is not running in the default URL (for instance because it is running behind a load balancer), then indicate here the URL that should be used to access it from the “My Applications” area.

Servers Configuration

Each group of servers has the following specific configuration:

Dialog to create a cluster (Instances)

Dialog to create a cluster (Instances)

  • Enabled: Whether the cluster will contain instances of this type of server or not. This is not shown for Virtual DataPort as a cluster must always contains at least 1 instance of such server type.

  • Number of instances: Number of instances of the server type that the cluster will have.

  • Instance Type: Types of the EC2 instance that will be created, each type fits different use cases. Please check AWS instance types for further details. Not all instances types are supported, only supported types are listed.

Load Balancing & Auto Scaling

Each group of servers will have a network load balancer that will serve as the single point of contact for the instances. Please check AWS network load balancers for further details.

Dialog to create a cluster (Load Balancing & Auto Scaling)

Dialog to create a cluster (Load Balancing & Auto Scaling)

  • Load Balancer Name: The name of the load balancer that will be created. An autogenerated one appears but you can set your own name.

  • Load Balancer DNS: Once correctly created, the DNS name assigned by Amazon to the load balancer. You will use this as the entry point for the group of servers.

  • Internet Facing Load Balancer: Whether the balancer will be reachable from the internet or not.

    When this option is enabled, Solution Manager will configure a default internet gateway for the VPC if needed.

  • Launch instances in Auto Scaling Group: If enabled the instances will be launched in an auto scaling group, this ensures you have the correct number of instance available at all moment. Please check AWS auto scaling for further details.

    This is not supported for Scheduler servers.

  • Auto Scaling Group Name: The name of the auto scaling group that will be created. An autogenerated one appears but you can set your own name.

Important

If you change the value of the Launch instances in Auto Scaling Group option once the cluster is already created, make sure to recreate the cluster without selecting the Minimizing downtime option.

EBS Storage

This collapsible lets you configure the servers storage.

Dialog to create a cluster (EBS Storage)

Dialog to create a cluster (EBS Storage)

  • Volume Size (GiB): Size of the EBS volume for each instance in gibibytes.

  • EBS Volume Type: AWS provides different volume types that differ in performance characteristics so you can choose the most appropriate for your use case. Please check AWS EBS types for further details.

    When the chosen volume supports it an IOPS field will let you specify the desired value.

  • EBS Optimized: Enable EC2 instances to fully use the IOPS provisioned on an EBS volume. Please check AWS EBS optimized for further details.

Advanced Options

This collapsible contains advanced options for the servers.

Dialog to create a cluster (Advanced Options)

Dialog to create a cluster (Advanced Options)

  • AMI: Amazon Machine Instance (AMI) that will be used as “template” to launch this AWS EC2 instance.

    You can enter either select an AMI provided by Denodo or enter the ID of your own AMI (Custom).

    When selecting Custom, you cannot use any AMI; you have to create one following the instructions of the appendix Creating A Custom AMI for the Denodo Platform.

    To obtain the list of AMIs provided by Denodo, the Solution Manager connects to the Denodo Support Site using the Denodo Support Access Keys you entered on the page Configuration > Automated mode > AWS.

    If you leave this empty, this instance will be launched with the AMI you entered in the box Default AMI of the page Configuration > Automated mode > AWS.

  • JVM Options: JVM options used in the Denodo Platform of each server.

  • Security Groups: A security group acts as a virtual firewall that controls the traffic for the instances. You can choose to create a new security group or choose from the existing ones. Please check AWS security groups for further details.

    If you choose to create a new security group you can specify a custom name instead of the autogenerated one.

  • Instance Name Prefix: All the instances in this cluster will be assigned names beginning with this prefix plus a random part.