Deploying Denodo in Amazon Elastic Container Service (ECS) using Fargate
You can translate the document:
Amazon Elastic Container Service (Amazon ECS) is a container management service that makes it easy to run, stop, and manage Docker containers on a cluster, letting you launch and stop container-based applications with simple API calls.
To use Amazon ECS you need a cluster that can be created within a new or an existing VPC. Then, after a cluster is up and running, the task definitions and services created will specify which Docker images run across the clusters. The container images are stored in and pulled from container registries, which may exist within the AWS infrastructure (Amazon ECR) or outside.
The services and tasks launched can use one of the following Amazon ECS launch types:
- AWS Fargate: The Fargate launch type allows running containerized applications without the need for provisioning or managing the backend infrastructure (Amazon EC2 instances). Just registering the task definition will make AWS Fargate launch the container.
- Amazon EC2: The EC2 launch type allows running containerized applications on a cluster of custom Amazon EC2 instances.
In this article, we will show how to run Docker images of Denodo on an Amazon ECS cluster using Fargate, for images stored in Amazon ECR
AWS Command Line Interface
The AWS Command Line Interface (AWS CLI) is an open-source tool that can be used for administering the AWS resources and we will use it extensively in the article, we will assume that it is installed and up to date as a prerequisite for executing the commands presented below.
AWS previously offered the Amazon ECS CLI (ecs-cli), a high-level tool for managing clusters and tasks from a local environment using Docker Compose, but that tool has since been archived. Its intended successor, the AWS Copilot CLI, has also reached end-of-support (June 12, 2026). For scripting Amazon ECS deployments today, use the AWS CLI directly, as in this guide, or an infrastructure-as-code tool such as AWS CloudFormation, the AWS CDK, or Terraform.
Although it is also possible to launch our first cluster using the Amazon ECS first-run wizard, in this article we have chosen to use the AWS CLI because usually, the commands are self-explanatory and also because it is possible to use them and test them easily in any custom AWS deployment.
Amazon ECS using Fargate
AWS Fargate is a compute engine for containers that can be used with Amazon ECS to run containers without having to manage servers or clusters of Amazon EC2 instances.
When running the ECS tasks and services with the Fargate launch type, Fargate will read the specifications from the configuration files and the containerized applications will be launched following the CPU and memory requirements specified. Notice that each Fargate task has its own isolation boundary and does not share the underlying kernel, CPU resources, memory resources, or elastic network interface with another task.
List of Prerequisites
Fargate Task-Level Requirements
Because a Fargate task does not run on infrastructure that you manage directly, AWS enforces the following task-level requirements when you define and run Denodo on Fargate:
- Network mode: Fargate tasks always use the awsvpc network mode. Set networkMode to "awsvpc" in the task definition; each task then receives its own elastic network interface (ENI) and does not share the kernel, CPU, memory, or network interface with any other task.
- CPU and memory: task-level cpu and memory must use one of the supported Fargate combinations, ranging from 256 CPU units (.25 vCPU) with 512 MiB, up to 16384 CPU units (16 vCPU) with up to 120 GiB of memory. See the AWS task size reference listed in the References section for the complete table.
- Ephemeral storage: every Fargate task automatically receives 20 GiB of ephemeral, encrypted storage at no extra cost on Linux platform version 1.4.0 or later. If the Denodo image or its working data needs more space, set ephemeralStorage.sizeInGiB in the task definition, up to a maximum of 200 GiB.
- Platform version: use the LATEST Fargate platform version unless there is a specific reason to pin an older one, so the task always runs on the most current platform version AWS supports.
- CPU architecture: Fargate supports both X86_64 and ARM64 (Graviton) Linux tasks, declared through the runtimePlatform block of the task definition (cpuArchitecture and operatingSystemFamily). The Denodo container images referenced in this guide are built for X86_64.
This article assumes that there is an environment already configured with the tools required to deploy a Docker image of Denodo in AWS, in other cases, it might be needed to install or configure something else before continuing with the next section. In summary, you need:
- A valid AWS account with enough privileges to build all the AWS elements involved in the deployment of the Denodo image.
- An IAM entity, to avoid accessing AWS directly with the credentials of our AWS account. With the AWS Identity and Access Management (IAM) service, we can have restricted permissions and policies over the AWS resources.
- A task execution IAM role (ecsTaskExecutionRoleARN) that provides permissions to Amazon ECS container agents to use the private registry authentication feature to pull the container image from Amazon ECR. It requires an IAM policy and role for the service to know that the agent belongs to you. Check the “AWS Permissions” section for details on the permissions required for every scenario.
- A Virtual Private Cloud (Amazon VPC) with public and private subnets (a list of subnet ids separated by commas - subnetIdList) for our clusters and a dedicated security group (securityGroupId). We want to expose the Denodo Server to the internet so in this guide we will be using a public subnet and assign a public IP to the ECS task, but you could use a private subnet configured with a NAT waterway with an elastic IP address.
- The latest version of the AWS CLI, because although the AWS Management Console can be used to manage Amazon ECS, the AWS CLI allows you to build scripts that can automate common management tasks in Amazon ECS.
- A locally accessible Docker image of Denodo. You can use the images provided by Denodo in the Denodo Container Registry or download it from the Denodo Support Site. If you prefer to use your custom one, check How to create your own Docker images for Denodo Platform Containers
AWS Permissions
An IAM entity (user, group, or role) must be set up with the privileges required by the AWS CLI for running the statements in this guide.
In this article, we have configured an IAM Role (roleARN) to use ECS and Fargate/EC2 services.
This IAM role must be able to list, push, and pull images in the ECR repository, which is granted with the following AWS managed policy:
- AmazonEC2ContainerRegistryPowerUser.
To grant someone permission to create an Amazon ECR repository with the Amazon ECR CreateRepository API operation, we must include the following action in their policy:
- ecr:CreateRepository
The task execution role must also be able to pull container images from the ECR repository and send container logs to CloudWatch Logs. Amazon ECS provides these permissions through the following AWS managed policy, which should be attached to the ecsTaskExecutionRole:
- AmazonECSTaskExecutionRolePolicy
The IAM role must be able to create and read secrets using the Secrets Manager (only for Solution Manager and Denodo Platform with StandAlone licenses) following the process described in the Amazon ECS documentation.
In addition, although this works automatically with the cluster creation, notice that Amazon ECS uses the service-linked role named AWSServiceRoleForECS to enable Amazon ECS to call AWS APIs on your behalf.
Configure the AWS CLI
To start with the AWS configuration, open a new console and execute the below AWS commands that will do the basic configuration of the AWS CLI.
Configure the AWS access keys for the AWS account by using the following command:
|
$ aws configure |
When entering this command, the AWS CLI prompts for the following information:
- AWS Access Key ID
- AWS Secret Access Key
- Default region name
- Default output format
To read more about the AWS CLI configuration check the AWS CLI user guide.
At this point, you need to decide if you will configure the default AWS CLI profile or if you want to create a custom profile. If you create a custom profile some of the commands in the rest of the guide will require an additional parameter to specify the profile to use (--profile profilename) so for simplicity, we will just configure the default profile in this guide. You can read more about named profiles in the AWS Command Line Interface documentation.
Use these commands to configure the IAM role in the default profile:
|
$ aws configure set role_arn <roleARN> $ aws configure set source_profile default |
The role_arn parameter specifies the Amazon Resource Name (ARN) of an IAM role that you want to use to perform operations with the default profile. It will have this format: arn:aws:iam::<accountNumber>:role/<roleName>
The source_profile parameter identifies what profile to use to find credentials that have permission to assume the IAM role.
Create the Container Registry
Create a new Elastic Container Registry by providing a custom name for the repository:
|
$ aws ecr create-repository --repository-name <denodo-ecr> |
Check that the denodo-ecr repository has been created successfully and obtain its repositoryUri from the output of the create-repository statement. The field repositoryUri will be used later to authenticate the local Docker client.
To obtain the repositoryUri value the following command can be used:
|
$ aws ecr describe-repositories --repository-name <denodo-ecr> --query "repositories[].repositoryUri" |
Authenticate the Docker client to the new registry to push the Denodo image to the repository using the repositoryUri:
|
$ aws ecr get-login-password | docker login --username AWS --password-stdin <repositoryUri> |
Finally, tag the Denodo image and push it to the Elastic Container Registry
|
$ docker tag denodo-platform:latest <repositoryUri> $ docker push <repositoryUri> $ aws ecr list-images --repository-name <denodo-ecr> |
Create the ECS Cluster
To deploy the Denodo image on ECS it is necessary to create a cluster. By default, the AWS accounts have a default cluster. The benefit of using this default cluster is that you don't have to specify the --cluster <cluster-name> option in the subsequent commands. But if you do create your own cluster, then you must specify --cluster <cluster-name> for each command that you intend to use with that cluster.
|
$ aws ecs create-cluster --cluster-name <denodo-ecs-cluster> |
It can take some time for the cluster to be available, but it is possible to check its status with the following command:
|
$ aws ecs describe-clusters --cluster <denodo-ecs-cluster> |
Creating a secret for storing the License file
Persistent Storage with Amazon EFS
Fargate tasks are ephemeral: the built-in local storage described above is removed when a task stops, and it is not shared between tasks. For data that must survive task restarts and redeployments, or that must be shared across multiple Denodo task definitions - for example, the same encryption keystore used by both Solution Manager and VDP, or externalized Denodo logs under /opt/denodo/logs - mount an Amazon EFS file system as a task volume instead of relying on ephemeral storage.
- Create the EFS file system and mount targets in the same VPC and subnets used by the Fargate service, and allow NFS traffic (port 2049) from the task security group to the EFS mount target security group.
- Reference the file system from the task definition with an efsVolumeConfiguration (specifying fileSystemId and, optionally, rootDirectory), then mount it into the container through mountPoints. The Denodo Platform on AWS ECS Troubleshooting Guide includes worked examples for sharing the encryption keystore and for persisting logs this way.
- Amazon EFS volumes for Fargate require platform version 1.4.0 or later, which is the default when platformVersion is set to LATEST.
The Denodo containers include the DENODO_LIC_CONTENT environment variable for scenarios where you can directly provide the License File content to the container. This environment variable will get the content of the value and configure it as a Denodo license in a transparent process. This will be useful for configuring the Solution Manager license.
This article describes a process to create the secret using the AWS CLI but you can follow your usual process to create secrets in your organization.
The process will get the license file referenced in the command as: <license file> , encode it as base64 and create a secret <license_secret> using AWS Secrets Manager. Remember to replace <license file> and <license_secret> with the desired values
- Create the secret by running the following command:
aws secretsmanager create-secret --name <license_secret> --secret-string "$(base64 <license file>)"
- Validate the secret creation by running the following command:
aws secretsmanager get-secret-value --secret-id <license_secret> --query SecretString --output text | base64 -d
Now you can take note of the secret arn to be used in your task definitions in the placeholder: <license_secret_arn>.
NOTE: You can also create the secret by using a Key value as explained in the AWS Secrets Manager documentation. This may require using different identifiers for the secret in the task definition file.
Register an ECS Task
Once the cluster is created, you can start registering the task definitions that will be run as tasks on the new cluster created.
This example uses a simple task definition named denodo-task-definition that makes use of the Denodo container image hosted on the new repository denodo-ecr.
Also, notice that the Denodo Platform requires a valid license to run, which will be obtained from a Denodo License Manager server.
For this process, we need to configure the specific environment variables pointing to the License Manager server.
The task definition can be specified in a JSON file, which improves the readability and the management of the tasks because these definitions can be very large. Below you will find two examples of very simple task definitions, one for configuring the Solution Manager (and License Manager) server using the License and another one for configuring a Denodo Platform that will connect to the Solution Manager server to retrieve the license.
Note: As AWS Fargate is a serverless compute engine, the characteristics of the server that will run the container are not known in advance. To be able to run Denodo with a core based license the number of cores used by the container needs to be limited. To do so, the “cpu” option can be used in the Denodo task definition configuration files. For more information see Task definition parameters - Amazon Elastic Container Service.
Solution Manager
To deploy a Solution Manager instance, it is mandatory to use a standalone license file. You need to flow the process described in the “Creating a secret for storing the License file” section to create the secret with the license file. Afterwards, we can use the DENODO_LIC_CONTENT environment variable to provide this license content using the Secret value.
Find below an example of denodo-task-definition.json file for the Solution Manager:
|
{
"family": "solution-manager-td", "containerDefinitions": [ { "name": "solution-manager",
"image": "<repositoryUriSolutionManager>:latest", "cpu": 4096, "memory": 8192, "portMappings": [ { "name": "web-port", "containerPort": 19090, "hostPort": 19090, "protocol": "tcp", "appProtocol": "http" }, { "name": "lm-port", "containerPort": 10091, "hostPort": 10091, "protocol": "tcp" }, { "name": "sm-port", "containerPort": 10090, "hostPort": 10090, "protocol": "tcp" } ], "essential": true, "entryPoint": [ "/opt/denodo/tools/container/entrypoint.sh" ], "command": [ "--lmserver", "--smserver", "--smadmin" ], "environment": [ { "name": "DENODO_DEBUG", "value": "true" } ], "environmentFiles": [], "mountPoints": [], "volumesFrom": [], "secrets": [ { "name": "DENODO_LIC_CONTENT", "valueFrom": "<license_secret_arn>" } ], "startTimeout": 40, "workingDirectory": "/opt/denodo", "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "/ecs/solution-manager-td", "mode": "non-blocking", "awslogs-create-group": "true", "max-buffer-size": "25m", "awslogs-region": "us-east-1", "awslogs-stream-prefix": "ecs" }, "secretOptions": [] }, "healthCheck": { "command": [ "CMD-SHELL", "curl -k http://localhost:10090/pingSolutionManager || exit 1" ], "interval": 10, "timeout": 9, "retries": 6, "startPeriod": 40 }, "systemControls": [] } ],
"taskRoleArn": "<ecsTaskExecutionRoleARN>", "executionRoleArn": "<ecsTaskExecutionRoleARN>", "networkMode": "awsvpc", "volumes": [], "placementConstraints": [], "requiresCompatibilities": [ "FARGATE" ], "cpu": "4096", "memory": "8192", "runtimePlatform": { "cpuArchitecture": "X86_64", "operatingSystemFamily": "LINUX" }
}
|
denodo-task-definition.json for the Solution Manager
Denodo Platform with Solution Manager License
As detailed in the Denodo Docker container configuration guide, the container provides a set of environment variables where you can configure several aspects of the environment as a straightforward process.
If there is a Denodo License Manager server running and accessible by the cluster then its connection details can be configured as environment variables for the container.
Important note: For the sake of simplicity, this document details how to configure the variables for retrieving a valid Denodo license, as it is a mandatory requirement for being able to start the Denodo servers. Keep in mind in a real production environment you may need to go through other environment variables for configuring the Denodo 9 keystore to use the same across all your Denodo and Solution Manager installations, or the TLS configuration.
As described here, these are the available environment variables for this purpose:
|
DENODO_LM_PROTO: Scheme or protocol of the License Manager. Default value: http DENODO_SM_PROTO: Scheme or protocol of the Solution Manager. Default value: http DENODO_LM_HOST: Hostname of the License Manager. DENODO_LM_PORT: Port of the License Manager. Uses 10091 as default. DENODO_SM_HOST: Hostname of the Solution Manager. DENODO_SM_PORT: Port of the License Manager. Uses 10090 as default. |
We are sharing here an example of values, but fill those with your own ones matching your Solution Manager installation:
|
DENODO_LM_PROTO: "http" DENODO_SM_PROTO: "http" DENODO_LM_HOST: "solution-manager" DENODO_LM_PORT: "10091" DENODO_SM_HOST: "solution-manager" DENODO_SM_PORT: "10090" |
At this point, you are ready to register the task definition, so take the following template and replace the placeholders DENODO_LM_PROTO,DENODO_LM_HOST,DENODO_LM_PORT, DENODO_SM_PROTO,DENODO_SM_HOST,DENODO_SM_PORT, repositoryUri, and ecsTaskExecutionRoleARN to start a Denodo container in the ECS cluster:
|
{ "containerDefinitions": [ { "name": "denodo-vdp", "image": "<repositoryUri>:latest", "repositoryCredentials": { "credentialsParameter": "credentials" }, "cpu": 4096, "memory": 8192, "portMappings": [ { "name": "denodo-vdp-9999-tcp", "containerPort": 9999, "hostPort": 9999, "protocol": "tcp" }, { "name": "denodo-vdp-9996-tcp", "containerPort": 9996, "hostPort": 9996, "protocol": "tcp" }, { "name": "denodo-vdp-9997-tcp", "containerPort": 9997, "hostPort": 9997, "protocol": "tcp" }, { "name": "denodo-web-port", "containerPort": 9090, "hostPort": 9090, "protocol": "tcp" } ], "essential": true, "entryPoint": [ "/opt/denodo/tools/container/entrypoint.sh" ], "command": [ "--vdpserver" ], "environment": [ { "name": "DENODO_DEBUG", "value": "true" }, { "name": "DENODO_LM_PROTO", "value": "http" }, { "name": "DENODO_LM_HOST", "value": "solution-manager" }, { "name": "DENODO_LM_PORT", "value": "10091" }, { "name": "DENODO_SM_PROTO", "value": "http" }, { "name": "DENODO_SM_HOST", "value": "solution-manager" }, { "name": "DENODO_SM_PORT", "value": "10090" } ], "systemControls": [] } ], "taskRoleArn": "<ecsTaskExecutionRoleARN>", "executionRoleArn": "<ecsTaskExecutionRoleARN>", "family": "denodo-task-definition", "networkMode": "awsvpc", "memory": "8192", "cpu": "4096", "requiresCompatibilities": [ "FARGATE" ], "runtimePlatform": { "cpuArchitecture": "X86_64", "operatingSystemFamily": "LINUX" }
} |
denodo-task-definition.json for the Denodo Platform (VDP), connecting to Solution Manager
Once the task definition is saved to the JSON file denodo-task-definition.json you can create the task definition with the following command:
|
$ aws ecs register-task-definition --cli-input-json file://denodo-task-definition.json |
Then, you can list the task definitions created with the following command:
|
$ aws ecs list-task-definitions |
You can also check the details of a task definition:
|
$ aws ecs describe-task-definition --task-definition <taskDefinitionArn> |
Create an ECS Service
After registering the task, the next step is creating the Amazon ECS service that will run the task on our cluster. In this case, the service will run a single instance of the task registered. Note that the task requires a route to the internet, as we want the Denodo server to be accessed from outside AWS:
|
$ aws ecs create-service \ --service-name <denodo-service> \ --cluster <denodo-ecs-cluster> \ --task-definition denodo-task-definition \ --desired-count 1 \ --launch-type "FARGATE" \ --network-configuration "awsvpcConfiguration={subnets=<subnetIdList>,securityGroups=<securityGroupId>,assignPublicIp=ENABLED}" \ --load-balancers "targetGroupArn=<nlbARN>,containerName=denodo,containerPort=9999" |
The create-service command returns a description of the task definition after it completes its registration. The following command gets a list of services created in the cluster:
|
$ aws ecs list-services --cluster <denodo-ecs-cluster> |
You can get the description of the service running using the service name retrieved earlier to get more information about the task:
|
$ aws ecs describe-services --cluster <denodo-ecs-cluster> --services <denodo-service> |
Connecting to Denodo Platform
Debugging with ECS Exec
Amazon ECS Exec lets you open an interactive shell or run a single command inside a running Fargate task, without opening inbound ports or managing SSH keys, which is useful when troubleshooting Denodo startup or registration issues directly on the container. It requires Fargate platform version 1.4.0 or later.
Enable it when creating (or updating) the service:
|
$ aws ecs create-service \\ ... \\ --enable-execute-command |
The task role (not the task execution role) must allow the SSM Session Manager actions used by ECS Exec (ssmmessages:CreateControlChannel, ssmmessages:CreateDataChannel, ssmmessages:OpenControlChannel, ssmmessages:OpenDataChannel), and the AWS CLI Session Manager plugin must be installed locally. Once enabled, open a shell in the running container with:
|
$ aws ecs execute-command \\ --cluster <denodo-ecs-cluster> \\ --task <taskARN> \\ --container denodo-vdp \\ --interactive \\ --command "/bin/sh" |
Finally, in order to connect to the server from a VDP client, you will need to ensure that the VDP client is able to resolve this hostname to the public IP address of the ECS service, and you can do that by adding an entry in the local hosts configuration file to map that hostname with the public IP address.
You can get the Public IP of the Task in the AWS Management Console or obtain it from the AWS CLI with the following process:
- First, we will start obtaining the Task ARN that is running Denodo.
- Then, with the Task we can get the Elastic Network Interface assigned to it, and
- Finally we can retrieve the Public IP address from the ENI:
|
$ aws ecs list-tasks --cluster <denodo-ecs-cluster> --query "taskArns[*]"
$ aws ecs describe-tasks --cluster <denodo-ecs-cluster> --tasks <taskARN> --query "tasks[0].attachments[0].details[?name=='networkInterfaceId'].value | [0]" --output text
$ aws ec2 describe-network-interfaces --network-interface-ids <eni> --query "NetworkInterfaces[*].Association.PublicIp" |
Then, add the entry with the AWS Public IP address in the local hosts file:
|
# Denodo ECS service <public-ip> <denodo-hostname> |
Now you can open a new Denodo Design Studio and connect to the server using the following url:
http://denodo-hostname:9090/denodo-design-studio
Denodo Design Studio
References
Denodo Platform Container 9 QuickStart Guide
Denodo Docker Container Configuration
How to create your own Docker images for Denodo Platform Containers
Amazon ECS Task Definition Parameters (task size)
Amazon ECS on AWS Fargate - Task Storage (Ephemeral Storage)
Using Amazon ECS Exec for Debugging
Amazon ECS - Amazon EFS Volumes Tutorial
Denodo Platform on AWS ECS Troubleshooting Guide
The information provided in the Denodo Knowledge Base is intended to assist our users in advanced uses of Denodo. Please note that the results from the application of processes and configurations detailed in these documents may vary depending on your specific environment. Use them at your own discretion.
For an official guide of supported features, please refer to the User Manuals. For questions on critical systems or complex environments we recommend you to contact your Denodo Customer Success Manager.
