Agora Quick Start Guide
You can translate the document:
The goal of this guide is to serve as a quick start to the main functionalities in Agora, the Denodo Cloud Service, like creating a new account and instantiating a new subscription. For general documentation and tutorials on how to use Denodo, please refer to the Denodo Community site
Before you start: the architecture of Agora
Agora operates using two different planes: a Control Plane and an Execution Plane. The following diagram represents the architecture and data flows:
- The Control Plane: managed by Denodo in its cloud accounts. It includes all the capabilities related with account and subscription management. Additionally, the Control Plane also includes a modified version of the Denodo Solution Manager adapted to the needs of Agora (which allows you to manage the environments and clusters of your Execution Plane), and hosts the web-based clients (Design Studio, Scheduler Administration Tool and the Monitoring and Diagnostics tool). This part of the Control Plane is deployed in a dedicated isolated instance.
- The Execution Plane: where your data is processed. It includes the VDP and Scheduler servers, and the Data Catalog. The Execution Plane is hosted in your cloud account.
To further clarify the concepts covered in this section, you can watch our detailed video tutorial here: https://youtu.be/rXQE99qUxK8 |
Getting started
If it is your first time using Agora you need to register a new user. In the login page, click on "Register", fill the form and click on the “Register” button to create a new user. If you already have a user, simply enter your username and password and click on “Sign In”.
In order to register a new user, you need to provide basic information like name, last name, email and password. Please keep in mind that this user will be the global administrator of the account for your organization. You will be able to create additional users later on.
Register your organization
Once you have registered and logged in, you need to register your organization. An organization can have multiple subscriptions, which is useful for example if you want to have multiple Denodo deployments in different cloud providers or in different regions, but manage them all under the same account.
For a quick overview and step-by-step walkthrough, watch the following video tutorial: https://youtu.be/mmDCxwR8wY4 |
Create a subscription
Once the organization is registered, the next step is to create your Agora subscription.
There is a wizard with several steps that will guide you through this process. First, you will need to select the plan for your subscription (Standard, Enterprise or Enterprise Plus).
In the second step you will select a billing account (you can select an existing billing account or create a new one in the Billing Accounts section). You can only select billing accounts that are associated with the plan selected for the subscription.
The third step includes general settings like name, subscription identifier and description.
Regarding the subscription identifier, note that it will be part of the URI of the service when you are accessing this particular subscription (e.g. https://console.agora.denodo.com/acme-inc/subscriptions/<my-subscription-id>). Be sure to verify if your organization has any rules about the naming of your services (e.g. including “denodo” or “agora” in your subscription id may generate a URI for your subscription that is not approved by you organization). Also, take into account that if you select something like “my-org-name” as your department’s subscription’s id, other departments in your organization will not be able to use the same subscription identifier for their own subscription.
Finally, in the last step you will select the cloud provider and the region where the infrastructure will be deployed.
Once you click on “create” your subscription is initialized and the application starts provisioning the resources needed to deploy the service, including Denodo’s Solution Manager (the Control Plane).
For a quick overview and step-by-step walkthrough, watch the following video tutorial: https://youtu.be/2U4oYhMjntE |
Billing accounts
A billing account is the logical entity used to manage payments and it is associated with a subscription plan (Standard, Enterprise or Enterprise Plus). Billing accounts are established at the organization level and can be associated with multiple subscriptions configured with the same plan.
Create a new billing account
To create a new billing account, you will need to access the billing account section in the Organization admin panel and click the “New billing account” button.
First you must choose the subscription plan associated with the billing account.
Then, you need to enter a name for the billing account and set an optional description..
Options
Once the billing account is created, you can configure a payment method or edit its information.
Configure payment method
In this option, you will be able to configure the payment method. To do this, you need to enter an email address, a name, and an address. When configuring the payment method, the corresponding subscription is automatically generated in Stripe, a payment platform that allows you to set up the payment method for the account.
After completing the initial configuration, you will be able to access the Stripe portal from this same options menu. There, you will be able to set up a credit card and finish the configuration process. Check the details on the Billing portal section.
Edit billing account
With this option, you can edit the name and description of a billing account.
For a quick overview and step-by-step walkthrough, watch the following video tutorial: https://youtu.be/3sjxaVSGXPA |
Billing portal
In the billing portal, you can review your current plan and your invoice history, add or edit your payment method and view or modify your billing information.
Complete the subscription configuration
As part of the subscription configuration you need to connect the Control Plane of Agora with your AWS account. The Control Plane needs to assume a role created in your AWS account, in order to deploy and manage the resources of the Execution Plane.
You must provide the ARN of an IAM role created as indicated below.
For a more visual explanation of this process, refer to the video tutorial: https://youtu.be/p62nhSpcGyA |
Create a role to be assumed by the Control Plane of Agora
You must create an AWS role for an AWS account of your choice, by following these steps:
- Create an IAM policy for Agora:
- Log into your AWS Console as a user with administrator privileges on the desired AWS account and go to the IAM console.
- Click the Policies tab in the sidebar.
- Click Create policy.
- Click JSON and use the following JSON, instead of the default one, to configure all the permissions required by the Execution Plane of Agora.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "autoscaling:AttachInstances", "autoscaling:AttachLoadBalancerTargetGroups", "autoscaling:CreateAutoScalingGroup", "autoscaling:DeleteAutoScalingGroup", "autoscaling:DescribeAutoScalingGroups", "autoscaling:DescribePolicies", "autoscaling:DescribeScalingActivities", "autoscaling:DescribeScheduledActions", "autoscaling:DetachInstances", "autoscaling:DetachLoadBalancerTargetGroups", "autoscaling:UpdateAutoScalingGroup", "autoscaling:PutScalingPolicy", "autoscaling:PutScheduledUpdateGroupAction", "autoscaling:ResumeProcesses", "autoscaling:SuspendProcesses", "ec2:AcceptVpcPeeringConnection", "ec2:AllocateAddress", "ec2:AssociateIamInstanceProfile", "ec2:AssociateRouteTable", "ec2:AttachInternetGateway", "ec2:AuthorizeSecurityGroupIngress", "ec2:CreateImage", "ec2:CreateInternetGateway", "ec2:CreateKeyPair", "ec2:CreateLaunchTemplate", "ec2:CreateLaunchTemplateVersion", "ec2:CreateNatGateway", "ec2:CreateRoute", "ec2:CreateRouteTable", "ec2:CreateSecurityGroup", "ec2:CreateSubnet", "ec2:CreateTags", "ec2:CreateVpc", "ec2:CreateVpcPeeringConnection", "ec2:DeleteInternetGateway", "ec2:DeleteKeyPair", "ec2:DeleteLaunchTemplate", "ec2:DeleteLaunchTemplateVersions", "ec2:DeleteNatGateway", "ec2:DeleteRoute", "ec2:DeleteRouteTable", "ec2:DeleteSecurityGroup", "ec2:DeleteSnapshot", "ec2:DeleteSubnet", "ec2:DeleteTags", "ec2:DeleteVpc", "ec2:DeleteVpcPeeringConnection", "ec2:DeregisterImage", "ec2:DescribeAccountAttributes", "ec2:DescribeAvailabilityZones", "ec2:DescribeIamInstanceProfileAssociations", "ec2:DescribeImages", "ec2:DescribeInstances", "ec2:DescribeInternetGateways", "ec2:DescribeInstanceTypeOfferings", "ec2:DescribeInstanceTypes", "ec2:DescribeKeyPairs", "ec2:DescribeLaunchTemplateVersions", "ec2:DescribeNatGateways", "ec2:DescribeRegions", "ec2:DescribeRouteTables", "ec2:DescribeSecurityGroups", "ec2:DescribeSubnets", "ec2:DescribeTags", "ec2:DescribeVpcAttribute", "ec2:DescribeVpcPeeringConnections", "ec2:DescribeVpcs", "ec2:DetachInternetGateway", "ec2:DisassociateIamInstanceProfile", "ec2:ModifyInstanceAttribute", "ec2:ModifyLaunchTemplate", "ec2:ModifySubnetAttribute", "ec2:ModifyVpcAttribute", "ec2:ModifyVpcPeeringConnectionOptions", "ec2:RebootInstances", "ec2:ReleaseAddress", "ec2:ReplaceRoute", "ec2:RunInstances", "ec2:StartInstances", "ec2:StopInstances", "ec2:TerminateInstances", "elasticloadbalancing:CreateListener", "elasticloadbalancing:CreateLoadBalancer", "elasticloadbalancing:CreateTargetGroup", "elasticloadbalancing:DeleteListener", "elasticloadbalancing:DeleteLoadBalancer", "elasticloadbalancing:DeleteTargetGroup", "elasticloadbalancing:DeregisterTargets", "elasticloadbalancing:DescribeListeners", "elasticloadbalancing:DescribeLoadBalancers", "elasticloadbalancing:DescribeTargetGroups", "elasticloadbalancing:DescribeTargetHealth", "elasticloadbalancing:ModifyListener", "elasticloadbalancing:ModifyTargetGroupAttributes", "elasticloadbalancing:RegisterTargets", "iam:AddRoleToInstanceProfile", "iam:CreateInstanceProfile", "iam:CreateRole", "iam:DeleteInstanceProfile", "iam:DeleteRole", "iam:DeleteRolePolicy", "iam:GetInstanceProfile", "iam:GetRole", "iam:GetRolePolicy", "iam:GetUser", "iam:ListInstanceProfiles", "iam:ListInstanceProfilesForRole", "iam:PassRole", "iam:PutRolePolicy", "iam:RemoveRoleFromInstanceProfile", "iam:UpdateAssumeRolePolicy", "s3:DeleteObject", "s3:GetBucketLocation", "s3:PutObject", "sts:DecodeAuthorizationMessage" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "iam:CreateServiceLinkedRole" ], "Resource":"arn:aws:iam::*:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling", "Condition": { "StringLike": { "iam:AWSServiceName": "autoscaling.amazonaws.com" } } }, { "Effect": "Allow", "Action": [ "iam:CreateServiceLinkedRole" ], "Resource":"arn:aws:iam::*:role/aws-service-role/elasticloadbalancing.amazonaws.com/AWSServiceRoleForElasticLoadBalancing", "Condition": { "StringLike": { "iam:AWSServiceName": "elasticloadbalancing.amazonaws.com" } } } ] } |
- Click the Next button and fill the name field with Agora_Execution_Plane_PolicyPermissions_v1 (or any other name you prefer) and the description field if desired.
- Press the Create policy button.
- Create a cross-account IAM role:
- Log into your AWS Console as a user with administrator privileges on the desired account and go to the IAM console.
- Click the Roles tab in the sidebar.
- Click Create role.
- In Select type of trusted entity, click the AWS account tile.
- Check the Another AWS account checkbox.
- In the Account ID field, enter the Agora AWS account ID indicated in the Configuration of the Cross-account IAM role wizard of Agora.
- Optional: to use an External ID, check Required external ID. Then, in the External ID field, enter the external ID indicated in the Agora configuration wizard when checking Require external ID
- Click Next to go to the add permissions section.
- Search for the Agora_Execution_Plane_PolicyPermissions_v1 (or the name you gave to the policy in the previous step 1.e) and assign it.
- Click on the Next button to go the Name, review, and create section.
- Fill the role name field with Agora_Execution_Plane_IAM_Role (or any other name you prefer) and press the Create role button.
- Press on the View role button.
- Copy the role’s ARN and use it to fill the wizard in the Configuration of the Cross-account IAM role wizard of the Control Plane application of Agora.
Choose a deployment template
We are now ready to deploy. In order to simplify this step, the application allows you to choose a template that will create the environments, clusters and servers in your infrastructure. Each template is focused on a specific scenario, ranging from a basic evaluation to a full-blown production-ready deployment with three environments.
Alternatively, you can also configure all the settings manually using Denodo’s Solution Manager, by clicking on the “Provision Manually” button.
Each template offers several configurable settings that you need to fill up in this form.
For each environment of the template, you can let the Control Plane create a VPC and the associated subnets automatically, or you can choose to use existing VPCs and subnets. You can also define custom CIDR blocks for the VPCs and subnets in case you let the Control Plane create them.
When the Control Plane creates the VPC and subnets for each environment, their configuration allows the execution plane to connect to data sources accessible through Internet (any port). You will need to create a peering (or to use an equivalent strategy) to connect the execution plane to other sources and consumers.
You can provide your own key pair to the process. In order to do that, you have to uncheck the “Auto generate key pair” field and fill in the key pair name.
If you choose to auto generate key pairs, you will be able to copy or download them at the end of the process. In this case, after you close the “Download key pairs” modal window you will no longer be able to obtain the keys.
Environments
Once the environments are successfully provisioned according to your template, they will be accessible in the Environments section of your panel. You can select each environment with a drop down, check its status, and launch the applications available for that environment (Design Studio, Data Catalog and Scheduler Web Admin Tool, depending on the configuration of the environment):
- Design Studio (deployed in the Control Plane), to connect to the Virtual DataPort cluster located in the execution plane.
- Data Catalog (deployed in the Execution Plane).
- Scheduler Web Admin Tool (deployed in the Control Plane), to connect to the Scheduler cluster located in the execution plane.
You can also open Solution Manager using the “Manage Environments” button on the top right corner for more advanced options.
For a quick overview and step-by-step walkthrough, watch the following video tutorials: https://youtu.be/_MZM3igFs40 |
You can use the “OAuth client configuration” button to show the configuration values required by client tools to connect to the execution plane of the selected environment with OAuth.
Other management options
Inside the subscription panel, on the left hand section of the screen, you have access to additional options to manage roles, users, and monitor usage.
Settings
On this page, there are two tabs: General settings and Cloud settings.
General Settings
You can edit the basic subscription information such as name and description. You also have the options to Cancel subscription or Upgrade plan.
Cloud settings
In this option, you can edit the AWS client account access configuration, such as the IAM role ARN.
Roles
A subscription has two different types of roles:
- Roles for permissions for Agora. They can be assigned to users in the “Users” section.
- Roles for permissions for the Solution Manager. In the “Roles” section, you can check the predefined Solution Manager roles, and you can create new custom roles which will be available in the Solution Manager and in the execution plane (you can use the Design Studio to assign execution plane permissions to these custom roles).
Users
Manage all the users of your subscription:
- Invite new users to the subscription (and organization)
- Remove users
- Suspend and activate users
- Manage user roles
For a quick overview and step-by-step walkthrough, watch the following video tutorial: https://youtu.be/5VWMQePQOfU |
Usage
In this page you can see the usage details for your subscription, like uptime and consumed credits. In the summary section you can check the information of the services you have consumed in a particular time period and filter using different parameters.
For a more visual explanation of this process, refer to the video tutorial: https://youtu.be/wcnJwJ9kGj0 |
Maintenance
This section describes the maintenance actions performed by Agora that are associated with a subscription and may affect its availability. These actions include planned tasks at improving performance, updating systems, or resolving technical issues. During these maintenance periods, the service may temporarily not be available or experience limited interruptions.
In this page, you can review the scheduled maintenance or the maintenance history.
Cancel a subscription
You can cancel an active subscription and remove all the resources created by Agora in your AWS account. To do this, go to the subscription settings page and press the “Cancel subscription” button. A confirmation dialog is shown to avoid possible mistakes.
You can still access a canceled subscription to check its usage details.
Organization admin panel
If you are the owner of an organization (the user created in the first step), or you have the required roles, you can access the organization admin panel using the link at the bottom-left of the screen.
In this panel you can perform some global actions, depending on your role:
- Settings: Update organization attributes (name, description).
- Subscriptions: Create or update subscriptions and navigate to their panel.
- Roles: Check which global roles are assigned to each user.
- Users: Add / remove / activate / suspend users and manage their global roles.
- Usage: You can check the usage information of all your subscriptions here and filter by date range, subscription, environment types, etc.
- Audit: You can check all the audit records for your entire organization, filtering by date range, action type, subscription, etc.
- Billing accounts: here you can obtain information for each billing account. You can also configure existing billing accounts and add new ones.