The goal of this guide is to provide a quick guide of how to use the Automated Mode of the Solution Manager to deploy the Denodo Platform.
The Solution Manager 8.0 can automate the deployment of the Denodo Platform on Microsoft Azure. That is, it automates:
The main benefit is that you can instantiate and manage your entire Azure deployment without having to create and configure custom Azure elements and without RDP connections to each individual server to configure various settings.
To automate the installation and launch of the Denodo Platform servers, the Solution Manager requires Azure credentials provided by your organization. That way, the Solution Manager automates the deployment of the Denodo Platform with the benefit that all the infrastructure is still under the control of your organization and no one outside your organization can access it.
The Automated Cloud mode support for Azure is available since 8.0u20220126 so this only applies for this update or a more recent one.
This guide explains how to:
When you create a cluster in automated mode for Azure, the Solution Manager creates Azure Virtual Machines for each component of the Denodo Platform: one or more instances for Virtual DataPort, one or more instances for Data Catalog and one or more instances for Scheduler.
We recommend being familiar with these concepts.
To be able to use Denodo in Automated Cloud Mode with Azure you need to obtain the needed credentials. The Azure credentials used in the Solution Manager are made up of the following fields: subscription ID, client ID, tenant ID and client secret. The subscription ID is obtained at the subscription detail form, the rest of the fields are obtained registering an application.
Usually, the administrators of Denodo will request these to their Azure administrator. These credentials need the right privileges to be able to create the desired resources on Azure.
Go to the next section if you already have these credentials.
If you cannot get them from the administrator, try these steps. Your account on the Azure portal may not have the enough privileges to complete them.
Check Configuring Automated Mode for more information.
Prerequisite: this Azure account must have at least one Virtual Network created (check the Azure documentation for further details).
To obtain the Azure credentials, an App registration is needed, where you can obtain the clientID, tenant ID and client secret. To register an application follow these steps:
It is important to have in mind that client secret values cannot be viewed, except for immediately after creation. Be sure to save the secret when created before leaving the page.
At this point, you should have all the fields used as Azure credentials: subscription ID, client ID, tenant ID and client secret.
In Azure, a Role Definition is a collection of permissions that you can use to grant access to Azure Resources. It lists the actions that can be performed, such as read, write, and delete. A Role Definition is typically just called a role, and it can be high-level, like owner, or specific, like virtual machine reader. Please check the Azure documentation for further details.
Before using the Automated Cloud Mode for Azure, you have to define a Custom role that will allow the App registration associated with the Solution Manager to invoke all the necessary operations of the Azure API. Once the Custom role is created, it has to be assigned to the App registration. Please check the Azure documentation for further details.
Follow these steps to do create the Custom role:
To learn more about the needed custom role, read Creating an Azure Custom role. It also explains what permissions are optional and which ones are mandatory.
Once the App registration and the Custom role are created, go to the subscription or resource group selected as assignable scope and assign the role following the next steps:
You need to create a Network Security Group to allow connections between your computer and the users’ computers, and the Virtual Machines in which you will install the Solution Manager.
This section explains how to do this. Please check the Azure documentation for further details.
Source: your public IPv4 address
Destination: VirtualNetwork.
Destination port ranges: 3389,19090,19443
Protocol: Any
Follow these steps to launch a Virtual Machine where you will install the Solution Manager.
After launching the virtual machine, wait until it finishes its creation. It takes several minutes.
This section explains how to install the Solution Manager on your virtual machine.
After installing the Solution Manager, you have to set it up. To do this, follow these steps:
Follow these steps to provide the required configuration for the automated mode. You can access the web tool from your local computer because the Network security group has your public ip configured.
User name: admin
Password: admin
These access keys are necessary for two things:
Important: obtain these access keys. Otherwise, you will have to create your own Denodo Platform Image (this process is not trivial) to be able to create a cluster of Denodo servers in automated mode. It is also necessary to allow connectivity from the machine where the Solution Manager is installed to the Denodo Support Site. It is mandatory in order to use the images provided by Denodo and download updates. |
Do the same for Data Catalog image and Scheduler image.
Later, when you create clusters of Denodo servers in automated mode for Azure, by default, the Solution Manager will create the virtual machines based on these images (in the configuration of the cluster, you can specify different values).
Even though you select the same image, when the Solution Manager launches a virtual machine based on this image, it instructs to start Virtual DataPort or Scheduler or Data Catalog.
To create a new environment in automated mode, follow these steps. Note that in Solution Manager version 8.0, you can still register servers as you did in version 7.0.
Important: in order to use the image provided by Denodo, it is mandatory to accept the terms, otherwise an error will be shown trying to create the cluster. They can be accepted in the Azure Cloud Shell through the next command: PS> az vm image terms accept --publisher denodo --offer denodo-8_0-vm-auto --subscription <your subscription id> --plan denodo-8_0-vdp-auto-win-<update>2200 The plan depends on the image selected. For version 8.0u20220126 the value for the plan parameter will be denodo-8_0-vdp-auto-win-202201262200 |
For evaluation purposes, set this to 1. To use more instances you will have to set-up the feature to store the configuration of Scheduler on an external database so they all share the same configuration.
Note: each instance runs one and only one Denodo service. That is, there will be one virtual machine for Virtual DataPort, one for Data Catalog and one for Scheduler.
The Solution Manager begins creating the instances based on the Image(s) you indicated.
It usually takes 5 to 15 minutes to create the instances and launch the Denodo servers.
If after a few minutes of refreshing, none of the nodes show up here, it means that that service has not started.
After this, click on the Denodo logo on the top left to go back to My Applications. Now you should see the links to access Data Catalog, Design Studio, Diagnostic & Monitoring Tool and Scheduler.
Solution Manager 8.0 is capable of integrating with Identity Providers (IdP) that support Kerberos, OAuth, SAML or OpenID. With this, the users will be able to use single sign-on. That is, they will only have to enter the credentials once on the identity provider (IdP) or never in the case of Kerberos. After that, they will be able to log in to Solution Manager and from there, to Design Studio (the new web interface for Virtual DataPort), Data Catalog and Scheduler, without entering their username and password.
The privileges system of the Solution Manager is equivalent to the one of Virtual DataPort. That is, the Solution Manager obtains the roles assigned to users. Then, the actions they can do depend on the privileges assigned to these roles in the Solution Manager.
To enable single sign-on, follow these steps:
Find more details about this process in the section Authenticating with Single Sign-On of the Solution Manager Administration Guide.
After doing this, you need to create the necessary roles in the Solution Manager and in the Virtual DataPort you are going to connect to:
After doing this, log out of the Solution Manager. You will now see a button Single Sign-On. If you click on it, you will automatically log in to the Solution Manager, if you already logged in to your Identity Provider.
To obtain support, post your question on the Q&A page of the Denodo Community Site. The Denodo Team and data virtualization professionals and enthusiasts will assist you.
Please check more information about Denodo for Azure Support and Denodo Maintenance and Support Service Guide for further information.
No.
The Solution Manager and the Denodo servers run on the Azure subscription of your organization. That is why, to work in automated cloud mode you have to provide your own Azure credentials.
The Solution Manager uses the Azure SDK for Java provided by Azure. Using this SDK, the Solution Manager launches new instances based on the image indicated when creating the cluster. To do this, it uses your Azure credentials.
When the Solution Manager launches an instance, it uses a feature of Azure called User Data for Azure Virtual Machine that is meant to pass information to virtual machine instances. In the case of the Solution Manager, it passes this information:
The Image for Denodo is configured to execute the script azure_init_config.py (in <Denodo_platform_installation_path>/tools/cloud/azure/) when the instance starts. This script reads the “user data” passed to this instance and among other things:
When you create a cluster of Denodo servers in automated mode, for each component (Virtual DataPort, Scheduler and Data Catalog), you can choose to either use an existing Network Security Group or for the Solution Manager to create a new one.
For evaluation purposes, it is easier to select Create a security group because the Solution Manager will automatically create one security group for each component and each one of these groups will allow inbound traffic only to the ports of that component. However, in a “real world scenario”, you should check with the Azure team of your organization to determine the best option depending on the architecture of the network.
Alternatively, you can edit the security group you created for the Solution Manager and allow inbound connections to the following ports
The incoming connections must be allowed from the computer of the users of Denodo and the client applications that connect to Denodo.
Please check the Azure documentation for further details.