Denodo Platform for GCP 9 BYOL Quick Start Guide
You can translate the document:
Overview
Denodo Technologies, the data virtualization leader for unifying enterprise data and delivering data services for the business, is providing Denodo Platform available on GCP as bring your own license (BYOL) virtual machine (VM). The VMs contain the latest versions of Denodo Solution Manager and Denodo VDP, optimized for use with the GCP Virtual Machines service.
Thank you for your confidence in choosing Denodo as your data virtualization technology!
This quick start guide will give you an overview of the Denodo Platform and how to quickly start building data virtualization solutions on GCP.
Marketplace Offering Overview
This Denodo for GCP offering is based on a Bring Your Own License (BYOL) model under your GCP project.
It is expected that you already have a GCP project that you can use to logon to the GCP Marketplace in order to create and launch the Denodo VM BYOL through the GCP Console. And you are responsible for operating your own GCP project.
The Denodo VM BYOL consists of the same engine as the Denodo Platform. Some of the key features include:
- The Denodo Solution Manager (SM), to manage your Denodo deployment.
- The Denodo Virtual DataPort (VDP), the core Denodo Data Virtualization engine, to integrate data from any data source and deliver it to any consumer in any format.
- The Denodo Scheduler, for scheduling data pipelines and cache loads among other tasks.
The VMs available are based on CentOS Linux:
- A Linux Server Virtual Machine with Denodo Solution Manager.
- A Linux Server Virtual Machine with Denodo VDP and Denodo Scheduler.
In the sections below you can find more information on how to deploy.
Denodo Platform License
The solution includes a limited Denodo Solution Manager license:
- It allows you to manage up to 2 Denodo Platform environments (PRODUCTION and DEVELOPMENT) in Standard Mode.
- It does not assign licenses to Denodo Platform Servers (VDP, Scheduler, Data Catalog) so it will require you to create Denodo Platform Servers referencing those Denodo Platform machines with already assigned licenses and Denodo services running, for example those from Denodo 9 PAYG offerings in the GCP Marketplace.
- It allows you to configure the Denodo Security Token to use Single Sign On for Denodo Web Tools (Design Studio, Scheduler Admin...).
In case you have a Denodo Solution Manager license procured from Denodo Technologies or resellers, you can replace the limited Denodo Solution Manager License that is included by the one you procured. If you currently do not hold a license agreement for Denodo Platform, and you are interested in, please contact Denodo directly to pursue a license to use the software.
Denodo BYOL Deployment Overview
A Denodo deployment is composed of:
- Deploy one Denodo SM 9 on Linux Server BYOL virtual machine.
- Deploy one or more Denodo VDP 9 on Linux Server BYOL virtual machines connected to the Denodo SM previously deployed.
Denodo SM 9 Deployment Steps
Overview
The high level list of steps needed to provision and configuring your Denodo SM BYOL on GCP is the following:
- Deploy the Denodo SM virtual machine through your GCP console.
- Install Denodo SM License in the Denodo SM VM.
- Start Denodo Platform Services in the Denodo SM VM.
- Log in the Solution Manager Administration Web Tool.
- Create one or more environments.
- Register one or more Denodo servers.
- Connect to a VDP Server with Design Studio
All the processes can take an approximate time of 30 to 50 minutes.
SM1 - Denodo SM Virtual Machine Deployment
You are responsible for operating your own GCP project.
You can create and launch your Denodo SM for GCP BYOL VM using your GCP user.
The GCP offering includes a Linux Server Virtual Machine with Denodo SM 9 installed.
Log on to the GCP Marketplace and search for the Denodo Platform 9 SM for GCP BYOL listing.
Note: The GCP interface is subject to change and the exact options you see depend on whether or not you have existing projects.
And then click the LAUNCH button.
Then, Select a project or Create a project page could be the next. Enter a project name and select a value from the Organization drop‑down, then click the Create button.
After selecting the Denodo Solution Manager 9 BYOL on Linux Server available through the GCP Marketplace you have to complete some few configuration steps in order to launch the Denodo virtual machine through the GCP console.
NOTE: see Denodo SM Hardware Requirements before choosing the VM Size.
Then Click the DEPLOY button.
As soon as the project deploys, and the new virtual machine (VM) instance starts running, the following services are automatically started with the limited Denodo Solution Manager license, and the password to enter into the services is the one generated by Google:
- Denodo License Manager Server.
- Denodo VDP Solution Manager Server.
NOTE: Before accessing the services, please wait for a few minutes to ensure that everything is configured correctly.
SM2 - Install Denodo SM License
The Denodo services require a Denodo license to start.
Then follow one of the below:
- If you have a Denodo Solution Manager license file that you want to install, do so via solution-manager-web-tool. See Install a License.
- Or, if you want to use the limited Denodo Solution Manager license included go to the next step directly.
SM3 - Log Into the Denodo SM Administration Tool
The Solution Manager is a component to help you manage Denodo deployments.
As soon as the new virtual machine (VM) instance starts running, the following services are automatically started with the limited Denodo Solution Manager license and the password to enter the services is the one generated by Google:
- Denodo License Manager Server.
- Denodo VDP Solution Manager Server.
NOTE: Before accessing the services, please wait for a few minutes to ensure that everything is configured correctly.
Once you are logged in, the Denodo Solution Manager Administration Tool provides a single point of entry to all the web applications of the Denodo deployment.
Check first the list of Supported Browsers for Solution Manager admin tool.
Then use a supported browser to point to the URL to access Denodo Solution Manager Administration Tool with the following pattern:
http://<sm_ip>:19090/solution-manager-web-tool/Login
Where <sm_ip> is the IP address or DNS name for the Solution Manager virtual machine in the stack deployed.
NOTE: the application may take some time to start so you may need to wait for a couple of minutes before the link works correctly. You may get a connection error or a ‘404 Not Found’ error if you try to access it before the service has completely started.
The user is admin and the password by default is admin so the recommendation is to change the default password first thing:
- Go to the Configuration menu.
- Click on User management.
- Click the Edit icon on the right side for the admin user.
- Set the new password.
Then logout and sign in again with admin as username and the new password.
SM4 - Create One or More Standard Mode Environments
A Denodo Environment is defined as a set of servers, of the same or different type, working together for a common purpose. For example, Production, Development or Staging environments.
For creating a new Platform Mode Environment from the Denodo Solution Manager Administration Web Tool:
- Click on Environments > New Environment.
- Choose Platform Mode.
- Click on Create Environment.
- Set a Name for this environment.
- Select the License scenario for this environment.
- Save this new environment.
- Click over the new environment on the left side > New Cluster.
- Set a Name for this cluster.
- Save this new cluster.
- Click on the left side of the Environment and the new cluster will be displayed underneath
Repeat above steps for all environments and clusters you want to set up.
With the environment and cluster created everything is ready to add Denodo servers like VDP or Scheduler.
SM5 - Register one or more Denodo servers
Last step is registering deployed Denodo servers (like VDP or Schedulers) in created clusters.
Instructions on how to deploy a Denodo VDP Virtual Machine are available in a chapter below in this guide.
For adding a new Denodo server to a cluster from the Denodo Solution Manager Administration Web Tool:
- Click over the environment
- Click over the cluster > New Server.
- Set a Name for this server.
- Select the Type of server. There are different types available: Virtual DataPort, Scheduler, ITPilot Browser Pool, ITPilot Verification or Data Catalog. In this case, add a Virtual DataPort Server.
- Set the Host, Port, Type, User, Password and Default database.
- Save this new server.
Repeat above steps for all deployed Denodo servers.
SM6 - Log into the Design Studio
The Design Studio provides a web interface to the developers to create data sources, base views, derived views, publish web services, etc.
Check first the list of Supported Browsers for the Design Studio tool.
Then use a supported browser to point to the following URL of the Design Studio Web Tool following the pattern:
http://<sm_ip>:19443/denodo-design-studio/
Where <sm_ip> is the IP address or DNS name for the Solution Manager virtual machine in the stack deployed.
In the Solution Manager Web Tool homepage you have links to the Design Studio and Scheduler Web Tools: select the Environment and then click on the Open link for going to the homepage of the Design Studio preloaded with the connection data to the cluster in that environment.
Refer to Main Areas of the Design Studio for more information.
Note that certain administrative tasks like cache configuration, setting-up the authentication, user management, etc can be done by graphical way only by using the Virtual Dataport Administration tool (desktop based).
Appendix - Start Denodo SM Services
Logged in with your user through SSH and once a Denodo license is installed you can start enjoying Denodo software.
Use the terminal to run these commands in your instance to start Denodo services in the following order:
sudo systemctl start licensemanagerserver
sudo systemctl start solutionmanagerserver
After executing those commands the following services will be started:
- Denodo License Manager Server.
- Denodo Solution Manager Server.
- Denodo Solution Manager Web Tools.
You can check the status of the Denodo Services with the following commands:
systemctl status licensemanagerserver
systemctl status solutionmanagerserver
And you can stop the Denodo Services with the following commands:
sudo systemctl stop licensemanagerserver
sudo systemctl stop solutionmanagerserver
NOTE: The Denodo services are not configured to start at instance boot time. You can configure those services to start at instance boot time with the commands:
sudo systemctl enable licensemanagerserver
sudo systemctl enable solutionmanagerserver
NOTE: Detailed instructions on how to connect to the virtual machine are available on Connect to instances
Deploy the Denodo VDP VM
Overview
The high level list of steps needed to provision and configuring your Denodo VDP BYOL is the following:
- Launch the Denodo Platform VM
- Configure a Solution Manager
- Start Denodo Platform Services
- Login to the Design Studio
Note: The GCP interface is subject to change and the exact options you see depend on whether or not you have existing projects.
VDP1 - Launch the Denodo Platform VM
Log on to the GCP Marketplace and access the Denodo Platform 9 VDP (BYOL).
And then click the LAUNCH button.
Then, Select a project or Create a project page could be the next. Enter a project name and select a value from the Organization drop‑down, then click the Create button.
After that the deployment configuration appears where you can select appropriate values for zone, machine type, and so on.
For choosing the machine type please see Denodo Hardware Requirements.
Then Click the Deploy button.
As soon as the project deploys, and the new virtual machine (VM) instance starts running, the following services are automatically started:
- Denodo VDP Server.
- Denodo Web Container for accessing Denodo Design Studio web tool.
- HTTP web server to allow access to Usage Instructions described in the following sections.
VDP2 - Configure a Solution Manager
The Denodo services require a Denodo license to start.
Log in the Denodo VDP for GCP BYOL on Linux with your user through a SSH Connection. Detailed instructions are available on Connect to a Linux Virtual Machine.
You need a Denodo Solution Manager installed and running, and accessible in the same network from the Denodo VDP for GCP virtual machine through internal IPs registered in the internal DNS.
The Denodo servers need to be previously registered in the Denodo Solution Manager in one of the clusters for an environment. You can find more detailed information on the Denodo official documentation on how to create a server in the Solution Manager. Be sure of declaring the correct hostname of the Denodo server. You can register the following Denodo servers in the Denodo Solution Manager: VDP Server; Scheduler Server; Data Catalog Server.
Then, in the Denodo VDP for GCP BYOL VM logged as your GCP user edit the /opt/denodo/denodo-platform-9-linux64/conf/SolutionManager.properties file and set the value of these properties:
- com.denodo.license.host: the License Manager’s host, this should be the IP or hostname for your previously deployed Denodo Solution Manager where the Denodo VDP server is declared.
- com.denodo.license.port: the License Manager’s port, by default is 10091.
VDP3 - Start Denodo Platform Services
Logged in with your GCP user through SSH and once a Denodo license is installed you can start enjoying Denodo software.
Use the terminal to run this command to start the Denodo VDP Service:
sudo systemctl start vqlserver
Additionally you can start other Denodo Services (if they are allowed by your acquired Denodo License):
sudo systemctl start schedulerserver
sudo systemctl start schedulerwebadmintool
sudo systemctl start designstudio
sudo systemctl start datacatalog
sudo systemctl start diagnosticmonitoringtool
You can check the status of the Denodo Services with the following commands:
sudo systemctl status vqlserver
sudo systemctl status schedulerserver
sudo systemctl status schedulerwebadmintool
sudo systemctl status designstudio
sudo systemctl status datacatalog
sudo systemctl status diagnosticmonitoringtool
NOTE: The Denodo services are not configured to start at instance boot time. You can configure those services to start at instance boot time with the commands:
sudo systemctl enable vqlserver
sudo systemctl enable schedulerserver
sudo systemctl enable schedulerwebadmintool
sudo systemctl enable designstudio
sudo systemctl enable datacatalog
sudo systemctl enable diagnosticmonitoringtool
VDP4 - Login to the Design Studio Web Tool
The Design Studio provides a web interface to the developers to create data sources, base views, derived views, publish web services, etc.
The URL of the Design Studio Web Tool follows the pattern:
http://<vm_ip>:9090/denodo-design-studio/
Where <vm_ip> is the IP address or DNS name for the Denodo Platform VDP virtual machine in the deployment.
The username is admin and the password is the one you can see in the GCP Console for the deployment.
Refer to Main Areas of the Design Studio for more information.
Security Configuration
There are a couple of optional steps that we recommend you to take in order to improve the security of your GCP deployment.
Changing the Default VDP Administration Password
As the first step of the configuration of your new Denodo Platform virtual machine you should change the default administrator password. It is extremely important that you change this default password to ensure that you are operating under a secure environment.
VDP Client Access from Outside of VM
You have to configure the Virtual DataPort server in order to receive connections from external clients to the VM.
The default ports for the Denodo Platform based on the connection protocol are:
- JDBC: TCP 9999 (additionally the TCP 9997 port has to be available).
- ODBC and ADO.NET: TCP 9996.
- Web Container: TCP 9090.
VM must be configured to allow the connections to ports enumerated above according to the connection protocol needed.
As a side note, take into account that by default an ephemeral external IP address is assigned to the VM instance. If you require a static external IP address, you may promote the address to static. Be sure that the VDP server is set correctly for accepting connections through that IP by changing the Host Name in the VDP server.
Enabling SSL
You can find details about how to secure with SSL the connections between the Solution Manager servers, the administration tools and their clients in Enable SSL in the Solution Manager.
Additionally you can find details about how to secure the connections for Denodo VDP servers in Enable SSL in Denodo.
Enabling Denodo Security Token
Solution Manager requires privileged connections to manage the Denodo VDP servers. It uses a temporary system token with the necessary permissions to perform administrative tasks and therefore, the Denodo VDP server must be configured with Denodo Security Token authentication in order to validate these tokens.
See Enable Denodo Security Token for details.
Further Recommended Steps
Once you are all set to start building your data virtualization solutions on GCP, we recommend that you check out all the available information: