Denodo Dashboard - User Manual

Last modified on: 21 Mar 2019

Preface

Scope

Denodo Dashboard provides a single interface for real time monitoring of Denodo Platform servers and environments (groups of servers) and for import/export operations between several Denodo Platform servers, it allows you to synchronize VDP servers. This document introduces the reader to the main functionalities of the tool.

Who should use this document

This document is aimed at system administrators that require detailed information of the status of a server in real time.

Summary of contents

This document describes:

How to install Denodo Dashboard

How to connect to a server in order to monitor its main parameters and data sources

How to define and monitor an environment (set of servers to be monitored together)

Installation and initial configuration

Software requirements

This software requires, at least, Java 2 Runtime Edition version 6 (JRE1.6) installed. The environment variable JAVA_HOME must be pointing to the folder containing the JRE installation.

In case you wish to use the import/export features, you will need to have the environment variable DENODO_HOME set and pointing to the installation folder of your Denodo Platform, or instead of using the platform you can use Denodo Tools, for this you will need to set the environment variable DENODO_DB_TOOLS_HOME. Otherwise, these variables are not necessary.

Installation

This software is distributed in a zip file that only has to be extracted for installation.

Execution

To launch the application, execute one of the scripts included in the Denodo “bin” folder:

For Windows systems, execute the script “denodo-dashboard.bat”

In Unix-like systems, execute the shell script “denodo-dashboard.sh”

For cygwin, execute the shell script “denodo-dashboard-cygwin.sh”

All the scripts use the JAVA_OPTS environment variable to configure the JVM. If it does not exist, it will be set to -Xmx1024m (usually enough in case you use import/export features). Otherwise it will use the set value. Please, make sure you use enough memory in case you wish to use import/export features, as this feature needs it.

Once the tool is launched, the initial empty screen will be displayed. Below there is an image of the main screen as it appears just after being started.

Connecting to an SSL-enabled server

Dashboard can be connected with a server that has SSL enabled. For this, the “Server uses SSL” option should be checked when registering the server. But some configuration might be needed at the startup scripts too:

If you use the trust store by default and if the server's public key is signed by a well known international CA no further configuration will be required. If not, you should configure the trust store you are using. A common way to do this is to set the Java system property javax.net.ssl.trustStore by adding -Djavax.net.ssl.trustStore=<path_to_truststore> at the launcher script or at the JAVA_OPTS environment variable (if the client allows to use it), and the same for the system property javax.net.ssl.trustStorePassword only if it is necessary by adding -Djavax.net.ssl.trustStorePassword=<truststore_password>.

Note that, depending on the Operating System you are using, you might need to modify denodo-dashboard.bat, denodo-dashboard.sh or denodo-dashboard-cygwin.sh.

For example, you could adapt your denodo-dashboard.bat script by replacing this line:

%JAVA_BIN% %JAVA_OPTS% -classpath "%DENODO_SERVERMONITOR_CLASSPATH%" com.denodo.connect.monitoring.dashboard.MonitorTool %OPTIONS%

...with:

%JAVA_BIN% %JAVA_OPTS% -Djavax.net.ssl.trustStore="C:/Program Files/Denodo Platform/certs/mycacerts" -Djavax.net.ssl.trustStorePassword=admin.10 -classpath "%DENODO_SERVERMONITOR_CLASSPATH%" com.denodo.connect.monitoring.dashboard.MonitorTool %OPTIONS%

Overview

The Denodo Dashboard has three main areas: the menu bar, the side tree, and the workspace.

The menu bar

It has the following menu entries:

File: contains the following submenu

New Server: opens the new server screen in the workspace area.

New Environment: opens the new environment screen in the workspace area.

Save Screenshot: saves a screenshot of the Denodo Dashboard window. A file chooser is opened to select the folder where the image will be stored in (the name of the file is autogenerated and will include the date and time of capture in the file name).

Exit: exits and closes the tool.

Help: has the following submenu

About: shows an informative popup dialog that displays the version of the Denodo Dashboard and copyright information.

The side tree

The side tree shows a list of the elements that this tool is monitoring. It has two top level categories: servers and environments:

Servers are individual Denodo Platform servers monitored by the Denodo Dashboard.

Environments are named groups of individual servers. These groups can be created for the different environments present in an enterprise Denodo Platform installation (for example, an environment could be created for the Development servers, another environment for the QA servers, another for the Production cluster, etc).

The side tree lists all the servers and environments that have been added to the Dashboard. The options available for each element are accessible using a right-click popup menu.

Servers section

Shows all the servers created in the tool. The following options are available from the “Servers” right click menu:

New server: opens the new server screen in the workspace area.

These are the right click menu options for each server in this section:

Monitor: opens the server monitor in the workspace area and starts monitoring the selected server, gathering real time information about its status.

Disconnect: disconnect from the server so that charts don’t keep receiving information from VDP. When connected, it does not matter whether the server monitor window is open or not, once opened the screen will show information as soon as the user starts monitoring it.

Ping: launches a ping query to this server. The ping time (in milliseconds) is displayed in a popup window.

Edit: opens the edit server dialog in the workspace area.

Delete: removes the server from the list and from all the environments it belongs to. A confirmation popup will be displayed prior to this operation.

Add to environment: adds the selected server to the environment selected from the submenu.

Environments section

Shows all the environments created in the tool. The following options are available from the right click menu:

New environment: opens the new environment dialog in the workspace area.

For each environment, it shows all the servers that belong to it. The following options are available from the right click menu of each environment:

Monitor: opens the environment monitor in the workspace area

Edit: opens the edit environment dialog in the workspace area

Delete: removes the environment from the list. A confirmation popup will be displayed prior to this operation.

Synchronize Environment: imports metadata from the Master node to the other nodes in the environment.

For each server that belongs to the environment, these are the right click menu options:

Monitor: opens the server monitor in the workspace area.

Disconnect: disconnects the environment so that charts don’t keep receiving information from VDP. When connected, it does not matter whether the environment monitor window is open or not, you will see information since the moment you started monitoring it

Ping: launches a ping query to this server. The ping time (in milliseconds) is displayed in a popup window.

Remove from this Environment: removes this server from the list of servers that belong to this environment. The server is not deleted from the servers section.

Set as Master/Clear Master Status: sets the server as the master node of the environment. If the server is already the master node, this status will be removed. The master node of an environment hosts the version of the metadata that will be propagated to the slave nodes.

Import Metadata from Master: this option is enabled only in case the environment contains more than one server and one of them is the master node. In such cases, you could import metadata from the Master node to another one by clicking “Import Metadata” from the node you wish to import metadata to.

The health of each server is described as follows:

When the server is “Up and running”, then the server icon is drawn in green:
When the server is “Down”, then the server icon is drawn in red:
When Dashboard does not have information about the server health (because no information is being gathered about that server at all), the server icon is drawn in grey:

For each server, it shows the monitoring status as follows:

When a server is being “actively monitored”, server name is displayed blue
Otherwise, the server name is displayed in black:

The workspace

It is the main part of the tool and it is where all the dialogs are displayed. These dialogs will be covered in the following sections.

Server monitoring

This section provides a detailed overview on the server monitoring capabilities of the tool.

Creating a new server

Before monitoring the status of a server, it must be added to the Denodo Dashboard. The screen where a new server is defined can be accessed in three ways:

From the main menu, “File > New Server”
From the right click popup menu on the “Server”
Using the keyboard shortcut “Ctrl+Shift+S”

The “New Server” form looks like this:

The following parameters must be filled:

Name: name that will be displayed in the server list in the side tree. This name is used only for display purposes.

Host: connection parameters for the Denodo server. Can be in the form <host>:<port> or simply <host>. If the latter, the port 9999 is assumed. <host> can be a machine name or an IP address.

JMX Web Server Port: port that allows monitoring of the embedded web server, it is not mandatory. The tab of Web Server overview will not appear if this field is empty.

Password: password to get the connection to the server.

Server uses SSL: if the server has SSL enabled you should check this field. If you don’t check this field when trying to connect to an SSL-enabled server, Dashboard will raise an error when trying to monitor the server.

Once the form is filled, click “Ok” to create the server, or “Cancel” to leave the dialog with no changes.

Editing an existing server

An existing server can be edited by clicking on the “Edit” option of the popup menu of the server in the side tree. The form is identical to the “New Server” form, but the inputs are filled in with the current parameters.

Once the modifications are done, click “Ok” to modify the server, or “Cancel” to leave the dialog with no changes.

In case the server you are editing is still being monitored (green icon in the side tree), monitoring will end once you update the server and start when you restart monitoring.

Monitoring a server

The server monitor can be launched by double clicking on its icon in the side tree, or using the “Monitor” option in its popup menu.

Whenever a server is being monitored either in the background or the foreground, a green icon will be displayed next to the server icon in the side tree.

The monitor window contains three tabs:

Server Overview
Data source activity
Cache summary

The monitor workspace contains two buttons at the bottom of the page (no matter which tab you select):

Ping: performs a ping to the server

Disconnect: when clicked you stop monitoring the selected database (it will not be monitored in the background)

Server overview

The server overview tab provides a detailed overview of the server most important data from a monitoring point of view: CPU load, memory load, incoming connections, and outgoing connections. The information is displayed in four charts displayed in a 2x2 grid.

Server info

This data provides information about the server. The following information is displayed above the tab:

Master: true if the server is running standalone or is the master instance of a group of instances of a server.

Instance name: if the attribute Master is false, this attribute contains the name of the instance.

Server name: Virtual DataPort server URI.

CPU load

This chart shows the CPU activity of the server process –in orange- and its garbage collector –in blue. The following information is displayed above the chart:

CPU Usage: percentage of the total CPU power used by Denodo VDP server process.

GC Activity: percentage of the total CPU power used by the garbage collector of Denodo VDP server.

Server memory

This chart displays the heap size in MB through time. The heap size is shown in orange, and the used size of the heap in blue. On top of the chart, the following information is shown:

Size: amount of memory that has been currently assigned to the Java Virtual Machine, in bytes

Used: actual amount of the assigned memory that is in use, in bytes

Max: maximum amount of memory that can be assigned to the heap space (configured in the JVM startup options), in bytes

Inbound connections

This chart shows the connections established from external clients to the Denodo Platform server. The chart displays the open connections (in orange) as well as the active requests (the ones executing a query, in blue) and the waiting ones (purple).

The following information is also displayed above the chart:

Waiting requests: requests that are in the waiting queue of the Denodo server waiting for an available connection

Active Requests: connections that are executing a request

Open Connections: active connections established from external clients

Total Connections: total number of connections opened to the Denodo server since its last restart

Outbound connections

This chart shows an overview of the connections opened to the external data sources from Denodo server. Each data source type is displayed in a different color. Only the types with existing data sources are represented.

Data source activity

The Data source activity panel provides detailed information for the activity of each data source created in the server. In the upper part of the tab, two drop down selectors allow the user to choose the virtual database to monitor, as well as filter by data source type.

The information is displayed in a matrix of charts, one per data source as seen in the image below.

The charts show the following information:

Active connections: number of connections active in the data source

Idle Connections (only for JDBC/ODBC): idle connections available in the connection pool.

On top of the charts, the following information is displayed:

Total Requests: total number of connections made to the data source since the last restart of the server

Max Active Connections (only for JDBC/ODBC): maximum number of connections allowed by the connection pool

Cache summary

The Cache summary panel provides detailed information related to the cache: cache status for each view, last refresh, … In the upper part of the tab, one drop down selector allows the user to choose the virtual database to monitor.

The information is displayed in the workspace as seen in the image below. Note only the views with the cache activated will be displayed in the table at the bottom of the window.

The cache monitor window contains:

A chart containing the connections

A table with the cache contents details. When you click on a view in this table, a new tab will be opened with properties on that view.

Connections chart

This chart represents the connections to the cache data source. The data source can be one of the following:

vdpcachedatasource: the global cache data source defined at the server level

customvdpcachedatasource: in cases where a VDP database defines its own data source for caching views of that virtual database

This chart displays the following information:

Active connections -in orange: number of connections currently active

Idle connections -in blue: idle connections available in the pool

On top of the chart, the following information is displayed:

Total Requests: number of total requests made to this data source since last server restart

Max Active Connections: maximum number of connections allowed by the pool

Cache refresh process info

One tab shows the active processes which are inserting data in the cache and the other one all processes(the previous ones and the ones that inserted data and finished).

Each of the tabs contains the following information:

View Name: name of the view in the chosen database

Received Rows: number of rows storage in cache

Start Date: the date when the process starts to storage in cache

End Date: the date when the process ends to storage in cache

Cache contents table

The cache contents table contains the following information:

View Name: name of the view in the chosen database

Cache Type: one of: OFF, ON, POST, PRELOAD depending on the cache type

TTL (seconds): time-to-live of the cached data. Cached data will expire after TTL seconds

Last Refresh: last time the cache was refreshed

Last Access: last time the cached data was accessed

Apart from that information, if you click on a view, a new tab will be opened with further details on that view cache. This tab contains the following information:

View Name: name of the view

Status: status of the cached data. Either valid, processing (data is being stored in the cache), invalid (expiration time has been reached)

Projected Fields: fields fetched from the data source

Conditions: conditions of the query where clause used to fetch data from the data source

Expiration Date: date of the cached data expiration

The table data is being updated based on the database contents so new views will be added/updated in case any change happens in the database being monitored. The view details table is also being updated based on the view cache configuration.

Web Server overview

This data provides information about the embedded web server of the platform. The following information is displayed above the tab:

HTTP Port(s): list of port numbers on which the web server listens

HTTPS Port(s): list of port numbers on which the web server listens using SSL

This tab only appears if it is configured a JMX Web Server Port in the parameters of the server.

CPU load

This chart shows the CPU activity of the web server process –in orange- and its garbage collector –in blue. The following information is displayed above the chart:

CPU Usage: percentage of the total CPU power used by web server process.

GC Activity: percentage of the total CPU power used by the garbage collector of the web server.

Server memory

This chart displays the heap size in MB through time. The heap size is shown in orange, and the used size of the heap in blue. On top of the chart, the following information is shown:

Size: amount of memory that has been currently assigned to the Java Virtual Machine, in bytes

Used: actual amount of the assigned memory that is in use, in bytes

Max: maximum amount of memory that can be assigned to the heap space (configured in the JVM startup options), in bytes

Threads

In this section the web server’s thread pools are monitorized. There is one for each connector, and you can select the one being monitored in the Thread Pool combo box.

The chart shows the following information:

Current Thread Count: threads that are ready to use, shown in blue.

Current Threads Busy: threads that are being used in this moment, shown in orange.

MaxThreads: concurrent threads that the pool is capable of managing, shown on top of the chart.

Sessions

In the last chart of this view, you can see session information for every web application on the web server. You can choose the application to monitor in the selector.

You can see the following fields in the chart:

Active Sessions: Number of currently active sessions, shown in orange.

Max Sessions: The maximum number of active sessions allowed --or none if there is no limit--, shown on top of the chart.

Note that most web applications shipped with the Denodo Platform are stateless and therefore create no sessions. But these graphs will help you monitor any additional web applications you deploy on the VDP embedded web server which might require the creation of new sessions.

Environment monitoring

This section provides a detailed overview on the environment monitoring capabilities of the tool. An environment is just a set of servers that will be monitored together.

Creating a new environment

The screen for defining a new environment can be accessed in three ways:

From the main menu, “File > New Environment”

From the right click popup menu on the “Environment”

Using the keyboard shortcut “Ctrl+Shift+E”

Below there is a screenshot of the “New Environment” form, displayed in the workspace area:

The following parameters must be filled:

Name: name that will be displayed in the environment list in the side tree

Servers in the environment: a list of the servers currently defined in the “Servers” section. Select on this list all the servers in the environment being created (to select multiple servers use CTRL+click on the name of the server)

Name is compulsory and at least one server must be selected. Once the form is filled, click “Ok” to create the environment, or “Cancel” to leave the dialog with no changes.

Editing an existing environment

An existing environment can be edited by clicking on the “Edit” option of the popup menu of the environment in the side tree. The form is almost identical to the “New Environment” one, but the inputs are filled with the current parameters and there is one additional button to clear the master node of the environment.

In case you wish to clear the master node, click “Clear Master Node”. Otherwise, once the modifications are done, click “Ok” to modify the environment, or “Cancel” to leave the dialog with no changes.

In case the environment you are editing is still being monitored (green icon in the side tree), monitoring will end up once you update the environment and start once you monitor it again.

Monitoring an environment

The environment monitor can be launched by double-clicking on its icon in the side tree, or using the “Monitor” option in its popup menu.

Whenever an environment is being monitored, either in the background or the foreground, a green icon will be displayed next to the environment icon in the side tree

The monitor window contains several charts representing:

CPU load
Environment memory use (Heap)
Connections

The following information is also displayed below the charts for each of the servers in the environment:

Server: name given to the server in the Denodo Dashboard

URI: server location (host:port)

Status: server status (i.e. Up and Running)

Uptime: time since last restart

Memory: percentage of memory used

CPU: percentage of CPU usage

Options: the only option available is a button to launch a ping to the server

The monitor environment workspace contains the following buttons at the bottom of the page:

Disconnect: stop monitoring the environment, when clicked (it will not be monitored in the background either)

CPU load

This chart shows the CPU activity of the servers in the environment. You can check/uncheck any server included in the environment to include/exclude it from the chart.

Environment memory use

The Heap chart displays the heap size in MB through time. The heap size of each server in the environment is shown in a different color. As with the other charts, you can check/uncheck any server to include/exclude it from the chart.

Connections

This chart shows the connections established from external clients to the Denodo Platform servers in the environment. The chart displays the open connections in a different color for each server. As with the other charts, you can check/uncheck any server to include/exclude it from the chart.

SYNCHRONIZING AN ENVIRONMENT

Once you have an environment defined and its master node configured, you can synchronize that environment by right clicking on the environment name in the Side tree and selecting the option “Synchronize Environment”. By selecting that option, metadata of the master node will be imported in all the servers of the environment.

The steps performed by the synchronization process are:

Exporting master node metadata: this exported metadata will be necessary to be imported in all the other servers of the environment. An exception at this point will abort the whole process (no synchronization will be possible under this condition) and an error icon will appear next to the master node name

For each server in the environment (except the master one):

Saving a backup of the server metadata (exporting its metadata) so that in case any exception arises during the synchronization of this server, the older status can be recovered. If an exception arises during this step, the synchronization of that server will stop while other servers will not be affected and the whole process will go on. (The progress bar will turn red to let you know there was an exception.)

Synchronizing the environment. That is, importing the master node metadata into a this server. If an exception arises, the previously saved backup will be imported and the process will go on with the next server (the progress bar will turn red).

Below is an image of the synchronization process once finished:

The window above contains three main areas:

Progress bar at the top
List of tasks at the left
Log window at the right

During the execution of the synchronization process, there can be both exceptions and errors. There is an important difference among them. If an exception arises, the progress bar will turn red and the task throwing the exception will be aborted. If there is an error, the task execution will go on and a warning icon will appear to the left of the window so that you know at what point the error happened and go to that part of the log in the window. An error can be a problem with the execution of any sentence contained in the master node export when imported into a server, whereas an exception may be a connection problem to a server node.

Progress bar

The progress bar at the top of the Synchronization window will let you know when the process finishes. If any exception arises, the bar will turn red.

List of tasks

All the different tasks of the current synchronization process will be displayed here. Each task has an icon to the right that lets you know its status:

	Waiting status: the task is waiting to be executed
	Processing: the task is being executed
	Ok: the task has finished correctly (no errors)
	Warning: the task has finished but there has been any error. You should check the log of the task
	Error: the task has not finished due to an exception. You should check the log of the task. In this case, the progress bar will turn red as a task execution has been aborted

Log window

This window displays the logging information of the tasks performed. Any error messages will be displayed in red so that they can be seen easily. Note that every task starts with a bold message. (i.e. “Synchronizing firstserver with master...”, “Saving secondserver backup...”, …)