Denodo Monitor Reports - User Manual

Last modified on: 30 Aug 2021

Introduction

This document contains information to use the Denodo Monitor Reports web application correctly. This document has a functional explanation of every screen of the Denodo Monitor Reports (configuration and the chart explanations).

Denodo Monitor Reports is a web application that displays tables, graphs and charts about the VDP server’s performance. Reports can be constructed to include global summaries of performance by date, by user, by state, etc. Or reports can be assembled by query or cache performance.

The reports are built on the logs generated by The Denodo Monitor tool. While Denodo Monitor Reports tool constructs its reports using the logged information from the Denodo Monitor tool, it is important to distinguish the Denodo Monitor as a separate tool from the Denodo Monitor Reports.

The next section of this guide will give an overview of the Denodo Monitor tool. Then the following sections will detail every aspect of the Denodo Monitor Reports.

denodo-monitor tool

The denodo-monitor is a tool included in the Denodo Platform that logs several parameters of the Denodo servers: Virtual DataPort, Aracne and Aracne Index, ITPilot’s browser pool and Scheduler.

Denodo-Monitor includes several types of “monitors” which obtain information from each server. For example, there is a monitor to store the active processes and connections of the local machine, there is another to log memory resources used by each Java Virtual Machine, and so on.

The denodo-monitor tool is located in the $DENODO_HOME/tools/monitor directory, compressed into a zip file.

There are three different types of “monitors”:

Local monitors: they gather information about the machine where the denodo-monitor is running.

Processes Monitor: this monitor logs information about all the running processes in the local computer. The log file is stored in $DENODO_MONITOR_HOME/logs/processes.log.

Sockets Monitor: this monitor logs the information of the active connections of the local computer. The log file is stored in $DENODO_MONITOR_HOME/logs/sockets.log.

Server monitors: they obtain monitoring information by connecting to a server via JMX [JMX]. Thus, they can obtain this information from remote servers.

Resources Monitor: This monitor logs several parameters of the memory footprint of the server. The log file is stored in $DENODO_MONITOR_HOME/logs/<server name>-resources.log.

Threads Monitor: This monitor logs basic information and CPU usage of every thread of the server. It is very useful to identify VQL queries that consume too much CPU time. The log file is stored in $DENODO_MONITOR_HOME/logs/<server name>-threads.log.

VDP Queries monitor: it logs several parameters of all the VQL queries received by a VDP server. It also obtains this information via JMX, so it can also monitor remote servers. The monitor stores the information contained in these notifications, in $DENODO_MONITOR_HOME/logs/<server name>-queries.log.

This log register can be saved in a database instead of the log file. To configure this database you have to configure the following parameters in the file ConfigurationParameters.properties:

vdpqueries.jdbcagent.enable=true

vdpqueries.jdbcagent.driverClassName: The driver to connect with the database.

vdpqueries.jdbcagent.url: The url to connect with the database where we want

vdpqueries.jdbcagent.user: User to connect with the database.

vdpqueries.jdbcagent.password: Password to connect with the database.

vdpqueries.jdbcagent.password.encrypted: True or false to specified if the password is encrypted.

vdpqueries.jdbcagent.insertStatement: This statement is used to insert correctly the log value.

vdpqueries.jdbcagent.literalQuoteCharacter: Indicate the literal quote character.

vdpqueries.jdbcagent.escapeCharacter: Indicate the escape character.

Cache monitor: it logs several parameters save information about the cache process in the views. The monitor stores the information contained in these notifications, in $DENODO_MONITOR_HOME/logs/<server name>-loadcacheprocesses.log.

This log register can be saved in a database instead of the log file. To configure this database you have to configure the following parameters in the file ConfigurationParameters.properties:

vdploadcacheprocesses.jdbcagent.enable=true

vdploadcacheprocesses.jdbcagent.driverClassName: The driver to connect with the database.

vdploadcacheprocesses.jdbcagent.url: The url to connect with the database where we want

vdploadcacheprocesses.jdbcagent.user: User to connect with the database.

vdploadcacheprocesses.jdbcagent.password: Password to connect with the database.

vdploadcacheprocesses.jdbcagent.password.encrypted: True o false to specified if the password is encrypted.

vdploadcacheprocesses.jdbcagent.insertStatement: This statement is used to instert correctly the log value.

vdploadcacheprocesses.jdbcagent.literalQuoteCharacter: Indicate the literal quote character.

vdploadcacheprocesses.jdbcagent.escapeCharacter: Indicate the escape character.

The Virtual DataPort Administration Guide has more information about the denodo-monitor tool. (Section 18.2 DENODO MONITOR).

Denodo Monitor Reports

The Denodo Monitor Reports is a web application that shows the generated information by the Denodo Monitor. This information is shown in different screens that provide tables and charts about this information. Besides, this application has a configuration screen to configure and connect with different VDP servers.

There is more information about how to install and use the Denodo Monitor Reports in following sections.

Deploy denodo monitor reports

This application can be downloaded through the Denodo Support Site (http://support.denodo.com).

When the zip file is downloaded and unzipped, you can start the application (the distribution includes a jetty web container). To start Denodo Monitor Reports application the start.bat script (start.sh in linux) must be executed.

Note: it is possible to deploy the web application in another web container. You can find DenodoMonitorReports.war file inside $DENODO_MONITOR_REPORTS_HOME/webapps directory. If you are deploying in an external server, you must include $DENODO_MONITOR_REPORTS_HOME/lib/ext/denodo-vdp-jdbcdriver.jar file into the classpath of your application server in order to connect to Virtual DataPort correctly.

Then, you can access into the application using http://host:8080/DenodoMonitorReports (by default, http://127.0.0.1:8080/DenodoMonitorReports).

Note: you can change the default jetty port if it was used by another application. To change the value you should open $DENODO_MONITOR_REPORTS_HOME/etc/jetty.xml and modify the entry:

To correctly use this application, the architecture must have the following requirements:

The Denodo Monitor Reports tool connects to any VDP 5.5, 6.0 or 7.0 servers (local or remote). The servers must be configured inside configuration option in Denodo Monitor Report application (see next section).

The Denodo Monitor Reports application distributes the Denodo JDBC driver. The version of this driver is latest version available when the distribution of Denodo Monitor Reports was created. This means if you are using a different version of VDP, the vdp-jdbc-driver of that version should be installed in $DENODO_MONITOR_REPORTS_HOME/lib/ext/ folder. If not, errors can appear in the application.

An instance of Denodo Virtual DataPort is necessary. Denodo Monitor Reports application connects to a database in the VDP server. Consequently, the current VDP server could be affected. So in an ideal scenario a different than the production server would be used for the Denodo Monitor Reports. However, separate servers are not a requirement.

Denodo Monitor Reports connects to a Virtual DataPort database that has a complete set of views and web service that read the denodo-monitor logs. So, the Virtual DataPort server needs access to the denodo-monitor logs.

The following illustration shows a ideal architecture to use this web application:

Deploy DMR in the Denodo Embedded Web Container

Also you can deploy the web application in the internal web container of the Denodo Platform.

You can find DenodoMonitorReports.war file inside $DENODO_MONITOR_REPORTS_HOME/webapps directory.

You should follow the following steps:

Copy the DenodoMonitorReports.war into the following location: <DENODO_HOME>/resources/apache-tomcat/webapps/

Create a context xml for the Denodo Monitor Reports. You can use the xml template $DENODO_MONITOR_REPORTS_HOME/files-for-denodo-container/DenodoMonitorReports.xml.

Copy the DenodoMonitorReports.xml file into the following folder: <DENODO_HOME>/resources/apache-tomcat/conf/DenodoPlatform-<VERSION/localhost

Note that <VERSION> is a placeholder for the specific version of the Denodo Platform you are installing the Denodo Monitor Reports for.

Check in <DENODO_HOME>/resources/apache-tomcat/conf/catalina.properties file that the common.loader property (shared.loader in Denodo 5.5) includes a reference to the VDP JDBC Driver.

If missing, add it:

${catalina.base}/../../tools/client-drivers/jdbc/denodo-vdp-jdbcdriver.jar for Denodo Platform 7.0
${catalina.base}/../../tools/client-drivers/jdbc/denodo-vdp-jdbcdriver-basic.jar for Denodo Platform 6.0
${catalina.base}/../../lib/vdp-jdbcdriver-core/denodo-vdp-jdbcdriver-basic.jar for Denodo Platform 5.5

Create launch scripts for the DenodoMonitorReports. You can use the templates dmr_startup.bat or dmr_shutdown.bat for windows or the templates dmr_startup.sh dmr_shutdown.sh located at $DENODO_MONITOR_REPORTS_HOME/files-for-denodo-container/.

Make sure to modify the DENODO_HOME variable in the script templates to point to your Denodo installation.

Copy the launch scripts into <DENODO_HOME>/bin.

After copying all the necessary files into the correct directories, run the <DENODO_HOME>/bin/dmr_startup.bat(.sh) launch script again and then navigate to:

http://yourlocalhost:9090/DenodoMonitorReports

Denodo Monitor Reports overview

Denodo Monitor Reports provides graphic information reading all the entrances created by denodo-monitor tool. Its aim is the analysis of how performance evolves over time. For these analyses, denodo-monitor exposes server-side parameters (e.g., number of connections, requests, memory usage, number of threads, ...), along with statistics on query performance (execution time, number of rows, queries by type of CRUD operation invoked, use of cache, ...).

Administrators can combine intelligence gathered from all of these monitoring sources to analyze the optimum performance of the instances of Denodo Virtual DataPort.

Denodo Monitor Report has three options in its main menu:

Configuration: choose this option to configure, add, delete or update different VDP servers.

Reports: in this option there are different graphics that are divided in different sections (global, specific-view, cache and performance information).

Guide: in this option you can read the user guide.

We are going to explain every option deeply.

Configuration

Authentication

The authentication of the Denodo Monitor Reports is against a VDP server. This authentication server is configured the first time that you access to the application. The privileges of the VDP user determines if it only can visualize the reports of the application or if it can also configure more servers in DMR.

First-time access

When you access the first time to the application, there aren’t any VDP servers configured, so there is no authentication server configured. If you access the application at http://localhost:8080/DenodoMonitorReports/ when there are no configured servers (and therefore no authentication server), a Wizard for configuring the server connection is launched. This first configured server will be automatically configured as the authentication server. The next time that you access the application, you will have to login against that VDP server.

Once you click on the “Apply configuration” button, you will be redirected to the login page and after signing in (using credentials valid in the configured authentication server) you can start using the application.

Login

Once you have configured at least one server, accessing Denodo Monitor Reports will require authentication with credentials valid at the server configured as “authentication server”.

In order to access Denodo Monitor Report’s configuration section, the user in VDP has to be admin user or a normal user with administrator privileges over the configured authentication server’s database. In other case the user can only access the reports, but not configure the application.

Select authentication server

You can change the authentication server in the Configuration section. Click on Select Authentication Server and a modal window will be displayed with the configured servers.

If you select a new server among the configured ones and you click Accept, authentication operations will be made against that VDP server from then on. Be careful, after changing the server and logging out of the application, you can't change the authentication server if you cannot connect to the new server and have administrator privileges on the new server.

Wizard

The first time that the Denodo Monitor Reports is executed, a wizard for configuring the connection server is launched.

This wizard will guide you through the steps to setup a connection to your VDP server. In this first screen the alias name can be configured.

In this screen the configuration of server connection can be configured. The following fields have to be filled:

VDP Server Name or IP: IP address or Server name to connect to.

VDP Server Port Number: by default, this value is 9999.

Embedded Tomcat Port: by default, this value is 9090. A JSON web service will be deployed in the internal Tomcat server. It will be used by the application to draw the charts, so connection is needed. If you select “https” as connection type, this value by default should be 9443.

Tomcat Connection Type: this value can be “http” or “https”.

Database Name: this database has to be created previously in the VDP monitorization server. Inside this database, a complete set of views and a web service will be created to read the monitorization inputs.

Create Database in VDP: if this option is checked the new database will be created in VDP. In this case, the configured user needs global administration privileges in VDP.

VDP server TimeZone: the timezone of the monitorization VDP server can be configured with this field.

User name: user name to connect with the monitorization VDP server. This user has to be a “local administrator” of the database typed in the previous input.

Password: password to connect with the monitorization VDP server.

Username WS: user name to authenticate the exposed web service to connect with the monitorization VDP server. This field is not mandatory, but for security if the field is empty the default authentication user is “dmr”.

Password WS: password to authenticate the exposed web service to connect with the monitorization VDP server. This field is not mandatory, but for security if the field is empty the default authentication user is “dmr”.

Cors Origin URL’s: this is the allowed URL’s configured in the web service. By default the creation form suggest the client origin, but is possible to change or add more origins (separated by commas).
Web content's origin is defined by the scheme (protocol), host (domain), and port of the URL used to access it. Two objects have the same origin only when the scheme, host, and port all match.
DMR web services are created with CORS enabled by default. The web service must define the allowed origins to return the Access-Control-Allow-Origin header in the responses.
In requests without credentials CORS can use * as origin, but our web services are authenticated and it is mandatory to define the allowed origin URL’s.

We shouldn't use http://localhost:9090/DenodoMonitorReports to access to DMR. Requests will result in a 403 forbidden response. We have to use the IP or the DNS of the machine instead of localhost. Starting with the update 8.0u20210705, the web container of Denodo refuses HTTP requests with the method OPTIONS. In order to use DMR, this configuration is required.

VDP Load Cache File: filename where the information about load cache processes logging is stored. By default vdp-loadcacheprocesses.log

VDP Queries File: filename where the information about VDP queries logging is stored. By default vdp-queries.log.

VDP Resources File: filename where the information about VDP resources logging is stored. By default vdp-resources.log.

In this screen you can configure the following parameters:

Log file path: The path of the logs can be changed.

Queries report configuration: The source of the queries information can be changed. If you select “LOG FILES” the data of the queries reports will be read from denodo-monitor logs files. If denodo-monitor tool stores the queries information in a database, the database parameters must be configured.

The following parameters have to be filled, if the option “Activate database configuration in Queries Reports” is checked to configure the database connection:

Database adapter: the adapter to connect to Database Management Systems.

Driver class path: the JDBC driver file or folder where the driver is stored.

Driver class name: the driver class name to connect with the database.

Connection URL: database url to establish the connection.

Table name: the database table name to get the information.

User name and password: login information to connect to the database.

Encrypted password: if the password is encrypted, this field has to be checked.

Note: For use the Denodo Monitor with database instead of logs: first of all you have to create the tables to store the logs, in the folder sql you have several sql files to create the tables for several DBMS. After you have to set the file ConfigurationParameters.properties and activate the line vdpqueries.jdbcagent.enable=true, and configure the JDBC Agent Parameters depending on the DBMS, you can find in the file sections of commented configured parameters for several DBMS which name starts by vdpqueries.jdbcagent like vdpqueries.jdbcagent.driverClassName.

Cache report configuration: The source of the cache information can be changed. If you select “LOG FILES” the data of the cache reports will be read from the log files. If denodo-monitor saves the cache information in a database, the database parameters must be configured.

The following parameters have to be filled, if the option “Activate database configuration in Cache Reports” is checked, to configure the database connection:

Database adapter: the adapter to connect to Database Management Systems.

Driver class path: the JDBC driver file or folder where the driver is stored.

Driver class name: the driver class name to connect with the database.

Connection URL: database url to establish the connection.

Table name: the database table name to get the information.

User name and password: login information to connect to the database.

Encrypted password: if the password is encrypted, this field has to be checked.

Note: For use the Denodo Monitor with database instead of logs: first of all you have to create the tables to store the logs, in the folder sql you have several sql files to create the tables for several DBMS. After you have to set the file ConfigurationParameters.properties and activate the line vdploadcacheprocesses.jdbcagent.enable=true, and configure the JDBC Agent Parameters depending on the DBMS, you can find in the file sections of commented configured parameters for several DBMS which name starts by vdploadcacheprocesses.jdbcagent like vdploadcacheprocesses.jdbcagent.driverClassName.

When every section is correctly configured and the button next is clicked, the application shows the following alert:

If you click “OK” the VQL scripts will be installed, configuring the VDP structure. Finally, the next screen will be showed:

Note: If you are using Denodo 7.0 or higher the Queries report configuration and Cache report configuration using log files is deprecated. In this case the application shows a warn message that inform you that the recommended configuration is through DATABASE.

Configuration screen

The reports configuration screen has two different sections:

List of Configured Servers: this section is used to configure the connection to the monitorization servers.

In this section we can see the following information:

Alias: Server alias.

Available: this icon indicates if the VDP server is available.

Configured: this icon indicates if the webservice to get the data is available, therefore, if the VDP configuration is correctly created.

The meaning of every icon:

Webservice is available.

Webservice is not available.

The server configuration has been modified but the VQL script hasn’t been executed.

VDP server: hostname and port of the VDP server.

Conn. URL: the url to connect to the data web service.

Create new server using Wizard: same steps that the Wizard section explains step by step.

Every configured server can be edited and deleted (Note: it is not allowed to delete the Default server).

Clicking on Edit button the following screen is opened:

In this screen you can see the configuration values of the server. These values can be edited.

If “Edit” button is selected in Server Information section, the following screen is shown:

The fields that can be changed in this screen were described in section “4.1 Wizard”.

In Reports Configuration section the source information to create the reports can be configured:

Log file path: The path of the logs can be changed.

Queries report configuration: The source of the queries information can be changed. If you select “LOG FILES” the data of the queries reports will be read from the logs files. If denodo-monitor stores the queries information in a database, the database parameters must be configured.

Cache report configuration: The source of the cache information can be changed. If you select “LOG FILES” the data of the cache reports will be read from the logs files. If denodo-monitor tool saves the cache information in a database, the database parameters must be configured.

Note: if you edit any parameter in this screen, you will have to click on “Execute VQL script” to change the configuration in the VDP monitorization server. If you don’t execute the VQL script, you can see a warning icon in the configuration screen:

In the “List of configured servers” page you will find a “Create new server” button. It allows us to create a new server without the help of the wizard. When this button is clicked we can see the following screen:

The fields that have to be filled were described in section 4.1 Wizard.

When the server configuration is created, we can see in the server list, but the VQL scripts are not executed (the server will be shown as ‘not configured’ in the servers’ list). To execute these scripts, you have to click on “Edit”, configure the logs path and finally click on “Execute VQL script”.

Set the Active Server

In the application we can see the active server in the upper right of the page. This is the server used to get the data to paint the monitorization reports.

The active server can be changed clicking over the name. By default, the Default server is selected.

The reports will be shown depends on the selected server.

Note: This option is available in any report page, and can be changed when the user wants to.

Configure DMR with logs of Solution Manager monitor.

Denodo Monitor reports can read the logs created when you monitor a server from Solution Manager. These logs have a special condition, they are stored in different folders, a new session of monitoring creates a folder whose name is the timestamp.

To read the logs of SolutionManager Denodo you have to fill in the parameters VDP Load Cache File, VDP Queries File, VDP Resources File correctly. Be aware the log files are created with the name of the server name chosen in Solution Manager Administration Tool: serverVDPName-loadcacheprocesses.log, serverVDPName-queries.log, serverVDPName-resources.log. In addition, you have to fill in the parameter log path with a folder of one of the sessions of the monitorization. Note that the route to the logs of Solution Manager Monitor should be similar to this: /../SolutionManager/resources/solution-manager-monitor/work/enviromentName/clusterName/serverName/20191113141155/logs/.

Denodo Monitor Reports will show the information generated by Solution Manager Administration Tool in the different sessions.

Reports overview

In this option there are different charts that are divided in five different sections:

Global Summary

Performance Summary

Reports by Query

Reports by User

Reports by Cache

Note that the Performance Summary reports appears as deprecated in the Denodo 7.0 and later versions.

Reports by Agent only appears in the Denodo 7.0 and later versions. In previous versions only appears the Reports by user.

Global Summary reports

This section shows a summary about the VDP server behavior. We are going to see every chart in detail.

Summary by date

This screen shows two different charts:

Number of queries: this chart shows the number of executed queries in VDP by day.

Number of returned rows: this chart shows the number of returned rows in every executed query by day.

In the table, we can see a summary about the previous charts. Every value is divide by day. The following fields are shown:

Date

Count: This is the number of executed queries, which is shown in the first chart.

Avg duration: Average time that a query needs to be executed.

Avg waiting time: Average time that a query has to wait.

# rows: Number of total returned rows.

Avg # rows: Average number of rows that are returned.

# completed: Number of completed queries correctly.

# cached: Number of queries that have been cached.

Summary by database

This screen shows two different charts:

Number of queries: this chart shows the number of executed queries in VDP by every database that is accessed in the query. A query could access to several databases.

Percent of queries: this chart shows the same information than the previous chart but with a different chart.

Note: The pie chart groups the results and doesn’t show a result if the percent is less than 10%.

In the table, we can see a summary about the previous charts. In this case, we can see the divided results by database. The following fields are shown:

Database

Count: This is the number of executed queries by every database.

Avg duration: Average time that a query needs to be executed by every database.

Avg waiting time: Average time that a query has to wait by every database.

# rows: Number of total returned rows by every database.

Avg # rows: Average number of rows that are returned by every database.

# completed: Number of completed queries correctly by every database.

# cached: Number of queries that have been cached by every database.

Summary by user

This screen shows two different charts:

Number of queries: this chart shows the number of executed queries in VDP by every user

Perecent of queries: this chart shows the same that the previous chart but with a different chart.

Note: The pie chart groups the results and doesn’t show a result if the percent is less than 10%.

In the table, we can see a summary about the previous charts. In this case, we can see the divided results by user. The following fields are shown:

User name

Count: This is the number of executed queries by every user.

Avg duration: Average time that a query needs to be executed by every user.

Avg waiting time: Average time that a query has to wait with every user.

# rows: Number of total returned rows by every user.

Avg # rows: Average number of rows that are returned that are executed by every user.

# completed: Number of completed queries correctly by every user.

# cached: Number of queries that have been cached by every user.

Summary by cache

This screen shows two different charts:

Number of queries: this chart shows the number of executed queries in VDP that were cached.

Percent of cached queries: this chart shows the same that the previous chart but with a different chart.

Note: The pie chart groups the results and doesn’t show a result if the percent is less than 10%.

In the table, we can see a summary about the previous charts. In this case, we can see the results depending on if there were cached or not. The following fields are shown:

Cache

Count: This is the number of executed queries.

Avg duration: Average time that a query needs to be executed.

Avg waiting time: Average time that a query has to wait with every user.

# rows: Number of total returned rows by every user.

Avg # rows: Average number of rows that are returned that are executed by every user.

# completed: Number of completed queries correctly by every user.

# cached: Number of queries that have been cached by every user.

Summary by state

This screen shows two different charts:

Number of queries: this chart shows the number of executed queries with different results.

Percent of queries: this chart shows the same that the previous chart but with a different chart.

Note: The pie chart groups the results and doesn’t show a result if the percent is less than 10%.

In the table, we can see a summary about the previous charts. In this case, we can see the divided results by state. The following fields are shown:

State.

Count: This is the number of executed queries by every state.

Avg duration: Average time that a query needs to be executed by every state.

Avg waiting time: Average time that a query has to wait depending on the state.

# rows: Number of total returned rows by every state.

Avg # rows: Average number of rows that are returned by state.

# completed: Number of completed queries correctly by state.

# cached: Number of queries that have been cached by state.

Summary by request type

This screen shows two different charts:

Number of queries: this chart shows the number of executed queries. These results are divided by type of query.

Percent of queries: this chart shows the same that the previous chart but with a different chart.

Note: The pie chart groups the results and doesn’t show a result if the percent is less than 10%.

In the table, we can see a summary about the previous charts. In this case, we can see the divided results by type of query. The following fields are shown:

Request type

Count: This is the number of executed queries by every type.

Avg duration: Average time that a query needs to be executed by every type.

Avg waiting time: Average time that a query has to wait depending on the type.

# rows: Number of total returned rows by type.

Avg # rows: Average number of rows that are returned by type.

# completed: Number of completed queries correctly by type.

Summary by access interface

This screen shows two different charts:

Number of queries: this chart shows the number of executed queries. These results are divided by type of client that performed the request.

Percent of queries: this chart shows the same that the previous chart but with a different chart.

Note: The pie chart groups the results and doesn’t show a result if the percent is less than 10%.

In the table, we can see a summary about the previous charts. In this case, we can see the divided results by type of client that performed the request. The following fields are shown:

Access interface

Count: This is the number of executed queries by every type of client that performed the request.

Avg duration: Average time that a query needs to be executed by every type of client that performed the request.

Avg waiting time: Average time that a query has to wait depending on the type of client that performed the request.

# rows: Number of total returned rows by type of client that performed the request.

Avg # rows: Average number of rows that are returned by type of client that performed the request.

# completed: Number of completed queries correctly by type of client that performed the request.

Summary by User-Agent

This screen shows two different charts:

Number of queries: this chart shows the number of executed queries in VDP by every User-Agent

Percent of queries: this chart shows the same that the previous chart but with a different chart.

Note: The pie chart groups the results and doesn’t show a result if the percent is less than 10%.

In the table, we can see a summary about the previous charts. In this case, we can see the divided results by User-Agent. The following fields are shown:

User-Agent.

Count: This is the number of executed queries by every user-agent.

Avg duration: Average time that a query needs to be executed by every user-agent.

Avg waiting time: Average time that a query has to wait with every user-agent.

# rows: Number of total returned rows by every user-agent.

Avg # rows: Average number of rows that are returned that are executed by every user-agent.

# completed: Number of completed queries correctly by every user-agent.

# cached: Number of queries that have been cached by every user-agent.

Note: This report is only included in the Denodo 7.0 and later versions.

Summary by Client IP

This screen shows two different charts:

Number of queries: this chart shows the number of executed queries in VDP by every Client IP

Percent of queries: this chart shows the same that the previous chart but with a different chart.

Note: The pie chart groups the results and doesn’t show a result if the percent is less than 10%.

In the table, we can see a summary about the previous charts. In this case, we can see the divided results by Client IP. The following fields are shown:

Client IP.

Count: This is the number of executed queries by every Client IP.

Avg duration: Average time that a query needs to be executed by every Client IP.

Avg waiting time: Average time that a query has to wait with every Client IP.

# rows: Number of total returned rows by every Client IP.

Avg # rows: Average number of rows that are returned that are executed by every Client IP.

# completed: Number of completed queries correctly by every Client IP.

# cached: Number of queries that have been cached by every Client IP.

Note: This report is only included in the Denodo 7.0 and later versions.

Performance Summary

This section shows information about the server performance. We are going to see every part in more detail.

Load (incoming requests)

This chart shows information about the load of the system. We can see information about the incoming requests.

This first chart shows information day by day. We can see the chart with more detail clicking over “hour-by-hour” option or “minute by minute” option. Also, we can interact with every chart. We can do zoom in on the chart and see the chart with more detail.

Note: This report is marked as deprecated in the Denodo 7.0 and later versions.

Open Connections

This chart shows information about the number of connections in every instant.

We can interact with this chart. We can do zoom in on the chart and see the chart with more detail.

The table shows a summary about the previous information. We can see the following fields:

Date

Active connections

Total connections

Note: This report is marked as deprecated in the Denodo 7.0 and later versions.

Execution Threads

This chart shows information about the loaded threads in every instant.

We can interact with this chart. We can do zoom in on the chart and see the chart with more detail.

The table shows a summary about the previous information. We can see the following fields:

Date

Threads

Peak threads

Note: This report is marked as deprecated in the Denodo 7.0 and later versions.

Memory consumption

This chart shows information about the state of the memory in every instant. The table shows a summary about the previous information.

Note that if you are using Denodo 7.0 or later this report have a little difference because in Java 8, the 'Perm Gen' memory space is removed and appears the 'Metaspace'.

We can interact with this chart. We can add or remove the information with the checkboxes. Also, we can do zoom in on the chart and see the chart with more detail.

In this screen, there are three different sections:

Heap Memory Usage: shows information about the heap memory, reserved memory and maximum memory in every instant.

Non-heap Memory Usage: shows information about Non-heap memory and maximum memory in every instant.

Instant Loaded Classes: shows information about the loaded classes and about the total classes.

In every chart, there is the same table with summary information. The following fields are shown:

Date

Heap (KB): (Heap memory) The heap memory is the runtime data area from which the Java VM allocates memory for all class instances and arrays.

Non-heap (KB): Non-heap memory includes a method area shared among all threads and memory required for the internal processing or optimization for the Java VM.

Eden (KB): (Eden space) The pool from which memory is initially allocated for most objects.

Survivor (KB): (Survivor space) The pool containing objects that have survived the garbage collection of the Eden space.

Tenured (KB): (Tenured Generation) The pool containing objects that have existed for some time in the survivor space.

Perm (KB): (Permanent Generation) The pool containing all the reflective data of the virtual machine itself, such as class and method objects. With Java VMs that use class data sharing, this generation is divided into read-only and read-write areas.

Code cache (KB): The HotSpot Java VM also includes a code cache containing memory that is used for compilation and storage of native code.

Loaded class count: Number of loaded classes.

Total loaded class count: Number of total loaded classes.

Note that if you are using Denodo 7.0 or later this report have a little difference because in Java 8, the 'Perm Gen' memory space is removed and appears the 'Metaspace'.

Note: This report is marked as deprecated in the Denodo 7.0 and later versions.

CPU and GC usage

This chart shows information about the usage of the CPU and the Garbage Collector in every instant. The table shows a summary about them.

We can interact with this chart. We can add or remove the information with the checkboxes. Also, we can do zoom in on the chart and see the chart with more detail.

The table shows a summary about the previous information. We can see the following fields:

Date

CPU (%): The CPU usage by the platform in every instant, expressed in percentage.

GC (%): (Garbage Collector) the CPU usage by the garbage collector in every instant, expressed in percentage

Note: This report is marked as deprecated in the Denodo 7.0 and later versions.

Reports by View

In this section, we can see in detail the results of every view. To see a view in detail, we have to introduce the name of the view in the report menu, click over the icon and select the wanted chart. A query could access to several views, in this case one query would generate several entries in these charts.

These charts can be accessed by clicking over every view in the Performance analysis screen.

Global views performance

This screen shows three charts and three tables with the global views.

The first chart shows the number of queries execute by every view. The associated table shows a summary of every view:

VDP view: The view name.

Count: Number of total executed queries.

# rows: Number of returned rows.

# completed: Number of completed queries.

# cached: Number of cached queries.

In this table we have a couple of fields that show information about cache and index optimizations for each view:

Activate Cache: The application has the information about the number of queries and the average duration for each view. The field checks if this values exceeds a limit and in this case, make a recommendation to the user. In case this field has a 'true' value, it might be a good idea to activate the cache in that view.

Create index: The application has the information about the number of rows and the minimum duration of each query. The field checks if this values exceeds a limit and in this case, make a recommendation to the user. In case this field has a 'true' value, it might be a good idea to create an index in that column at the data source (or in the cache if that view is being cached).

Note: The fields “Activate Cache” and “Create index” are marked as deprecated in the Denodo 7.0 and later versions.

The second chart shows the time (minimum, maximum and average) that a query spends when it is executed; these values are divided by every view. The associated table shows the same information as the chart.

The third chart shows the number of rows (minimum, maximum and average) that are returned by every view. The associated table shows the same information as the chart.

View detail

This screen shows a detailed information of the queries of the selected view with the following fields:

Req. Date: Request date.

Query

Duration: Duration in the request.

# rows: Rows in the request.

Completed: True if the query has completed correctly.

Cached: True if the query has accessed to cached data.

View timeline

When you click over an anchor in the previous screen (or the name of the view is introduced in the reports menu), the following chart is shown:

This chart shows the behavior of every request in a view. We can see different information in this chart about the selected view:

Incoming request

Completed request

Cached request

We can interact with this chart. We can add or remove the information with the checkboxes. Also, we can do zoom in on the chart and see the chart with more detail.

The reset button is used to see the charts with the original information.

In the table, we can see a summary about the previous information. The following fields are shown:

Date

Count: This is the number of executed queries of the selected view.

Avg duration: Average time that a query needs to be executed of the selected view.

Avg waiting time: Average time that a query has to wait of the selected view.

# rows: Number of total returned rows of the selected view.

Avg # rows: Average number of rows that are returned of the selected view.

# completed: Number of completed queries correctly of the selected view.

# cached: Number of cached queries correctly of the selected view.

View by type

This screen shows three charts with information about the selected database and view. We are going to see every chart in detail:

Completed/Non completed: This chart shows the percent of completed and non-completed queries.

Cached/Non cached: This chart shows the percent of cached and non-cached queries.

Query type: This chart shows the percent of every query type.

In the table, we can see a summary about the previous information. The following fields are shown:

Request type: Select, update, insert…

Count: Total number of requested queries.

# completed: Number of completed queries.

# cached: Number of cached queries.

View performance

This screen has two different parts. The first part has three different charts that show the different between the cached and non-cached queries of the selected database and view. And the second part shows the different between the type requests of the selected view.

This first part shows the following charts:

The first chart shows information about the different duration between cached and non-cached queries.

The second one shows information about the different number of returned rows.

The pie chart shows the percent of cached and non-cached queries.

The table shows information about the previous information. The following fields are shown:

Cached: True or false if the view was cached.

Count: Total number of requested queries.

Max duration: Maximum time that a query needs to be executed.

Min duration: Minimum time that a query needs to be executed.

Avg duration: Average time that a query needs to be executed.

Max # rows: Maximum number of rows that are returned.

Min # rows: Minimum number of rows that are returned.

Avg # rows: Average number of rows that are returned.

# rows: Total number of return rows in every cached/non-cached queries.

This second part shows information about the request type of the selected database and view. There are three charts to show this information.

The first chart shows information about the duration (Max, Min and Avg) depending on the request type in the selected view.

The second chart shows information about the returned rows (Max, Min and Avg) depending on the request type in the selected view.

The pie chart shows information about the percent in every request type.

The table shows information about the previous information. The following fields are shown:

Request type

Count: Total number of requested queries in every request type of the selected view.

Max duration: Maximum time that a query needs to be executed of every request type.

Min duration: Minimum time that a query needs to be executed of every request type.

Avg duration: Average time that a query needs to be executed of every request type.

Max # rows: Maximum number of rows that are returned of every request type.

Min # rows: Minimum number of rows that are returned of every request type in the selected view.

Avg # rows: Average number of rows that are returned of every request type.

# rows: Total number of return rows of every request type.

View by conditions

This chart shows information about the different query condition of the selected database and view.

The table shows a summary about the previous information:

Query

Count: Total number of requested queries of every query.

View by user

This chart shows information about the percent of request of every user of the selected database and view.

The following table shows information about the previous information. The following fields are shown:

User

Executions: Number of times that the query has been executed.

Avg duration: Average duration in a request.

Avg # rows: Average rows in every request.

# completed: Number of completed requests.

# cached: Number of cached requests.

View by access interface

This chart shows information about the percent of request of every type of client that performed the request of the selected database and view.

The following table shows information about the previous information. The following fields are shown:

Access interface

Executions: Number of times that the query has been executed.

Avg duration: Average duration in a request.

Avg # rows: Average rows in every request.

# completed: Number of completed requests.

# cached: Number of cached requests.

View by User-Agent

This chart shows information about the percent of request of every User-Agent that performed the request of the selected database and view.

The following table shows information about the previous information. The following fields are shown:

User-Agent

Executions: Number of times that the query has been executed.

Avg duration: Average duration in a request.

Avg # rows: Average rows in every request.

# completed: Number of completed requests.

# cached: Number of cached requests.

Note: This report is only included in the Denodo 7.0 and later versions.

View by Client IP

This chart shows information about the percent of request of every Client IP that performed the request of the selected database and view.

The following table shows information about the previous information. The following fields are shown:

Client IP

Executions: Number of times that the query has been executed.

Avg duration: Average duration in a request.

Avg # rows: Average rows in every request.

# completed: Number of completed requests.

# cached: Number of cached requests.

Note: This report is only included in the Denodo 7.0 and later versions.

Reports by Cache

In this section we can see in detail the results of every view. To see a view in detail, we have to introduce the name of the view in the report menu, click over the icon and select the wanted chart.

Beside, these charts can be acceded clicking over every view in the Performance analysis (Global cache performance) screen.

Performance analysis

This screen shows three charts and three tables with the global queries.

The first chart shows the number of refresh cache process executed by every view. The associated table shows a summary of every view:

VDP view: The view name.

Count: Number of total refresh cache processes.

Full load: Number of full load processes.

Partial load: Number of partial load processes.

Partial load: Match exact query: Number of partial load matching query processes.

Partial load: Explicit loads: Number of partial load with explicit load processes.

Partial loads: Match exact query and explicit loads: Number of partial load matching query and explicit load processes.

The second chart shows the data storage time (minimum, maximum and average) that spends when cache process is executed; these values are dividied by every view. The associated table shows the following information:

VDP view: The view name.

Count: Number of total refresh cache process.

Max Data Storage time.

Min Data Storage time.

Avg Data Storage time.

Max Metadata Storage time.

Min Metada Storage time.

Avg Metada Storage time.

The third chart shows the number of rows (minimum, maximum and average) that are inserted in every process. The associated table shows the following information:

VDP view: The view name.

Count: Number of total refresh cache process.

Min inserted rows.

Max inserted rows.

Avg inserted rows.

Max received rows.

Min received rows.

Avg received rows.

Cache queries

This screen shows a detailed information of the cached queries of the selected view with the following fields:

Req. Date: Request date.

Database

VDP view

Conditions

Data storage time

Inserted rows

Received rows

Load type: The cache type.

FULL
PARTIAL
PARTIAL EXACT
PARTIAL PRELOAD
PARTIAL EXACT PRELOAD

Cache timeline

When you click over an anchor in the previous screen (or the name of the view is introduced in the reports menu), the following chart is shown:

This chart shows the behavior of every request in a view. We can see different information in this chart about the selected view:

Cache processes

Full load cache

Partial load cache

We can interact with this chart. We can add or remove the information with the checkboxes. Also, we can do zoom in on the chart and see the chart with more detail.

The reset button is used to see the charts with the original information.

In the table, we can see a summary about the previous information. The following fields are shown:

Date

Count: This is the number of executed cache processes of the selected view.

Avg data storage time

Avg # of inserted rows

Avg # of received rows

Full load: Number of full load cache.

Partial load: Number of partial load cache.

Cache by type

This screen shows two charts with information about the selected view. We are going to see every chart in detail:

Full/Partial Load cache: This chart shows the percent of full/ partial load cache.

Type load cache: This chart shows the percent of every load cache type.

In the table, we can see a summary about the previous information. The following fields are shown:

Cache Status: Full, Partial, Partial exact, Partial preload and Partial exact preload

Count: Total number of load.

Avg storage time

Avg # inserted rows.

Avg # received rows.

Cache performance

This screen has three different charts that show the differences between every type of cache load in the selected view.

This first part shows the following charts:

The first chart shows information about the storage time depends on the type of load cache.

The second one shows information about the number of inserted rows, depends on the type of load cache.

The pie chart shows the percent of every type of load cache.

The table shows information about the previous information. The following fields are shown:

Type

Count: This is the number of executed cache processes of the selected view.

Max data storage time

Min data storage time

Avg data storage time

Max # of inserted rows

Min # of inserted rows

Avg # of inserted rows

Full load: Number of full load cache.

Partial load: Number of partial cache loads.

Cache conditions

This chart shows information about the different condition in the load cache of the selected view.

The table shows a summary about the previous information:

Query

Count: Total number of queries.

Cache by database

This chart shows information about the percent of every database of the selected view where the load cache is executed.

The following table shows information about the previous information. The following fields are shown:

Database

Count

Avg storage time

Avg # inserted rows

Avg # received rows

Reports by Agent

In this section, we can see in detail the actions of every agent. To see a agent in detail, we have to select the VDP Agent and enter the name of the agent in the reports main menu, click on the right icon and select the wanted chart.

For Denodo 7.0 and later versions we have three VDP Agent options (User, User-Agent, Client IP and Access Interface). In previous versions we only have the user option.

Agent queries

This screen shows detailed information of the queries executed by a determined agent. It is showed the same information than in the report Query detail (see section 9.2 View detail), but by a specific user.

Agent timeline

This chart shows the behavior of a user in the timeline. It shows the same information that the report Query Timeline (see section 9.3 View Timeline), but for the selected agent. Also, it is possible view the same chart filtering the View Timeline report by user.

Agent by view

This screen shows three charts and three tables with the queries that a specific agent has performed. This screen shows the same charts and tables that Global queries performance (see section 9.1 Global views performance), but for the selected agent. Also, it is possible view the same chart filtering the Global Views Performance report by user.

Agent by query

This chart shows information about the different query condition of the queries executed by a determined agent. It is showed the same information than in the report Query conditions (see section 9.6 View by query), but by a specific agent.

Agent by database

In this screen, you can see the databases accessed by a determined agent. It is showed the same information than in the report Summary by database (see section 7.2 Summary by database), but by a specific agent.

Filter option

Filter By Date

In some reports, there is a form to filter the data by date (see next screenshot).

In this form, there are two dates to fill. The first date is the “start date” and the second one is “end date”.

To filter by date, the form should be filled at least in one date (e.g; you can insert a start date only, an end date only or both at the same time).

The hours and minutes are not mandatory. If the time is not filled, by default the value is 00:00.

Filter by user

In the Queries section, there is a form to filter the data by user, besides by date(see next screenshot).

You can combine the filter by user with the filter by date, so you can obtain information about what a particular user made on specific dates.

Filter by access interface

In the Queries section, there is a form to filter the data by access interface, besides by date and by user, in order to get information about the type of client that performed the request.

You can combine the filter by access interface with the filter by date and with the filter by user.

User Guide

To download and open this document you only have to click on the “User Guide” link.