USER MANUALS

Configuration Files of the Monitoring in Solution Manager

Solution Manager includes a Denodo Monitor that you can launch to collect the execution logs from a single Virtual DataPort server, or from all the servers of a cluster or environment.

This appendix explains how to manually configure it, which is not recommended because it is error-prone. Use the Solution Manager to graphically create a global monitoring configuration (Configuration > Monitoring configuration) or per environment monitoring configuration (open the environment and go to the Monitoring section).

There are two configuration template files in the Solution Manager which allow to manually configure its Denodo Monitor. These files are located in the folder <SOLUTION_MANAGER_HOME>/conf/solution-manager/denodo-monitor:

  1. ConfigurationParametersGeneral.template: general configuration template file.

  2. ConfigurationParametersServer.template: configuration template file with specific server properties (ping timeout, monitors, list of data sources to ping…).

Solution Manager generates a configuration file for each monitoring execution based on these template files. The generated file is called ConfigurationParameters.properties and it is stored in the conf folder, in the working folder of the monitored element.

JDBC Logging Configuration

You can configure the Denodo Monitor to store the information generated by the following monitors in a database:

  • Queries monitor

  • Cache monitor

  • Connections monitor

  • Data Sources monitor

  • Resources monitor

  • Threads monitor

When you enable this feature, Denodo Monitor will keep storing this information in the log files.

To enable this feature, follow these steps:

  1. Copy the JDBC driver of the database (i.e. its jar file(s)) to the folders <SOLUTION_MANAGER_HOME>/resources/solution-manager-monitor/denodo-monitor/lib and <SOLUTION_MANAGER_HOME>/lib/solution-manager-extensions (see section Monitor Tables Automatic Creation for further detail).

  2. Go to the folder <SOLUTION_MANAGER_HOME>/resources/solution-manager-monitor/denodo-monitor/sql and pick the script of the database where you want the logs to be stored.

  3. Execute this SQL script on the database. This script creates the tables REQUEST_NOTIFICATION, CACHE_NOTIFICATION, CONNECTION_NOTIFICATION, DATASOURCE_POLLING, RESOURCE_POLLING and THREAD_POLLING.

  4. Edit the file <SOLUTION_MANAGER_HOME>/conf/solution-manager/denodo-monitor/ConfigurationParametersGeneral.template:

  5. To store in the database the information of the “Queries Monitor”, do this:

    1. Set the property vdpqueries.jdbcagent.enable to true

    2. Look for the group of properties of the database you are going to use and uncomment its properties. For example, if you want to use MySQL, look for “JDBC Agent Parameters: MySQL” and uncomment the nine properties below. Change the following properties (do not modify the others):

      1. vdpqueries.jdbcagent.url: set to the URL to the database.

      2. vdpqueries.jdbcagent.user: set to the user name that will connect to the database.

      3. vdpqueries.jdbcagent.password set to the password of the user name that will connect to the database.

        To enter the password encrypted, open a command line and run this:

        For Windows:

        cd <SOLUTION_MANAGER_HOME>\resources\solution-manager-monitor\denodo-monitor\bin
encrypt_password "<password of the database>"

        For Linux:

        cd <SOLUTION_MANAGER_HOME>/resources/solution-manager-monitor/denodo-monitor/bin
./encrypt_password.sh "<password of the database>"

        Copy the result of this program to the property vdpqueries.jdbcagent.password and set vdpqueries.jdbcagent.password.encrypted to true.

  6. To store in the database the information of the “Cache Monitor”, “Connections Monitor”, “Data Sources Monitor”, “Resources Monitor” and/or “Threads Monitor”:

    1. Set the property [vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.enable to true.

    2. Repeat the same steps above, but for the properties whose name start with [vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].

Kerberos Configuration

The JDBC connections can be configured to use Kerberos credentials: user and pass or user and keytab options are supported. To use a Kerberized JDBC connection establish the following properties:

  • [vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.useKerberos: if true, the information generated by the queries monitor, the cache monitor, the connections monitor, the data sources monitor, the resources monitor and/or the threads monitor will attempt to create a JDBC kerberized connection.

  • [vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbUser: Kerberos user.

  • [vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbPassword and [vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbPassword.encrypted: Kerberos password.

  • [vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbKeyTab: Kerberos keytab path.

  • [vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbConfigFile: path to the krb5.ini file in case it is not in the default paths.

  • [vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbDriverOptions: custom properties to be used by the driver when establishing a JDBC kerberized connection. If present, it will ignore natively supported connection properties. It must follow the format: prop1=value1;prop2=value2;…

Remember that you can encrypt the krbPassword property value by using the script encrypt_password script as explained in the previous section.

Cloud Storing Configuration

By default, the Denodo Monitor stores the log files on the local file system. You can configure it to store these log files in the cloud as well: in AWS S3 and Azure Blob Storage.

AWS

To do this, first you have to create an S3 bucket and obtain the AWS region of this bucket, the name of the bucket and the path within this bucket in which you want to store the log files.

To enable this feature, edit the file <SOLUTION_MANAGER_HOME>/conf/solution-manager/denodo-monitor/ConfigurationParametersGeneral.template and do these changes:

  • Uncomment the property monitors.cloudStorage.provider and change its value to AWS_S3.

  • Uncomment these properties and set its value to the key and secret respectively of your S3 account. Set encrypted value to true if the awsSecret property value has been encrypted with the Denodo Monitor encrypt_password script:

    monitors.cloudStorage.s3.credentials.awsKey
monitors.cloudStorage.s3.credentials.awsSecret
monitors.cloudStorage.s3.credentials.awsSecret.encrypted

  • Uncomment these properties and set its value

    monitors.cloudStorage.s3.destination.region
monitors.cloudStorage.s3.destination.bucket
monitors.cloudStorage.s3.destination.path

    These properties represent respectively, the region of the AWS bucket, the name of the bucket and the path within the bucket to store the log files.

Azure

To do this, first you have to create an Storage Account and get its connection string. You can create the desired container where you want to store the log files.

To enable this feature, edit the file <SOLUTION_MANAGER_HOME>/conf/solution-manager/denodo-monitor/ConfigurationParametersGeneral.template and do these changes:

  • Uncomment the property monitors.cloudStorage.provider and change its value to AZ_BLOB.

  • Uncomment these properties and set its value. Set encrypted value to true if the connectionString property value has been encrypted with the Denodo Monitor encrypt_password script:

    monitors.cloudStorage.azure.storage.connectionString
monitors.cloudStorage.azure.storage.connectionString.encrypted
monitors.cloudStorage.azure.blob.container
monitors.cloudStorage.azure.blob.prefix

    These properties represent respectively, the storage account connection string, a boolean that represents if the connection string was encrypted with the Denodo Monitor encrypt_password script, the name of the container where to store the log files and the prefix to assign to the blobs, this can be interpreted like the path within the container.

Common

The next configuration properties are used both in AWS and Azure.

  • To store the logs compressed, uncomment the property monitors.cloudStorage.compression.enabled and change its value to true.

  • The property monitors.cloudStorage.buffer.size controls the maximum number of log events that the Denodo Monitor will hold in memory, before storing them. The default value is correct for the majority of scenarios. To change it, uncomment the property.

  • The property monitors.cloudStorage.buffer.age controls the maximum amount of time in minutes that Denodo Monitor will hold a log event in memory, before storing it. The default value is correct for the majority of scenarios. To change it, uncomment the property.

Note

To configure both AWS S3 and Azure Blob Storage the property monitors.cloudStorage.provider must have both values separated by comma: AWS_S3,AZ_BLOB.

To “Uncomment a property” remove the character # at the beginning of the line.

The default values for these properties are correct for the majority of scenarios.

Add feedback