Configuring the Denodo Monitor¶
To start using the Denodo Monitor, follow these steps:
Go to the directory
<DENODO_HOME>/tools/monitor
.On Windows, decompress the file
denodo-monitor.zip
.On Linux, execute this:
tar -xzf denodo-monitor.tar.gz
.Edit the file
conf/ConfigurationParameters.properties
to configure it.Although usually the default configuration of the Denodo Monitor is correct, you can find a description of the main properties right after this list of steps.
To start the Denodo Monitor go to bin, in the directory of the Denodo Monitor and do this:
On Linux, execute
./denodomonitor_startup.sh
.On Windows, execute
denodomonitor_startup.bat
.
Then, change the values of all the properties jmx.host
,
jmx.port
, jmx.user
and jmx.password
. Each monitor will read
these properties to know the URI and credentials of the Server it has to
monitor.
Almost all the properties in this configuration file are self-explanatory, so we are going to review only the most important ones:
jmx.user
. User name of an administrator or a user with the rolemonitor_admin
. Other users cannot connect to the JMX interface of Virtual DataPort.monitors.local
. Comma-separated list of local active “monitors”:processes
andsockets
(see section Local Monitors).monitors.remote
. Comma-separated list of the monitors you want to launch:vdp
,scheduler
orschedulerindex
.{ scheduler | schedulerindex | vdp }.monitors
. Properties that indicate which remote “monitors” will be active for each Server.The possible values for these properties are:
threads
resources
vdpqueries
(only forvdp
): to enable the “Virtual DataPort Queries Monitor”.vdploadcacheprocesses
(only forvdp
): to enable the “Virtual DataPort Cache Monitor”.vdpconnections
(only forvdp
): to enable the “Virtual DataPort Connections Monitor”.vdpdatasources
(only forvdp
): to enable the “Virtual DataPort Data Sources Monitor”.
{ processes | sockets | resources | threads }.interval
. Number of seconds between two JMX requests issued by that monitor.The properties
.jmx.password
store the password to connect to the Server of that property. These passwords can be stored encrypted. To do this, set the value of the property.jmx.password.encrypted
totrue
if the value of.jmx.password
is encrypted.For example, by default the Virtual DataPort password is
admin
. If you changed it and you want to store it in this configuration file encrypted, do the following:Execute the script
<DENODO_HOME>/tools/monitor/denodo-monitor/bin/encrypt_password <new password>
The output of this script is the encrypted password that you will put in the configuration file of the Denodo Monitor.
Open the file
<DENODO_HOME>/tools/monitor/denodo-monitor/conf/ConfigurationParameters.properties
Set the value of the property
vdp.jmx.password
to the encrypted password you obtained in step 1.Set the value of the property
vdp.jmx.password.encrypted
totrue
.
Important
When using the zipped version the encrypt_password
script will not be able to locate the encryption key from the installation so a fallback encryption key will be used.
In order to configure a unique encryption key for the Denodo Monitor a new keystore configuration should be added to the conf
directory. How to generate keystores is covered
in the section Generating a New Keystore with an Encryption Key and its Configuration.
configure_encryption_key_for_metadata_and_settings
--auto-generate
--keystore <path to new keystore>
--denodo-home <path to db-tools directory>
vdp.datasources.ping
. List of comma-separated data sources to which the Denodo Monitor will perform a ping operation. Each data source is specified by a reference like <database_name>.<data_source_type>.<data_source_name>, where data_source_type can only take one of the following values:jdbc
,odbc
,ldap
,olap
,mongodb
,sapbwbapi
,saperp
,salesforce
.For example:
customer360.jdbc.ds_jdbc_prod_oracle,customer360.salesforce_ds_sf_finance
Note that the ping is invoked periodically, every time the Virtual DataPort Data Sources Monitor registers information in the logs, which is controlled by the parameter
vdp.datasources.interval
.
JDBC Logging Configuration¶
You can configure Denodo Monitor to store the information generated by the following monitors in a database:
Queries monitor
Cache monitor
Connections monitor
Data Sources monitor
Resources monitor
Threads monitor
To enable this feature, change the following properties in the configuration file <DENODO_HOME>/tools/monitor/denodo-monitor/conf/ConfigurationParameters.properties
:
vdpqueries.jdbcagent.enable
,vdploadcacheprocesses.jdbcagent.enable
,vdpconnections.jdbcagent.enable
,vdpdatasources.jdbcagent.enable
,resources.jdbcagent.enable
andthreads.jdbcagent.enable
. Set totrue
to store the information generated by the queries monitor, the cache monitor, the connections monitor, the data sources monitor, the resources monitor and the threads monitor respectively in an external database.
Uncomment and configure the other jdbcagent
properties.
Before launching the Denodo Monitor with this option enabled, you have to do this:
Create the tables in the database that will store the information generated by the desired monitors. Denodo Monitor does not create the tables, just inserts data in them.
The directory
<DENODO_HOME>/tools/monitor/denodo-monitor/sql/
contains SQL scripts to create these tables, for several vendors (MySQL, Oracle, SQL Server, etc.). If there is no script for the database you want to use, you can use one of them as a template. Denodo Monitor only executes simple INSERT statements so you should be able to use any database.Copy the JDBC driver of the database (the jar files) to the folder
<DENODO_MONITOR>/lib
.
Kerberos Configuration¶
Denodo Monitor can connect to a database using Kerberos authentication. It can use either Kerberos authentication with username and password, or using a keytab file. To do this, uncomment and provide a value for these properties:
[vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.useKerberos
: set totrue
to use Kerberos authentication.[vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbUser
: Kerberos user.[vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbPassword
: Kerberos password.[vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbPassword.encrypted
: set totrue
if you encrypted the password with the script “encrypt_password” (see above). If not, set tofalse
.[vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbKeyTab
: path to the Kerberos keytab.[vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbConfigFile
: path to the krb5.ini file if it is not in the default path.[vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads].jdbcagent.krbDriverOptions
: custom properties to be passed to the driver. The value must follow the format:property1=value1;property2=value2;
About [vdpqueries|vdploadcacheprocesses|vdpconnections|vdpdatasources|resources|threads]...
: this means that you to you have to add this property the number of times you need, one for the each of the monitors you want to configure. For example, you have to add this:
vdpqueries.jdbcagent.useKerberos=true
vdploadcacheprocesses.jdbcagent.useKerberos=true
vdpconnections.jdbcagent.useKerberos=true
vdpdatasources.jdbcagent.useKerberos=true
resources.jdbcagent.useKerberos=true
threads.jdbcagent.useKerberos=true
Cloud Storing Configuration¶
The logs that are generated by Denodo Monitor in the local file system can also be stored in the cloud. This is possible for Amazon S3 cloud storage service and Azure Blob Storage. To enable storing the logs produced by Denodo Monitor into S3 storage configure the following parameters:
monitors.cloudStorage.provider
: this parameter is optional, if it is not present the logs will be generated in the local file system. If it contains the valueAWS_S3
the logs will also be stored in the S3 storage service. If it contains the valueAZ_BLOB
the logs will also be stored in the Azure Blob Storage service. The values must be separated by commas.monitors.cloudStorage.buffer.size
: represents the maximum number of log events before being sent to the cloud storage. This parameter is mandatory if the following one has no value.monitors.cloudStorage.buffer.age
: represents the maximum amount of time in minutes that the monitor will be holding the log events before being sent to the cloud storage. This parameter is mandatory if the previous one has no value.monitors.cloudStorage.compression.enabled
: optional parameter. Iftrue
, the logs will be stored compressed in the cloud storage.
The following parameters are specific to the S3 configuration:
The credentials required to access the S3 service:
monitors.cloudStorage.s3.credentials.awsKey
,monitors.cloudStorage.s3.credentials.awsSecret
.The location where the logs will be defined by the following parameters
monitors.cloudStorage.s3.destination.region
,monitors.cloudStorage.s3.destination.bucket
,monitors.cloudStorage.s3.destination.path
.
The following parameters are specific to the Azure Blob Storage configuration:
The storage account connection string have to be specified in
monitors.cloudStorage.azure.storage.connectionString
.The container and prefix where to store the logs files will be defined by the following parameters:
monitors.cloudStorage.azure.blob.container
andmonitors.cloudStorage.azure.blob.prefix
.
Obfuscation of Sensitive Information¶
You can configure the Denodo Monitor to obfuscate several types of information that can be considered sensitive data.
The Denodo Monitor can perform following transformations:
It replaces text literals and user names with **************.
It replaces number literals with 0, and 0.0 for decimal values.
IP addresses of client application, with XXX.XXX.XX.XX.
It replaces hostnames with **************.
It replaces connection ports with -1.
To enable these protections, the following properties can be defined:
com.denodo.vdb.util.log.obfuscate
: iftrue
, literals, IPs, user names, hostnames and ports will be obfuscated. The default value isfalse
.com.denodo.vdb.util.log.obfuscate.managedserver
: iftrue
, hostnames and ports will be obfuscated.com.denodo.vdb.util.log.obfuscate.ip
: iftrue
, IPs will be obfuscated.com.denodo.vdb.util.log.obfuscate.user
: iftrue
, user names will be obfuscated.com.denodo.vdb.util.log.obfuscate.literal
: iftrue
, literals will be obfuscated.
Notice that only the first property has a default value. So, if one of the other properties are defined its value prevails over the value of the first property.