USER MANUALS

Global Monitoring Configuration

Note

Only global administrators, monitor administrators (users with the role monitor_admin) and users with the privilege Monitor for this environment can execute monitoring operations. More information is available in the Authorization section.

With the Solution Manager, you can launch the Denodo Monitor to collect the execution logs from a single Virtual DataPort server, or from all the servers of a cluster or environment.

The following section describes how to create a general monitoring configuration (in opposition to a per environment monitoring configuration). For a detailed explanation about launching a monitoring on Solution Manager once you have configured it correctly see Monitoring section.

Solution Manager allows you to create a general monitoring logging configuration, which gives a default configuration for every environment. To configure this, log in to the Solution Manager Administration Tool with an administrator account and click the menu Configuration > Monitoring configuration.

Global monitoring configuration menu

Global monitoring configuration menu

This configuration will apply to all the environments that do not set a specific monitoring configuration.

In this menu section the following monitorization parameters can be configured:

  • List of active monitors: list of the monitors to apply to the monitored environment.

  • Cloud storage appenders: if enabled, allows to store in AWS S3 and Azure Blob Storage logging files.

  • JDBC appenders: if enabled, allows to store logging events in one of the following DBMS: Oracle, MySQL, PostgreSQL, DB2 and SQL Server (Microsoft or jTDS driver).

  • Autostart active monitors: if enabled, when the Solution Manager Server is started, if there are any monitors active and not running it will try to launch them.

Note

Remember that during a normal restart of the solution manager, if the com.denodo.solutionmanagerserver.monitoring.stopRunningMonitorsOnShutdown property is set to true, the monitors will not be launched, despite the value of the Autostart active monitors property.

While the user has not configured the aforementioned parameters, global monitoring configuration section will contain the values configured in the template files ConfigurationParametersGeneral.template and ConfigurationParametersServer.template. The user could modify these default values and, once this is done, the values persisted in Solution Manager database will prevail.

Remember that other monitor parameters that are not shown can be manually configured as described in the Monitoring section.

Cloud Storage Appenders

Cloud storage appenders, if configured, allow to send logging events into files to be stored in the following cloud storage services: AWS S3 and/or Azure Blob Storage. Take into account that local log files are still generated even if this option is enabled. Click on the button Enable Cloud storage, which will allow you to configure them.

Global monitoring configuration Cloud tab

Global monitoring configuration tab

AWS S3

After enabling the cloud storage option, the AWS subtab allows to configure the S3 appender parameters.

Global monitoring configuration Cloud S3 tab

Global monitoring configuration S3 tab

Fill in the following mandatory fields:

  • Access key ID and Secret access key: The access key ID and the secret access key of the AWS account of your organization.

    You can check the page Access Keys of the AWS documentation for further details. The Solution Manager store the secret access key encrypted.

  • Region: The Region where the bucket belongs.

  • S3 base location URL: The URL of the bucket where the Solution Manager will store the log files. For instance, s3://my-bucket/denodo/.

Azure Blob

The Azure subtab allows to configure the Azure Blob Storage appender parameters.

Global monitoring configuration Cloud Azure tab

Global monitoring configuration Azure blob tab

Fill in the following mandatory fields:

  • Storage account connection string: The connection string that can be used to authorize access to data in your storage account via Shared Key authorization. The Solution Manager store the connection string encrypted.

  • Blob storage base location URL: URL of the blob storage with container and prefix where the Solution Manager will store the logging files. For instance, https://my-storage-account.blob.core.windows.net/my-container/denodo-logs/

Advanced Options

This subsection allows to configure general parameters that apply for all the cloud appenders.

Global monitoring configuration Advanced options section

Global monitoring configuration advanced options

It allows to configure:

  • Rotation type: logging rotation policy. The rotation can be established for a duration of time or for a size of logging events. Admits values Time and Size.

    • Time: if rotation by time is enabled, time to wait in minutes before publishing the file to the cloud storage service.

    • Size: if rotation by size is enabled, number of logging events before the contents are published into the storage service as a file.

  • Compression: if enabled, the log files are sent and stored compressed to the storage service.

JDBC Appenders

JDBC appenders, if configured, allow to send logging events to database tables in one of the following DBMS providers: Oracle, MySQL, PostgreSQL, DB2 and SQL Server (with Microsoft or jTDS driver). Take into account that local log files are still generated even if this option is enabled. Click on the button Enable DB storage to configure the details about db logging storage.

Global monitoring configuration DB tab

Global monitoring configuration DB tab

The following data must be provided:

  • Database: the DBMS where the monitor will have its database tables created.

    • Oracle 12cR2.

    • Oracle 19c or higher.

    • MySQL 5.7 or higher .

    • PostgreSQL 11 or higher.

    • IBM DB2 9.7 or higher.

    • Microsoft SQL Server 2014 or higher, which must use the Microsoft driver or the jTDS driver.

    Important

    Remember to place a suitable JDBC driver for the selected DBMS in the folder <SOLUTION_MANAGER_HOME>/resources/solution-manager-monitor/denodo-monitor/lib so that the Denodo Monitor application can establish a JDBC connection to store logging events. Besides this, this driver must also be placed in the folder <SOLUTION_MANAGER_HOME>/lib/solution-manager-extensions to allow the Solution Manager to automatically manage the monitor metadata. See section Monitor Tables Automatic Creation for further detail.

  • Driver class: class name of the JDBC driver. E.g. for Oracle oracle.jdbc.OracleDriver.

  • URL: JDBC URL to connect to your database server.

  • Authentication type: authentication type to be used when creating the JDBC connection, the following authentications types are supported, which require new dependent data:

    • User and password.

    • Kerberos user and password.

    • Kerberos user and keytab.

User and Password

When the authentication type “User and password” is selected the following fields to introduce are shown:

Global monitoring configuration DB tab with user and password authentication

Global monitoring configuration DB tab with user and password authentication

The fields User and Password are mandatory. These are the credentials to connect to the database. This user account needs privileges to insert rows in these tables. The password is stored encrypted by the Solution Manager.

Kerberos User and Password

Global monitoring configuration DB tab kerberos with user and password authentication

Global monitoring configuration DB tab kerberos with user and password authentication

The following fields must be provided:

  • Kerberos User and Password: credentials to connect to the database. This user account needs privileges to insert rows in these tables. The kerberos password is stored encrypted by Solution Manager.

  • Kerberos configuration file: (Optional) The path of a krb5.conf file if it is not going to be placed in its default location in the host where the Solution Manager is installed. See further detail in Kerberos config file documentation.

Kerberos User and Keytab

Global monitoring configuration DB tab with kerberos user and keytab authentication

Global monitoring configuration DB tab with kerberos user and keytab authentication

The following fields must be provided:

  • Kerberos User and Keytab: credentials to connect to the database. The kerberos user is the Service Principal Name in this case. The user account matching the Service Principal Name needs privileges to insert rows in these tables. The keytab represents the path to a keytab file, which will be stored encrypted by Solution Manager. See further detail about keytab files in Kerberos keytab documentation.

  • Kerberos configuration file: (Optional) The path of a krb5.conf file if it is not going to be placed in its default location in the host where the Solution Manager is installed. See further detail in Kerberos config file documentation.

Monitor Tables Automatic Creation

Solution Manager allows to automatically create or update the monitor metadata (tables that the Denodo Monitor needs to store its logging events and to store some monitor configuration) in the DBMS supported by the monitor.

For doing that, when the user saves the monitoring configuration if it contains JDBC configuration the Solution Manager will try to check if the JDBC source contains the tables required by the monitor to store its logging events. In order to do that, it will establish a JDBC connection using the configured connection data and check the state of the monitor metadata (if any) to decide which action must be taken.

Important

Place a suitable JDBC driver for the configured DBMS in the path: <SOLUTION_MANAGER_HOME>/lib/solution-manager-extensions so that the Solution Manager can establish JDBC connections to check the status or create or update the metadata. Notice that, in this case, it is the Solution Manager (and not the Denodo Monitor) the one that is going to establish JDBC connections.

Important

The user account must have privileges to create, drop and alter tables and insert/update/delete rows in these tables.

After this checking, the monitor metadata could be:

  • Empty: the schema referred in the JDBC section of the configuration is empty with regard to monitor tables, this is, no monitor tables are located in it. The user will be asked for permission to create the monitor metadata in that schema.

  • Invalid: some tables are found but the state is inconsistent. The metadata does not contain at least the monitor tables in Solution Manager GA version: REQUEST_NOTIFICATION and CACHE_NOTIFICATION. The user will be asked for permission to recreate all the tables, all data in the monitor existent tables will be lost.

  • Outdated: some tables are found but its state corresponds to an older version of monitor metadata (it was created with a previous update of Solution Manager either manually or automatically). The user will be asked for permission to update the metadata and the Solution Manager will apply the changes from the outdated version until the current version of the running Solution Manager.

  • Up to date: all the tables are in their last version. No changes are required. The user will not be reported about any change to do.

After checking the state the user might be asked for permission to perform changes in the JDBC source depending on the state of the metadata. For example, the Solution Manager could ask for permission to update the monitor metadata:

Global monitoring configuration DB tab updating metadata

Global monitoring configuration DB tab updating monitor metadata

The user has the possibility to cancel any change on the source and postpone it to the moment of launching a monitoring that uses that JDBC configuration when he will be asked again. However, if in that moment the user does not take the recommended action or there is no connection to the JDBC source, the launching will not take place.

Add feedback