You can translate the document:

Introduction

The Denodo Monitor is a tool included in the Denodo Platform that gathers information about the Denodo servers periodically. By default, this information is stored in logs as tab-separated values files.

By default, the Denodo Monitor logs information about:

  • Virtual DataPort.
  • Scheduler and Scheduler Index

Optionally, it can be also configured to log information about the Embedded Apache Tomcat as described in the article Monitoring the embedded Tomcat web container.

Many options are monitored by the tool as described in the Denodo Monitor manual. The most relevant are:

  1. Processes Monitor. It logs information about all the running processes in the local host. The output is logged to processes.log. In Windows OS, it logs the output of the command tasklist.exe, and in Unix/Linux, the output of ps.
  2. Sockets Monitor. It logs the information of the active connections of the local computer by executing the command netstat. The output is logged to sockets.log.
  3. Resources Monitors. The Resources monitor logs several parameters of the memory footprint of a Denodo server. The output is logged to <denodo_server_name>-resources.log.
  4. Threads Monitors. The Threads monitor logs the threads launched by a Denodo server and how much CPU time they consume. Its log file is stored in <denodo_server_ name>-threads.log.
  5. Virtual DataPort Queries Monitor. It logs all the statements processed by Virtual DataPort. It is specific to Virtual DataPort and cannot be used to monitor other Denodo servers. The output is logged to vdp-queries.log.
  6. Virtual DataPort Cache Monitor. It logs the activity of the processes that load the Virtual DataPort cache. The output is logged to vdp-loadcacheprocesses.log.
  7. Virtual DataPort Connections Monitor. It logs information about opened and closed connections, the user account that opened/closed the connection, the interface it used (JDBC, ODBC, web service, etc.), from which IP, etc. The output is logged to vdp-connections.log.
  8. Virtual DataPort Data Sources Monitor. Every number of seconds (determined by the interval property), it logs information about all the data sources of the Virtual DataPort server. The output is logged to vdp-datasources.log.

What is a Custom Monitor

The information provided above is quite useful but sometimes it might not be enough or there is just a requirement to run a command regularly without setting a cron expression and to save the output of that command.

To resolve these scenarios, the Denodo Monitor supports the creation of “Custom Monitors”.  Custom monitors are local monitors but the command they execute is customizable.

This option provides the flexibility to execute any command at your convenience and store the result in a log file and do it regularly to follow up the evolution of the output.

Parameters of the Custom Monitor

Configuring a Custom Monitor is very simple and it only requires you to uncomment and edit a few lines at the end of the <DENODO_MONITOR_HOME>/conf/ConfigurationParameters.properties file.

The parameters to configure the Custom Monitors are:

  • monitors.custom: this parameter contains the comma-separated values representing the names of the custom monitors. For instance:

        

        monitors.custom=my_monitor_1,my_monitor_2

  • Then, for every monitor name, the following parameters must be defined:
  • interval: execution time in seconds.
  • log.datepattern: the date pattern to log the date for each entry in the log file.
  • command: the command to be executed.

my_monitor_1.interval=30

my_monitor_1.log.datepattern=yyyy-MM-dd-a

my_monitor_1.command=df -h /

Practical example

We have a requirement to log and track the percentage of disk space used by the Denodo server which is running on a Linux machine.

In a very simple setup, we could use the command df -h / to check the percentage of disk used.

The configuration of this monitor will look like this:

# List of enabled custom monitors

monitors.custom=monitor1_diskfree

monitor1_diskfree.interval=30

monitor1_diskfree.log.datepattern=yyyy-MM-dd-a

monitor1_diskfree.command=df -h /

When the Denodo Monitor is running a new file “monitor1_diskfree.log” will be created.

 

The content of each entry in the file will look like this:

================================================================

2023-12-06T11:53:55.264

        Filesystem      Size  Used Avail Use% Mounted on

        overlay         251G  165G   74G  70% /

Now, we could leverage the Denodo capability to read log files through the Denodo Log Custom Wrapper to create a base view so we can analyze the evolution of the disk space over time.

                                Data source configuration 1

                                Data source configuration 2

                                Data source configuration 3

The input fields required for the creation of the data source are:

  • Date Pattern: yyyy-MM-dd'T'HH:mm:ss.SSS
  • Date Extractor Pattern: (?:.*?)(\\d*-\\d*-\\d*T\\d*:\\d*:\\d*\\.\\d*) (?:.*?)
  • Log timezone: GMT 
  • Content Extractor Pattern: (\\d\\d\\d\\d-\\d\\d-\\d\\dT\\d\\d:\\d\\d:\\d\\d\\.\\d\\d\\d).*\\n\\s*.*\\n\\s*overlay\\s*(\\d*)G\\s*(\\d*)G\\s*(\\d*)G\\s*(\\d*)%\\s*\\/\\n\\s*\\n=*\\n\\s*

Note that the Content Extractor Pattern will depend on the command you execute and the Operating System so this example must be adapted to the contents of your custom monitor log files.

After creating the  base view and renaming the output fields, the base view will look like this:

And if we run the view

We will have our expected results coming from the log files.

We could use this information for two purposes.

  1. Create an alert if the latest value is higher than a threshold.
  2. Create a report to analyze the evolution of the used disk.

References

Denodo Monitor

Monitoring the embedded Tomcat web container

Denodo Log Custom Wrapper

Disclaimer
The information provided in the Denodo Knowledge Base is intended to assist our users in advanced uses of Denodo. Please note that the results from the application of processes and configurations detailed in these documents may vary depending on your specific environment. Use them at your own discretion.
For an official guide of supported features, please refer to the User Manuals. For questions on critical systems or complex environments we recommend you to contact your Denodo Customer Success Manager.

Questions

Ask a question

You must sign in to ask a question. If you do not have an account, you can register here