Applies to:
Denodo 8.0
Last modified on: 21 Jan 2022
Tags:
JMX
Monitoring
Prometheus
This document describes the detailed steps on monitoring Denodo with Prometheus including steps to install, configure and monitor Denodo from Prometheus via the JMX Exporter.
Prometheus
Prometheus is an open source monitoring and alerting tool which stores and exposes application metrics, statistics, etc. It collects metrics from targets by scraping HTTP endpoints at given intervals, evaluates rule expressions, displays the results and can trigger alerts when specific conditions are observed.
Prometheus Exporters
Prometheus Exporters are used to export data metrics from Prometheus. These exporters need to be downloaded and installed in order to pull appropriate metrics. Among the different types of Prometheus exporters, a JMX Exporter can be used to perform the integration with Denodo for monitoring.
JMX Exporter
JMX (Java Management Extensions) is a Java-native specification which allows developers to expose a Java application’s metrics, statistics and basic operations. Since most enterprise-class monitoring solutions support gathering JMX metrics, using a Prometheus JMX exporter is considered as the standard and practical way to gather data about running Java applications.
This section explains the detailed steps for configuring Prometheus with the Denodo Virtual DataPort server for monitoring purposes. It includes:
If the Virtual DataPort server and Prometheus run on the same machine, you can skip this section. However, if the Virtual DataPort server runs on a different machine then the steps mentioned in the following documents may be needed:
Set the Host Name of Virtual DataPort
Enable Authentication on the Monitoring Interface
These steps will allow monitoring a Virtual DataPort server running on a remote machine.
The Prometheus installation comes with a sample configuration of the prometheus.yml file. In this file, the "scrape_configs" element controls the resources monitored by Prometheus.
In addition to the default configuration, configure a new job for Denodo under the "scrape_configs" block. Below is a sample Denodo job configuration for the prometheus.yml file, where localhost and 8081 are the Prometheus host and the Prometheus default port number:
- job_name: "Denodo" static_configs: - targets: ["localhost:8081"] |
You can use the below sample for the config.yaml file:
--- startDelaySeconds: 0 hostPort: localhost:9997 username: admin password: admin ssl: false lowercaseOutputName: false lowercaseOutputLabelNames: false rules: - pattern: ".*" |
NOTE: This is a simple configuration and users can add additional configuration parameters based on their requirements.
To use Prometheus, a JVM parameter must be added to the Java application (here, the Denodo Virtual DataPort server) requesting that it loads the agent on application startup. The configuration file for specifying the JVM parameters need to be modified accordingly for integrating Prometheus with the Virtual DataPort server.
For this, the property "java.env.DENODO_OPTS_START" of the VDBConfiguration.properties file under directory "<DENODO_HOME>/conf/vdp" needs to append the following "-javaagent" parameter:
-javaagent:<PATH>/<JAR_NAME>.jar=[HTTP PORT]:<PATH>/config.yaml
If the JMX exporter JAR file and the config.yaml are placed under "<DENODO_HOME>/bin" then configure the value like below (specify the complete path in case of other location):
-javaagent:jmx_prometheus_javaagent-0.16.1.jar=8081:config.yaml
The port 8081 is the same configured in prometheus.yml. The final value of the property will be something like:
java.env.DENODO_OPTS_START=-Xmx4096m -XX\:+DisableExplicitGC -XX\:+UseG1GC -XX\:ReservedCodeCacheSize\=256m -javaagent\:jmx_prometheus_javaagent-0.16.1.jar\=8081\:config.yaml
Once the changes are made, make sure to run the script <DENODO_HOME>/bin/regenerateFiles.bat/.sh in order to propagate the changes in the startup scripts.
To modify the JVM parameter graphically, open the Denodo Platform Control Center, go to Configure > JVM Options and append the "-javaagent" parameter to the existing value (make sure to remove backslashes \).
Finally, restart the Virtual DataPort server.
prometheus --config.file=prometheus.yml --web.listen-address=:8082
To validate the status of the server, navigate to the Status -> Targets section and check if the status of the Denodo server is "Up". Also, you can explore more about the data collected by Prometheus under different sections of the tool.
If the javaagent parameter configuration is not correct then the Virtual DataPort server will not start.
CAUSE: This is due to incorrect JVM configuration.
ACTION: For this, the appended JVM property needs to be reviewed.
The Prometheus startup returns the error "parsing YAML file prometheus.yml: yaml: line 23: did not find expected '-' indicator".
CAUSE: This error indicates that the service expects an '-' indicator in the specific line. This usually occurs when the prometheus.yml file has unwanted spaces instead of the expected '-' indicator in the line.
ACTION: Removing extra spaces in the line will resolve the error.
Prometheus includes an additional optional Alert Manager component. The AlertManager takes care of handling alerts, grouping them, deduplicating them, and routing them to the appropriate services like email, chat, and PagerDuty. It can also silence and inhibit certain alerts during planned events or downtime. For configuring alerts with Prometheus, you can use the How to Configure Prometheus AlertManager for Grafana Alerts and Prometheus Alerts documentation as reference.