USER MANUALS

Configuring the Logging System

Virtual DataPort, its administration tool and the Design Studio use the library Apache Log4j 2 to log its activity.

They store their log files in these directories:

  • Virtual DataPort: <DENODO_HOME>/logs/vdp/.

  • Administration Tool: <DENODO_HOME>/logs/vdp-admin/. This is in the computer of the user that runs the administration tool, not on the server.

  • Design Studio: <DENODO_HOME>/logs/design-studio/.

The following files control the configuration of the logging system of Virtual DataPort:

  • Virtual DataPort: <DENODO_HOME>/conf/vdp/log4j2.xml.

  • Administration Tool: <DENODO_HOME>/conf/vdp-admin/log4j2.xml. This is in the computer of the user that runs the administration tool, not on the server.

  • Design Studio: <DENODO_HOME>/resources/apache-tomcat/webapps/denodo-design-studio/WEB-INF/classes/log4j2.xml.

To apply the changes made on these files, restart Virtual DataPort, the Administration Tool or the Design Studio respectively.

You can also configure these components to store the logs on Amazon AWS S3. The Knowledge Base article How to store Denodo logs in Amazon S3 explains how to set this up.

You can also configure these components to store the logs on Azure Blob Storage. The Knowledge Base article How to store Denodo logs in Azure Blob Storage explains how to set this up.

To change the level (i.e. “ERROR”, “WARN”, “INFO”, “DEBUG” or “TRACE”) of a log category, you have these options:

  1. Modify the configuration file <DENODO_HOME>/conf/vdp/log4j2.xml and restart Virtual DataPort to apply the changes.

  2. Invoke the stored procedure LogController.

  3. From the monitoring interface of Denodo (JMX), invoke the operation setLogLevel of the LogManagementInfo MBean.

With option #1, you need to restart Virtual DataPort for the changes to take effect but they are permanent. With options #2 and #3, the changes are applied immediately but lost when restarting Virtual DataPort.

Important

Do not set the log level to “TRACE” or “DEBUG” unless instructed by the Support Team of Denodo.

Reasons to avoid setting the log level to “TRACE” or “DEBUG”:

  1. The logging system may log sensitive information like the data retrieved from the data sources. For example, if you set the log category “com.denodo” or “com.denodo.vdb.catalog.type” to “TRACE”, the logging system will log the results of the queries.

  2. It makes troubleshooting more difficult because the log files may be filled up with messages irrelevant to the problem at hand. For example, if you set “com.denodo” or “com.denodo.vdb.interpreter” to “TRACE”, the logging system will log a lot information about the process of parsing each query received by Virtual DataPort. This is necessary if there are problems in the parsing but it will fill up the logs with unnecessary messages if the problem is different.

  3. In production servers, avoid setting the log category “com.denodo” to “DEBUG” or “TRACE” because it will reduce the performance of Virtual DataPort, due to the amount of messages that the logging system will have to store.

Obfuscation of Sensitive Information of Requests

You can configure the logging system of Virtual DataPort to log the queries it receives in <DENODO_HOME>/logs/vdp/vdp.log. One way of doing this is with the stored procedure LOGCONTROLLER. For example, with this command:

CALL LOGCONTROLLER('com.denodo.vdp.requests', 'INFO');

When the logging system of Virtual DataPort logs a request in <DENODO_HOME>/logs/vdp/vdp.log, it obfuscates the requests. Specifically, it replaces the following before logging the query:

  • It replaces text literals and user names with **************.

  • It replaces number literals with 0, and 0.0 for decimal values.

  • IP addresses of client application, with XXX.XXX.XX.XX.

The goal is to avoid storing data in the log files that may be sensitive. For example, if a user executes:

SELECT *
FROM employee
WHERE social_security_number = '555-01-0002'

and you configured the logging system to log the requests, the “vdp.log” file will have this:

280993143 [DNI(169)-XXX.XXX.XX.XX-75] INFO  2021-02-08T13:37:44.247 com.denodo.vdp.requests [] - SELECT *
FROM employee
WHERE social_security_number = '***********'

Note that the IP of the client application is modified to “XXX.XXX.XX.XX” and in the query, the value of the social security number is replaced with asterisks.

These transformations only affect the way the logging system stores the queries, they do not affect the execution of the queries.

To disable these protections, execute these commands:

  • If you do not want to obfuscate literals, execute this:

    SET 'com.denodo.vdb.util.log.obfuscate.literal' = 'false';

  • If you do not want to obfuscate the user names, execute this:

    SET 'com.denodo.vdb.util.log.obfuscate.user' = 'false';

  • If you do not want to obfuscate the IP address of the users, execute this:

    SET 'com.denodo.vdb.util.log.obfuscate.ip' = 'false';

  • To disable all log obfuscation, execute this:

    SET 'com.denodo.vdb.util.log.obfuscate' = 'false';

To only obfuscate the IP addresses of the client applications but leave the other values in clear text, set “com.denodo.vdb.util.log.obfuscate” to “false” and “com.denodo.vdb.util.log.obfuscate.ip” to “true”.

Add feedback