How to store Denodo logs in Azure Blob Storage
You can translate the document:Introduction
By default, the Denodo components store the log files on the local filesystem. However, it is possible to configure them to store the log files on Azure Blob Storage as well. This is useful for Denodo deployments that run on Azure and that are switched off when they are no longer needed. With this feature, the logging information is saved even if the virtual machines are deleted.
This feature was added in Denodo Platform 8.0 update 20220126 and it has been added for the Solution Manager in Denodo 8.0 update 20220815.
Configuration
Authentication
To configure the access to Azure Blob Storage, we have to provide the Connection String of the storage account to be used, in order to authenticate the applications’ requests to this storage account.
The Microsoft documentation about how to configure Azure Storage connection strings is available here.
In the Azure portal, in the storage account detail page we can access the connection string value through the Access keys option of Security + networking.
Log4j2.xml file configuration
To store the log files of a Denodo Virtual DataPort server on an S3 bucket you need to modify the file <DENODO_HOME>/conf/vdp/log4j2.xml.
Add the CloudAppender inside the “Appenders” section:
|
<CloudAppender name="CloudAppender">
<stagingBufferSize>2500</stagingBufferSize>
<azureBlobContainer>[container_name]</azureBlobContainer> <azureBlobNamePrefix>[blob_name_prefix]</azureBlobNamePrefix> <azureStorageConnectionString>[access_key_connection_string]</azureStorageConnectionString>
|
Add the appender to the Root logger:
|
<Root level="error">
|
The following properties control how the logs are stored in Azure Blob Storage:
- azureBlobContainer: the container to use. The appender will attempt to create the container if it does not exist.
- azureBlobNamePrefix: the prefix where each log file will be stored inside the container.
- azureStorageConnectionString: the connection string used to authenticate the applications’ requests to this storage account.
Since Denodo 8.0 update 20240306, the value of azureStorageConnectionString can be encrypted using the script <DENODO_HOME>/bin/encrypt_password to avoid specifying its value as plain text.
These properties control when the appender will send the logs:
- stagingBufferSize: number of log messages after which the content is written.
- stagingBufferAge: number of minutes after which the content is written. If this property is defined, stagingBufferSize property value is ignored.
NOTE: to log the server start / stop messages, add these log categories in the Loggers section:
|
<Logger name="server.start" level="info" />
|
The log4j2.xml files are available in the following paths in a Denodo Platform installation for each type of server:
- Virtual DataPort server: <DENODO_HOME>/conf/vdp/log4j2.xml
- Scheduler server: <DENODO_HOME>/conf/scheduler/log4j2.xml
- Scheduler Index: <DENODO_HOME>/conf/arn-index/log4j2.xml
- Data Catalog: <DENODO_HOME>/conf/data-catalog/log4j2.xml
In each case, you will need to stop the corresponding server, make the changes in the file and start the server with the new appender configured.
The information provided in the Denodo Knowledge Base is intended to assist our users in advanced uses of Denodo. Please note that the results from the application of processes and configurations detailed in these documents may vary depending on your specific environment. Use them at your own discretion.
For an official guide of supported features, please refer to the User Manuals. For questions on critical systems or complex environments we recommend you to contact your Denodo Customer Success Manager.
Recommended resources Recommendations generated by AI
Questions
