Applies to:
Denodo 8.0
,
Denodo 7.0
Last modified on: 16 Jul 2020
Tags:
Administration
Amazon S3
Cloud
Logs
By default, the Denodo components store the log files on the local filesystem. However, you can configure them to store the log files on Amazon AWS S3 as well. This is useful for Denodo deployments that run on AWS and that you plan on switching off when you no longer need them. With this feature, you make sure that the logging information is saved even if the instance is deleted.
This feature was added in Denodo 7.0 update 20190903 and it has not been added for the Solution Manager.
You have to provide credentials to interact with S3 in order to allow you to send logs to the bucket.
The recommended ways are:
The AWS documentation about instance profile roles is available here.
In order to give the instances the needed permissions you have to:
Follow these steps to create security policy:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:CreateBucket", "s3:HeadBucket", "s3:PutObject" ], "Resource": [ "arn:aws:s3:::your-bucket-name/*" ] } ] } |
Follow these steps to create an IAM role:
When you launch an instance normally, in order to launch the new instances with permissions, select the corresponding role in the step “Configure instance”.
If you use autoscaling, you can specify the IAM role:
The appender also allows to configure the access and secret keys with the properties previously described inside the appender section:
When s3AwsKey and s3AwsSecret are present, they have precedence over other possible authentication methods defined in the credentials chain.
This method is available, but it is recommended to use the instance profile role option.
To store the log files of a Denodo server on an S3 bucket you need to modify the $Denodo_Home/conf/vdp/log4j2.xml file of the Denodo component.
Add the S3Appender inside the “Appenders” section:
<S3Appender name="S3Appender"> <stagingBufferSize>2500</stagingBufferSize> |
Add the appender to the Root logger:
<Root level="error"> |
The following properties control how the logs are stored in S3:
With these properties you can control when the appender will send the logs to the S3 bucket:
There is also the following connection property:
The following access properties are optional:
We recommend using instance profile roles instead of configuring AWS credentials directly in the log4j2.xml file.
NOTE: to log the server start / stop messages, you will also need to add these loggers categories in the Loggers section:
<Logger name="server.start" level="info" /> |
The log4j2.xml files are available in the following paths in a Denodo Platform installation for each type of server:
In each case, you will need to stop the corresponding server, make the changes and start the server with the new appender configured.