Audit Information¶

The audit information contains the following fields:

• StartTime: start time of the requested action.

• EndTime: finish time of the requested action.

• Elapsed Milliseconds: duration (in milliseconds) of the action.

• SessionId: internal identifier for the user’s session.

• ClientIP: the IP address of the final client (i.e. the one that sends the HTTP request. Usually, the machine with the client browser).

• IntermediateClientIP: (Only in Scheduler) the IP address where the Scheduler Administration Tool is running.

• serverIP: the IP address where the server is running.

• UserName: login name of the user executing the action.

• UserAuthenticationMethod: which authentication method used the user to log in (LOCAL / REMOTE / SSO / KERBEROS).

• UserBatchMode: (Only in Scheduler) whether the user executing the action was in batched mode or not.

• UserPermissions: the permissions of the user executing the action.

• RequestType: type of operation performed by the method (START / STOP / ENABLE / DISABLE / CREATE / READ / UPDATE / DELETE, etc).

• ElementTypes: element types (Job / DataSource / Project / ServerInfo, etc) over the request was done.

• AffectedElements: information about the elements (identifier and name) that were affected by the execution of the audited method.

• Endpoint: the action performed. It contains the method signature with its name and parameters.

• Arguments: input arguments for the executed method.

• State: Status of the action (OK / ERROR / WARNING).

• Warnings: only for bulk methods, the information about the elements that could not be executed will be shown. It only applies when State=WARNING.

• Thread Name: Name of the thread that generated the logging event.

• ThreadID: identifier of the thread that generated the logging event.

• LogLevel: log level the event was generated at.

Masking Sensitive Information¶

By default, when auditing events, sensitive or environment-dependent information will be masked.

It is possible to control which information should be masked with the following properties:

• com.denodo.util.audit.config.AuditMaskingConfig.obfuscate.ip: set to false when you do not want to obfuscate the IP address of the users.

• com.denodo.util.audit.config.AuditMaskingConfig.obfuscate.user: set to false when you do not want to obfuscate the user names.

• com.denodo.util.audit.config.AuditMaskingConfig.obfuscate.auditMask: set to false when you do not want to obfuscate sensitive properties (mainly, environment dependent properties, such as server uris, host names, ports, etc).

• com.denodo.util.audit.config.AuditMaskingConfig.obfuscate:

  • true will enable all obfuscation (no matter the values from the other properties).

  • false will disable all obfuscation (no matter the values from the other properties).

  • If not set, the previous properties will control the obfuscation.

How to Configure Audit in Scheduler¶

To enable auditing in Scheduler, you have to follow these steps:

1. Enable loggers in <DENODO_HOME>/conf/scheduler/log4j2.xml

   1. Replace <Configuration> with <Configuration packages="com.denodo.util.audit.log">

   2. Configure appenders: you can define appenders (inside <Appenders> section) to a JSON file and/or to a database (you must configure at least one of them):

      • Appender to JSON:

        <RollingFile name="REQUESTOUT-JSON" fileName="<DENODO_HOME>/logs/scheduler/scheduler-requests.json"
              filePattern="<DENODO_HOME>/logs/scheduler/scheduler-requests.json.%i">
              <JsonTemplateLayout eventTemplateUri="classpath:DenodoJsonLayout.json"
                 stackTraceEnabled="false" charset="UTF-8" />
              <SizeBasedTriggeringPolicy size="10 MB" />
              <DefaultRolloverStrategy max="7" />
        </RollingFile>
        

        Replace <DENODO_HOME> properly.

      • Appender to database:

        <JDBC name="REQUESTOUT-DB" tableName="sched_audit_logs" ignoreExceptions="false">
              <ConnectionFactory
                 class="com.denodo.scheduler.core.server.quartz.connection.SchedulerPoolingConnectionProvider"
                 method="getAuditDBConnection" />
              <AuditDBLayout />
        
              <ColumnMapping name="startTime" type="java.time.ZonedDateTime" />
              <ColumnMapping name="endTime" type="java.time.ZonedDateTime" />
              <ColumnMapping name="elapsedTime" type="java.lang.Long" />
              <ColumnMapping name="sessionID" />
              <ColumnMapping name="clientIP" />
              <ColumnMapping name="intermediateClientIP" />
              <ColumnMapping name="serverIP" />
              <ColumnMapping name="userName" />
              <ColumnMapping name="userAuthMethod" />
              <ColumnMapping name="userBatchedMode" type="java.lang.Boolean" />
              <ColumnMapping name="userPermissions" />
              <ColumnMapping name="requestType" />
              <ColumnMapping name="elementTypes" />
              <ColumnMapping name="affectedElements" />
              <ColumnMapping name="endpoint" />
              <ColumnMapping name="arguments" type="java.sql.Clob" />
              <ColumnMapping name="state" />
              <ColumnMapping name="warnings" type="java.sql.Clob" />
              <ColumnMapping name="threadName" />
              <ColumnMapping name="threadId" type="java.lang.Long" />
              <ColumnMapping name="logLevel" />
        </JDBC>
        

   3. Add logger (with appenders to file and/or database, depending on the previous step):

      <Logger name="EventLogger" level="INFO" additivity="false">
            <AppenderRef ref="REQUESTOUT-JSON" />
            <AppenderRef ref="REQUESTOUT-DB" />
      </Logger>
      

      Depending on the log level: - INFO: default level. It will mainly record actions that make changes to the metadata, such as creations, updates and deletions (for instance: create a job). - DEBUG: in addition to the information registered with INFO, it will include getters (for instance: get jobs). - TRACE: in addition to the information registered with DEBUG, it will include internal metadata calls and minor getters (for instance: get meta configuration for VDP jobs).

      • Besides auditing more methods from the API, it also includes more information for each one, as explained in Audit Information.

        • UserAuthenticationMethod. - UserBatchMode. - UserPermissions.

2. Only when the appender to database is configured, you have to do these additional steps:

   1. Configure the auditing database:

      • You can configure the target database where to store the audit information. Note that by default, the audit will use the same database as the metadata.

      • If you want to use a different database than the one used to store the Scheduler metadata (this is highly recommended), then you have to manually configure its settings in <DENODO_HOME>/conf/scheduler/ConfigurationParameters.properties

        Example of audit database settings for PostgreSQL¶

        # Audit DataSource
        AuditDataSource/driverClassName=org.postgresql.Driver
        AuditDataSource/url=jdbc:postgresql://auditdb.denodo.com:5432/audit
        AuditDataSource/user=audit_user
        AuditDataSource/password=audit_password
        AuditDataSource/databaseName=PostgreSQL
        AuditDataSource/useKerberos=false
        AuditDataSource/classpath=postgresql-12
        

   2. Create the table (sched_audit_logs) to store the audit information in the audit database set up in the previous step:

      • Execute sql script tables_audit_trail_<db>.sql from <DENODO_HOME>/scripts/scheduler/sql/<db>.

How to Configure Audit in Solution Manager¶

To enable auditing in Solution Manager, you have to follow these steps:

1. Enable loggers in <DENODO_HOME>/conf/solution-manager/log4j2.xml

   1. Replace <Configuration> with <Configuration packages="com.denodo.util.audit.log">

   2. Configure appenders: you can define an appender (inside <Appenders> section) to a JSON file (JDBC appender is not supported for now):

      • Appender to JSON:

        <RollingFile name="REQUESTOUT-JSON" fileName="<DENODO_HOME>/logs/solution-manager/solution-manager-requests.json"
              filePattern="<DENODO_HOME>/logs/solution-manager/solution-manager-requests.json.%i">
              <JsonTemplateLayout eventTemplateUri="classpath:DenodoJsonLayout.json"
                 stackTraceEnabled="false" charset="UTF-8" />
              <SizeBasedTriggeringPolicy size="10 MB" />
              <DefaultRolloverStrategy max="7" />
        </RollingFile>
        

        Replace <DENODO_HOME> properly.

   3. Add logger (with appender to file):

      <Logger name="EventLogger" level="INFO" additivity="false">
            <AppenderRef ref="REQUESTOUT-JSON" />
      </Logger>
      

      Depending on the log level:

      • INFO: default level. It will mainly record actions that make changes to the metadata, such as creations, updates and deletions (for instance: create an environment).

      • DEBUG: in addition to the information registered with INFO, it will include getters (for instance: get environments).

      • TRACE: in addition to the information registered with DEBUG, it will include internal metadata calls and minor getters.

        • Besides auditing more methods from the API, it also includes more information for each one, as explained in Audit Information.

          • UserAuthenticationMethod.

          • UserPermissions.

2. Enable audit mode in <DENODO_HOME>/conf/solution-manager/SMConfigurationParameters.properties

   com.denodo.solutionmanager.audit.enabled=true