This section explains how to configure and personalize several aspects of the tool. It covers the following topics:
These parameters are global for all the users of the tool.
Elements Shown to the Users¶
You can specify which elements from the catalog the users are allowed to see:
Only Web Services
It affects the results shown in the following sections:
You can choose whether to Allow users to browse by Databases/Folders or not. If this option is disabled, users will not be able to access the Databases/Folders panel. In this case, you can also choose to Hide uncategorized views and web services from users (uncategorized views and web services will not appear in search results or when browsing by Categories or Tags).
You can customize this tool to adapt it to the branding of your company. You can change the following:
The logo displayed on the top left.
The image displayed on the background of the Login page.
The image displayed on the background of the Welcome page.
Maximum size for uploaded images is 5MB.
You can add an informative message that will be displayed to all users when they access the Data Catalog. It will show up at the top of the page.
It can be used to display a welcome message, inform users when the service is expected to be down for maintenance, legal disclaimers, etc.
To enter the message in HTML, click Source in the editor, enter the HTML and click Source again. The Save button is disabled while in Source mode.
Allow HTML descriptions and properties¶
In this section, there is an option for specifying if user can create HTML properties and add HTML descriptions for categories, tags, databases, views, web services, groups and properties. If HTML inputs are not allowed, element descriptions will always be displayed as plain text and existing HTML type properties will be converted to Long Text type. In addition, it will not be allowed to create HTML type properties.
Export Format Configuration¶
The available export formats, among: CSV, HTML, Excel, Tableau (.tde) and Tableau (.hyper) (the section below explains how to enable the exports to Tableau). All enabled by default.
If by default the header of the files should be exported for the different formats (CSV, HTML, Excel). For the formats tableau (.tde) and tableau (.hyper) the header will always be exported.
The maximum number of rows using each of the available formats.
If only the users with the role data_catalog_exporter in Virtual DataPort can export the results of the queries using each of the available exporters.
The default field delimiter character for exports to CSV format. As a delimiter character you can choose comma, pipeline, semicolon, space, tab or a user-defined delimiter character.
The ” character cannot be set as a delimiter character.
Enabling Exports to Tableau¶
To enable the feature of exporting data to Tableau, you need to install the “Tableau Data Extract API” (for .tde format) or “Extract API 2.0” (for .hyper format) in the host where the Denodo Platform is installed. To do this, follow these steps:
In this page, sign in and download the package C/C++/Java 64-bit version. Select Windows or Linux depending on the operating system where the Denodo server runs.
Decompress the downloaded file.
If the Denodo server runs on Windows, copy the files of the folder
binin the downloaded file to
If the Denodo server runs on Linux:
Copy the files of the folder
bin(for .tde) or
bin/hyper(for .hyper) in the downloaded file to
Copy all the .so files of the folder
lib64/dataextract(for .tde) or
lib64/tableausdk(for .hyper) in the downloaded file to
<DENODO_HOME>/jre/bin. Java subfolder files are no needed. Note that for some Linux distributions, it is necessary to copy all the mentioned .so files to
/libdirectory of the Linux root folder.
Restart the Data Catalog.
Database, View and Web Service Connection URIs¶
You can choose whether to display the
Show Connection URIs button or not in
the databases’, views’ and web services’ details page. If you choose to show it, you can customize several
Introduction text of the connections panel: this will be shown in the panel instead of the default introduction.
URI Patterns: these describe how to access the server’s databases through JDBC, ODBC, REST and OData. You may use the following variables in the pattern definitions:
$webcontainer_port. These variables will be replaced by the appropriate values when showing the URIs to the user.
Option to configure whether to show or hide the Tableau data source connection button on the Summary tab of the views page.
The Data Catalog allows showing usage statistics of the views and web services by analyzing the usage data collected by Denodo Virtual DataPort. This feature requires some preparation steps, that will be explained later on this section.
When this feature is enabled, the details page of views and web services includes a new tab that shows some statistics about its usage (after they are collected and computed).
Steps to Configure the Gathering of Statistics¶
In order to show the Usage tab in the details page of views and web services, a user with administration privileges has to perform the following steps:
Configure the Denodo Monitor to store the log of requests in a database (for instance, a table in Oracle). By default, Denodo Monitor saves the log information to a file. You need to modify this to use a database instead the log file.
<DENODO_HOME>/tools/monitor/denodo-monitor/conf/ConfigurationParameters.properties, search for the following text and follow the instructions of the file.
# ================================================================================ # Settings to store the output of the Virtual DataPort query monitor in a database # ================================================================================
Data Catalog requires to use the Denodo Monitor included in update 20180926 or later.
Do the following steps in Virtual DataPort:
Due to license restrictions, you cannot use the Virtual DataPort server included in the Solution Manager.
Create a database called denodo_logs. Although you could use a different name, if you do, you will have to modify the properties file of the Scheduler project that you will import in step #4.
Create a JDBC data source that points to the database where Denodo Monitor will store the log of requests. If you already have this data source, you do not need to create a new one.
Create a base view over the table
request_notificationof this data source. This is the table in which Denodo Monitor will store the requests log.
This base view has to be called
request_notification. Do not modify the names of the fields.
Check that the cache engine of Virtual DataPort is enabled.
Edit the file
<DENODO_HOME>/resources/data-catalog/statistics/usage-statistics.propertiesand do these changes:
The properties ending with URL have to point to the Data Catalog. You may need to change them if the Tomcat server where the Data Catalog is running is not listening on the default port and/or is not resolvable as localhost from Virtual DataPort.
The properties ending with USER, PASSWORD, and PASSWORD.ENCRYPTED indicate the user name and password that will be used to call the REST API of the Data Catalog. By default, they are configured to use the default admin user with its default password.
The properties ending with TIMEOUT indicate the timeout (in milliseconds) of the calls to the Data Catalog API.
Import the file
<DENODO_HOME>/resources/data-catalog/statistics/usage-statistics.vqlalong with the properties files you just modified, into to database “denodo_logs”.
This will create a new FULL CACHE view, usage_statistics_src and associated objects.
Enable the usage statistics in the Data Catalog. Follow these steps:
Configure the period to compute the usage statistics: Last day, Last month and/or All time.
Configure the elements over which to calculate the usage statistics: User, User Agent and/or Query.
Configure the connection details to the Virtual DataPort server containing the views over the Denodo Monitor requests log (needed to calculate the usage statistics), as explained before.
Database: the database where the views were created. If no value is specified in this field, the database of the Connection URI will be used.
View Name: the name of the view to query the usage statistics. The default value usage_statistics_src will be used if no value is specified in this field.
Compute the statistics:
At this point, the statistics need to be computed so Data Catalog can display them in the tab Statistics of the Data Catalog. To do this, there are two options:
Scheduled with Denodo Scheduler:
Create the INCREMENTAL_CACHE_LOAD stored procedure in Denodo Virtual DataPort.
Import the file data_catalog_statistics_scheduler.zip from
<DENODO_HOME>/resources/data-catalog/statistics/into Scheduler. It creates a project named data_catalog_statistics. This project contains a VDP data source that should be edited to point to the VDP server and database configured in the step 1 (make sure to configure it with a user with enough privileges on that server and database). Besides, it contains several jobs that will be different depending on the update you have installed.
Since Denodo Platform 7.0 update 20190903, this file includes the job load_cache_and_update_statistics, a VDP job that executes a view over the Data Catalog API to incrementally load the cache of the view usage_statistics_src and compute the usage statistics. You just need to schedule this job to be executed periodically. It is recommended to schedule it to be executed at least once a day (or more times if you desire to refresh the usage statistics more frequently).
If you are using an older update, use the following three jobs:
cache_load_first_time: A VDPCache job to load the cache of the view usage_statistics_src the first time. It is not mandatory to use this job, but it is recommended to use it if you have stored the Denodo Monitor logs for several days or weeks and you have not executed the job cache_load_incremental (explained in the next step) yet. Execute this job just once, and then use the job cache_load_incremental to load the cache incrementally, as explained in the following step.
cache_load_incremental: A VDP job to call the incremental cache load stored procedure over the view usage_statistics_src. It is recommended to schedule it to be executed at least once a day (or more times if you desire to refresh the usage statistics more frequently).
update_statistics: A VDP job to execute a view over the Data Catalog API to compute the usage statistics using the data obtained from the view usage_statistics_src. It is recommended to schedule it to be executed just after the job cache_load_incremental, on which it depends.
If your project contains the job load_cache_and_update_statistics, use it. Otherwise, you are forced to use the other three jobs as explained.
Manually from the Data Catalog:
You can compute the usage statistics by clicking the Compute usage statistics now button in the Data Catalog. The action dispatched with this button loads the cache with new data since the last time the cache was refreshed, and then computes the statistics (as Denodo Scheduler does, but in a scheduled way).
The recommended option to keep your statistics up to date is by using the Denodo Scheduler. You can modify the scheduling triggers according to your needs.