This section explains how to configure and personalize several aspects of the tool. It covers the following topics:
These parameters are global for all the users of the tool.
Elements Shown to the Users¶
You can specify which elements from the catalog the users are allowed to see:
- Only Views
- Only Web Services
It affects the results shown in the following sections:
You can choose whether to Allow users to browse by Databases/Folders or not. If this option is disabled, users will not be able to access the DB/Folders panel. In this case, you can also choose to Hide uncategorized views and web services from users (uncategorized views and web services will not appear in search results or when browsing by Categories or Tags).
You can customize this tool to adapt it to the branding of your company. You can change the following:
- The logo displayed on the top left.
- The image displayed on the background of the Login page.
- The image displayed on the background of the Welcome page.
Maximum size for uploaded images is 5MB.
Export Format Configuration¶
- The available export formats, among: CSV, HTML, Excel and Tableau. All enabled by default.
- The maximum number of rows using each of the available formats.
- If only the users with the role data_catalog_exporter in Virtual DataPort can export the results of the queries using each of the available exporters.
Database and View Connection URIs¶
You can choose whether to display the
Show Connection URIs button or not in
the databases’ and views’ details page. If you choose to show it, you can customize several
- Introduction text of the database connections panel: this will be shown in the panel instead of the default introduction.
- URI Patterns: these describe how to access the server’s databases through
JDBC, ODBC, REST and OData. You may use the following variables in the
$database_name. These variables will be replaced by the appropriate values when showing the URIs to the user.
The Data Catalog allows showing usage statistics of the views and web services by analyzing the usage data collected by Denodo Virtual DataPort. This feature requires some preparation steps, that will be explained later on this section.
Steps to Configure the Gathering of Statistics¶
In order to show the Usage tab in the details page of views and web services, a user with administration privileges has to perform the following steps:
Configure the Denodo Monitor to store the log of requests in a database (for instance, a table in Oracle). By default, Denodo Monitor saves the log information to a file. You need to modify this to use a database instead the log file.
<DENODO_HOME>/tools/monitor/denodo-monitor/conf/ConfigurationParameters.propertiesand search for the following text and follow the instructions.
# ================================================================================ # Settings to store the output of the Virtual DataPort query monitor in a database # ================================================================================
Data Catalog requires the Denodo Monitor included in update 20180926 or later.
Create the necessary views in a Virtual DataPort server:
Create a database named as denodo_logs. The next steps are assuming this database.
Create a base view over the previously configured table (request_notification) in the Denodo Monitor for the requests log.
- You have to name that base view as “request_notification” and include the columns with the default names.
Create the view usage_statistics_src with full cache enabled:
It is recommended to use the Denodo Virtual DataPort server shipped with the Solution Manager installer (but any other will work).
Enable the usage statistics in the Data Catalog:
Configure the period/s to compute the usage statistics: Last day, Last month and/or All time.
Configure the element/s over which to calculate the usage statistics: User, User Agent and/or Query.
Configure the connection details to the Virtual DataPort server containing the views over the Denodo Monitor requests log (needed to calculate the usage statistics), as explained before.
- Database: the database where the views were created. If no value is specified in this field, the database of the Connection URI will be used.
- View Name: the name of the view to query the usage statistics. The default value usage_statistics_src will be used if no value is specified in this field.
- Once you get to this step, you have everything configured but nothing is shown in the new statistics tab from the details page. This is because no statistics were computed yet. The next step, is to make the Data Catalog process the information stored in Denodo Virtual DataPort to compute and show the statistics. To do that, there are several options, as explained below.
Computing Usage Statistics¶
After having configured the Data Catalog to show the usage statistics, it needs to compute them. There are two ways to achieve this:
Scheduled with Denodo Scheduler:
You can use Denodo Scheduler to plan the computing of the statistics in an incremental way. Denodo Scheduler retrieves the raw data from the Denodo Monitor and loads it incrementally in the cache (with the new usage data). You have to follow these steps:
Create the INCREMENTAL_CACHE_LOAD stored procedure in Denodo Virtual DataPort.
- cache_load_first_time: A VDPCache job to load the cache of the view usage_statistics_src the first time. It is recommended to use this job if you have stored the Denodo Monitor logs for several days or weeks and you have not used the job load_cache_incremental to load the cache of its view incrementally. Execute this job once, and then use the job load_cache_incremental to load the cache incrementally from that moment.
- cache_load_incremental: A VDP job to call the incremental cache load stored procedure over the view usage_statistics_src. It is recommended to schedule it to be executed at least once a day (or more times if you desire to refresh the usage statistics most frequently).
- update_statistics: A VDP job to execute the view over the Data Catalog endpoint to compute the usage statistics using the data obtained from the view usage_statistics_src. It is recommended to schedule it to be executed just after the job cache_load_incremental, on which this depends.
This is the highly recommended option to keep your statistics up to date. You can modify the scheduling triggers according to your needs.
Manually from the Data Catalog:
You can compute the usage statistics by clicking the Compute usage statistics now button in the Data Catalog. Note that the action dispatched with this button does not load the cache with new data since the last time the cache was refreshed, so it only computes the statistics based on the current information stored in the Data Catalog (which might be outdated). This means this button is only useful if you have previously executed the job cache_load_incremental from Denodo Scheduler. You should rarely use this option, since the preferred way to update the statistics is by using the Denodo Scheduler.
The Compute usage statistics now button is intended for testing. It only recalculates the statistics with the data currently available in the view usage_statistics_src, but it does not load/update the cache of that view. So, if you want to calculate the statistics with the last available data, you should previously load/update the cache of the view usage_statistics_src (you can do it by manually executing the appropriate query or using the Scheduler jobs provided for that purpose).