Personalization¶
From the Personalization dialog you can configure the aspect of the Data Marketplace and the available features in its interface. It is organized in the following tabs:
Elements. The features related to the elements of the catalog are configured here.
Information. In this tab, you will find the features related to the information of the catalog.
Export. In this tab you configure which options are available to export the results of a query.
Connection. From this tab you can customize the connection URLs and decide where to show them.
Usage statistics. You can enable the usage statistics in this tab and which options are available.
Data preparation. This tab allows you to configure the strategy for loading data in a data preparation session.
Denodo Assistant. This tab includes the configuration for denodo assistant of the Data Marketplace.
Theme. You will find here a dialog to redefine the visual aspect of the Data Marketplace.
Language and region. In this tab you can configure the available languages to display the user interface and some metadata in the Data Marketplace and set default internationalization for query results.
Elements¶
In the Search and browse section you configure the following settings:
Select the elements that will be shown to the users. You can specify if the users have access to views, web services or external elements. In addition to searching and browsing, this setting will affect the home page and the saved queries.
Choose how the elements will be displayed by default. Select the display mode of the elements by default: list or grid.
Allow users to browse by Databases/Folders. If disabled, the Browse menu will not show the Databases / Folders option.
Show only views and web services belonging to any category. In case the browsing by database and folders is disabled, you can decide if the Data Marketplace will show all elements in the server when searching or browsing, or only those elements with a category.
Show the elements belonging to the selected category or any of its subcategories. If enabled, the search in Browse > Categories will show the elements belonging to the categories selected or its subcategories.
Allow logical names for view fields. If enabled, users will see the logical field name column in the views Schema tab. In addition, users will be able to search views containing certain logical field names in metadata search when the search filter Field name is applied.
Search and browse section in the Elements tab¶
In the Queries section you can configure the following settings:
Allow saving queries (show “My queries” menu). If disabled, users are not allowed to save queries, and the My queries menu will not be shown.
Number of relationship levels to expand. This option sets the depth level for expanding one-to-one associations. The deepest level of association is five and the lowest level is one.
Show generic error messages. If enabled, you can configure the generic error message to be shown when an error occurs while users execute or export queries.
Allow copy results to clipboard. If disabled, users are not allowed to copy the displayed query results to the clipboard in the VQL Shell.
In addition, you can configure some options regarding suggestions about filters and derived fields used to build queries:
Collect data on executed queries to make smart suggestions. If enabled, the information required to make suggestions is collected when users execute queries in Data Marketplace, even if the smart suggestions are not enabled.
Enable smart suggestions. If enabled, the Data Preparation wizard, the Query wizard and the VQL Shell will display smart suggestions about filters and derived fields to build queries.
Delete old suggestions. With this option, you can delete suggested filters and derived fields not used in any query since the specified date.
Note
Suggestions about filters and derived fields feature is only available with the AI & Denodo Assistant FeaturePack. To find out the subscription bundle you have, open the About dialog of the Data Marketplace.
Queries section in the Elements tab¶
In the Descriptions and properties section you can configure the following settings:
Enable HTML descriptions and properties. You decide if HTML is supported in descriptions for databases, views, web services, categories or tags; and if a property group supports HTML type properties.
If disabled:
Element descriptions will always be displayed as plain text,
Existing HTML type properties will be converted to type Long text, and
Property groups will not allow to create properties of type HTML.
Enable images in HTML. With this option you can allow images in the HTML descriptions. The images will be displayed within an iframe (sandboxed for security reasons). When allowing images, you must specify the domains on which they are located. These domains will be white-listed in the Content Security Policy.
If you enable images, a button
will appear in the HTML editor that will open a popup to specify the image URL, width, and height.
If disabled:
Existing HTML type properties will be converted to remove the images.
Dialog to add an image¶
Descriptions and properties section in the Elements tab¶
Information¶
In the Informative message section you can configure an informative message that will be displayed to users as a modal pop-up before they log in the Data Marketplace. It can be used to display a welcome message, inform users when the service is expected to be down for maintenance, legal disclaimers, etc.
Informative message shown before a user logs in the Data Marketplace¶
In the editor you can enter plain text or HTML text with the help of the
toolbar. Use the icon in the toolbar to enter HTML code in the editor.
Dialog to configure the informative message¶
In the Others section, with the Enable community resources you decide if the related documentation resources will be displayed in the modal pop-up when the documentation icon is clicked. If disabled, only the documentation link will be displayed.
Others section in the Information tab¶
Export¶
In the General section you can configure the following settings about the exportation of the query results of views and web services:
The available export formats, between
CSV,HTML,Excel,Tableau (.tde)andTableau (.hyper). By default, all formats are enabled.See also
The section Enabling Exports to Tableau explains how to configure the Denodo Platform to enable the exports to Tableau.
If only users with the
data_catalog_exporterrole in Virtual DataPort can export the results of the queries using each of the available exporters.If Data Marketplace should generate these files with a header by default. This setting only applies to
CSV,HTMLandExcelfiles. Exporting toTableau (.tde)andTableau (.hyper)always include the header.Data Marketplace allows BOM configurable for exported query CSV file.
The maximum number of rows that will be exported using each of the available formats. First, you need to check the Limit maximum number of rows to export option and then provide a value in the corresponding Maximum numbers of rows field.
The field delimiter character used when exporting to a
CSVfile. You can choose betweenComma,Pipeline,Semicolon,SpaceandTab; or click thebutton and enter a user-defined delimiter character.
Note
The double quote character (
") cannot be set as a delimiter character.
General section in the Export tab¶
In the Excel section you can configure how numeric and date values are exported to Excel. The following settings can be configured:
Export numbers as. Choose the Excel format that should be used to export numeric values between the options:
GeneralNumber. If you choose this option, you should specify in the field Number of decimals how many decimals will be displayed in the cell for decimal values.Text
Note
If you export numbers with
GeneralorNumberformats, Excel will only store the 15 most significant figures. If you want to preserve all figures, you should export them asText.Date pattern. The pattern to be applied to values of type
localdateif results are not internationalized or if you choose an i18n map not defining a value for the fielddatesubtypepattern.Time pattern. The pattern to be applied to values of type
timeif results are not internationalized or if you choose an i18n map not defining a value for the fieldtimesubtypepattern.Timestamp pattern. The pattern to be applied to values of type
timestampif results are not internationalized or if you choose an i18n map not defining a value for the fielddatepattern.
Note
The syntax used in date, time and timestamp patterns is the same as the Java date patterns
detailed in the section Date and Time Pattern Strings.
Excel section in the Export tab¶
Enabling Exports to Tableau¶
To enable the feature of exporting data to Tableau, you need to install the
Tableau Data Extract API (for .tde format) or Extract API 2.0 (for
.hyper format) in the host where the Denodo Platform is installed.
To configure Tableau libraries, follow these steps:
Go to the directory
<DENODO_HOME>/resources/data-catalog/tableau.Extract the content of the compressed file in the
Tableau_Data_Extract_APIfolder to configure Tableau Data Extract API or in theExtract_API_2.0folder to configure Extract API 2.0.If the Denodo server runs on Windows, copy the files from the recently decompressed
binfolder to<DENODO_HOME>/jre/bin.If the Denodo server runs on Linux:
Copy all the
.sofiles fromlib64/dataextract(for.tdeformat) orlib64/tableausdk(for.hyperformat) in the decompressed folder to<DENODO_HOME>/jre/bin. Remember to verify that the files just copied have the necessary permissions.Copy the files from
bin(for.tdeformat) orbin/hyper(for.hyperformat) in the decompressed folder to<DENODO_HOME>/jre/bin.
Restart the Data Marketplace.
Important
According to your specific Linux distribution, you should follow the next considerations:
Red Hat and Ubuntu distributions. It is necessary to copy all the
.sofiles mentioned in section 4.1 to/libdirectory of the Linux root folder with the appropriate permissions.CentOS distributions. It is necessary to copy all the
.sofiles mentioned in section 4.1 to/lib64directory of the Linux root folder with the appropriate permissions.
It is necessary to have the libncurses5 library installed, so it is recommended to check if this library is installed on the system. If a later version of this library is installed, it is necessary to make a symbolic link like the one indicated below since the Tableau libraries need that specific version to work.
ln -s libtinfo.so.<version> libtinfo.so.5
(e.g. If the installed library is libncurses6 then the command to use would be ln -s libtinfo.so.6 libtinfo.so.5)
Take into account that there is a limitation in Linux environments that prevents the Tableau libraries for both formats from being installed at the same time. In these environments it is only allowed to configure libraries for one format (.tde or .hyper) in Data Marketplace.
Connection¶
You can choose whether to display the Connection URLs button or not in the Summary tab of databases, views and web services. If you choose to show it, you can customize several parameters:
Introduction text of the connections panel. This value will be shown in the panel instead of the default introduction.
URL patterns. These values describe how to access the databases and views of the server through JDBC, ODBC, REST, OData and GraphQL; and the URL for accessing web services. You may use the following variables in the pattern definitions:
$protocol,$vdp_host,$vdp_port,$odbc_port,$webcontainer_port,$user_name,$database_nameand$view_name. These variables will be replaced by the appropriate values when showing the URLs to the user.
Dialog to configure the connection URLs for databases, views and web services¶
In the External Tools section you can configure the integration with third-party applications.
Power BI. Enable this option to show the Power BI button in the Summary tab of views, which generates a data source file to query the view from Power BI.
DSN. By default, a DSN-less connection string with the data of the server the user is connected to in the Data Marketplace is used. It is also possible to specify a custom DSN or a DSN-less connection string. For details about the DSN-less connection string format go to Denodo Power BI Custom Connector.
Dialog to configure the integration with Power BI¶
Tableau. Enable this option to show the Tableau button in the Summary tab of views, which generates a data source to query the view from Tableau.
Connector. The connector to use to create the connection. Denodo, that uses ODBC, or Denodo JDBC.
Connection. Parameters needed to create the connection. By default, the data of the server the user is connected to in the Data Marketplace is used. It is also possible to specify different values by selecting Use custom connection parameters.
JDBC connection parameters. Additional connection parameters. Only for the Denodo JDBC connector.
Dialog to configure the integration with Tableau¶
Usage Statistics¶
Virtual DataPort produces metadata on every query it processes like how long it takes to execute, how many rows it returns, who executes it, from which application, etc. Data Marketplace can analyze this metadata and show a set of statistics that summarizes how views and web services are queried. When this feature is enabled, the details page for views and web services includes a new tab called Usage with this information.
To enable this feature you must perform some preparation steps:
Configure the Denodo Monitor to store the requests log in a database.
Configure Virtual DataPort to read, process and cache the requests data from database.
Enable usage statistics in Data Marketplace.
Compute usage statistics.
Important
Denodo Monitor and Virtual DataPort must be configured with the same time zone so the cache works properly.
Let us see in more detail each one of the preparation steps.
Configure the Denodo Monitor to Store the Requests Log in a Database¶
By default Denodo Monitor saves monitoring data to a set of files. Follow the steps below to configure Denodo Monitor to store requests data in a database:
Edit the file
<DENODO_MONITOR_HOME>/conf/ConfigurationParameters.properties.Search for the text
# ================================================================================ # Settings to store the output of the Virtual DataPort query monitor in a database # ================================================================================
Follow the instructions below the text.
In the page Configuring the Denodo Monitor you have more information on how to manually configure the Denodo Monitor.
As an alternative, you can configure the Denodo Monitor from the Solution Manager. Take into account that a view or web service may exist on several environments at the same time. According to your preferences on how Data Marketplace should compute usage statistics, you must configure the Denodo Monitor in one of the following ways:
If you want Data Marketplace to compute usage statistics globally, the requests of all your environments must be stored in the same table. For that purpose it is enough to define the global monitoring configuration in Solution Manager.
If you want to separate out usage statistics by environment, the requests of each environment must be stored in a different table. Therefore, you need a specific monitoring configuration for each environment pointing to a different table.
Configure Virtual DataPort to Read, Process and Cache the Requests Data From Database¶
Next you have to configure Virtual DataPort to read the requests data from database, process it and be able to cache the results in an incremental way. Follow the steps below:
Enable the cache engine of Virtual DataPort.
Create a database called
denodo_logs. You can use another name if you want, but take into account that if you configure Scheduler to automatically compute usage statistics later on, you will have to modify the URL of the Virtual DataPort data source to point to the correct database.Create a JDBC data source pointing to the database where the Denodo Monitor stores the requests data.
Create a base view over the
request_notificationtable of the data source you just created. It is important that you do not modify the name of the view or the fields in its schema.Import the file
<DENODO_HOME>/resources/data-catalog/statistics/usage_statistics.vqlintodenodo_logsdatabase. This will create theusage_statistics_srcview configured with full cache, which is the view used by Data Marketplace to compute usage statistics.
Note
Due to license restrictions, you cannot use the Virtual DataPort server included in the Solution Manager.
Enable Usage Statistics in Data Marketplace¶
From the Usage Statistics tab of the Personalization dialog you must proceed as follows:
Enable this feature using the toggle Enable usage statistics.
Select all the periods for which to compute usage statistics:
Previous day,Previous 30 daysorAll time.Select if you want to compute the most frequent values for
User,User Agent (Applications)orQuery.Configure the connection details to the Virtual DataPort that contains the
usage_statistics_srcview:Connection URL. The connection URL of the Virtual DataPort server, which follows the pattern
//<host>:<port>/<database>.Login and Password. The credentials of the user. Take into account that this user must have enough privileges to connect to Virtual DataPort and execute the
usage_statistics_srcview.Database. The database that contains the
usage_statistics_srcview. You only need to enter a value if this database is different from the database in the Connection URL.View name. The view with the processed information on requests. By default it is
usage_statistics_src. You only need to enter a value if you have changed its default name.
Click the Save button to apply the configuration.
Dialog to configure the usage statistics¶
Compute Usage Statistics¶
At this point, you are ready to compute usage statistics so Data Marketplace can display them in the Usage tab of views
and web services. Just click the Compute usage statistics button. This process incrementally caches the data in the
usage_statistics_src view and then computes usage statistics for each element in the catalog mentioned in the
queries.
You can manually compute usage statistics whenever you want. However, our recommendation to keep them up to date is to run this process regularly using Scheduler, specially if there are several Virtual DataPort servers registered in Data Marketplace. To do this, follow the steps below:
Edit the file
<DENODO_HOME>/resources/data-catalog/statistics/compute_usage_statistics_api.propertiesas follows:Adapt the value of the property ending in
data_catalog_api_no_authentication.ROUTE.HTTP.URLso it points to your Data Marketplace.Adapt the value of the property ending in
data_catalog_api_vdp_authentication.ROUTE.HTTP.URLso it points to your Data Marketplace.Enter the credentials of the user that is going to run the process to compute the usage statistics in the properties ending in
data_catalog_api_vdp_authentication.ROUTE.HTTP.USERanddata_catalog_api_vdp_authentication.ROUTE.HTTP.PASSWORD. By default the password must be unencrypted. To enter it encrypted, proceed as follows:Obtain the encrypted value for properties file. For the password
my_passwordyou must execute in Virtual DataPort the command:ENCRYPT_PASSWORD 'my_password' FOR_PROPERTIES_FILE
Enter the result in the property ending in
data_catalog_api_vdp_authentication.ROUTE.HTTP.PASSWORD.Enter the value
ENCRYPTEDin the property ending indata_catalog_api_vdp_authentication.ROUTE.HTTP.PASSWORD.ENCRYPTED.
Import the file
<DENODO_HOME>/resources/data-catalog/statistics/compute_usage_statistics_api.vqlwith the properties file you just edited in thedenodo_logsdatabase of Virtual DataPort. This will create thecompute_usage_statistics_all_serversview, which runs the compute usage statistics process for all Virtual DataPort servers registered in Data Marketplace. This view produces a row for each server and astatuscolumn with theOKvalue if the process succeeded. Take into account that the same credentials are used to run this process on each server, so there are some requirements that must be met for the process to be successful:The usage statistics must be enabled for the server.
The user is able to connect to the server.
The user has the
Personalizepermission in the server.
Import the file
<DENODO_HOME>/resources/data-catalog/statistics/compute_usage_statistics_scheduler.zipin Scheduler. This will create thedata_catalog_statisticsproject, which contains:A Virtual DataPort data source named
VDP.An
Individual Queryjob namedcompute_usage_statistics_all_serversthat executes a query over the view with the same name.
Edit the following properties of the
VDPdata source:Connection URI. Enter the connection URI of the Virtual DataPort that contains the
compute_usage_statistics_all_serversview.Username and Password. Enter the credentials of the user. Take into account that this user must have enough privileges to connect to Virtual DataPort and execute the
compute_usage_statistics_all_serversview.
Edit the triggers of the
compute_usage_statistics_all_serversso it runs in a regular basis according to your needs. Our recommendation is to run this job at least once a day.
Data Preparation¶
A data preparation session starts loading data from a view. Not all the view data set, but a representative subset of the view data. This data is manipulated in real-time by the user to build a query. This is achieved by executing queries on the original data every time the user applies any action that modifies the data.
There are several strategies to build this representative subset of view data. You have to choose the most appropriate strategy for your environment, since it may have a great impact in performance.
In the Data preparation section you can configure how this representative subset of data is built, with the following options:
Number of results to show in the data preparation table. Number of rows to display in the table of a data preparation session. Note that the number of displayed rows may be lower than this value in the situations below:
When the view is smaller than the number specified.
If fewer rows match the filters applied to the data.
Query the view / Use the complete dataset. Executes a query on the view when loading the data and after each action performed in the data preparation table. Take into account that querying the view for each action may be slow and will put unnecessary load in the data source.
Use a sample of the dataset. Builds a sample of the view data when the Load data button is clicked to start the data preparation session. This sample is stored in Virtual DataPort as a temporary table. All actions done later will query the temporary table with the sample instead of the view. This is the recommended option for most environments.
To configure the sampling strategy, you have to define the following parameters:
Number of results of the sampling. The number of results from the view that will be considered for the sample.
Sampling method. The method to decide which data retrieve from the view. You have the options below:
Take the first results. The sample will contain the first results of querying the view.
Random. The data to build the sample will be retrieved in a random order from the view.
Note
Take into account the following considerations when the sampling strategy is selected:
To use temporary tables in Virtual DataPort, the cache has to be enabled.
Users need the
data_catalog_data_preparationrole to build queries with data preparation. This role grants privileges to create temporary tables in Virtual DataPort, so there is no need for thecreate_temporary_tablerole.Check the Temporary Tables page for more information.
Denodo Assistant¶
Assisted Query Configuration¶
In the Assisted Query pill of the Denodo Assistant tab you can configure the Assisted Query feature.
Assisted Query parameters.¶
Enable query generation. This enables or disables the feature.
Warning
Enabling this feature allows the Data Marketplace to send metadata information to the LLM when using it. Only if the data usage is enabled, data of the view is also going to be processed and sent to the LLM in order to improve the results. Make sure your organization is aware and agrees on this.
Enabling this feature will result in incurring expenses for requests made to an external API. Make sure that the organization is fully cognizant of this financial implication prior to enabling this option.
Note
To use the Assisted Query feature the user has to have the data_catalog_assisted_query role but to configure the Assisted Query feature the user must have the personalization role. See the Default Roles section for more information.
Use sample data. This enables the Data Marketplace to send actual data of the view to the LLM. This can help improving the results when the LLM needs to create queries that contain conditions.
Note
For using the data usage option of the Assisted Query, the VDP cache must be enabled. The section Configuring the Cache explains how to do this.
Sample data size. Configure the number of rows to retrieve for sampling. This will create a subset of rows to send to the LLM, which can improve query results. The maximum value is 500 rows.
Important
When the value of the Sample data size is changed, the Data Marketplace must be restarted.
Language options. This option allows the possibility to use multiple language configurations for the query explanation.
User locale. The language that corresponds to the user locale configured in the Data Marketplace is used.
Language. The language selected from a list of languages is used.
Browser language. The language configured in the web browser is used.
Execution mode. Choose wether if you have to execute the generated query yourself after the generation is done or if it is automatically executed.
Add extra instructions. This enables the Data Marketplace to send extra instructions that the Denodo Assistant will include in the request sent to the LLM. This option is intended to provide global context or business naming conventions.
Extra instructions. The extra instructions that the Denodo Assistant is going to send to the LLM.
Note
The utilization of the automatic execution mode will extend the duration of the assisted query generation process because after the query is generated it will automatically execute it, ultimately delaying the presentation of the final result. It is advisable to take this into consideration when assessing whether you wish to verify the query’s correctness against your requirements before execution. In such cases, opting for the manual mode is strongly recommended.
Theme¶
Data Marketplace contains two differents themes to change the interface: Default and Dark. Both themes can be customized to adapt to the branding of your company. Additionally, new themes can be created and customized.
In the theme tab, an example card is available that shows the characteristics of the theme in the selected pill. This card updates the changes applied during customization to provide a preview of the current theme.
Note
Only users with personalization permissions can create or customize themes but every user could change the local Data Marketplace theme between the avaliable ones.
Theme tab and example card¶
To customize a theme click the Edit theme button this shows the Theme settings popup where the current parameters can be modified:
The colors. You can also invert the colors of some elements (i.e. the page header), this will allow you to build a dark theme.
The border and shadows of the dialogs and forms.
The fonts.
The logo, header and other images of the application.
Note
The maximum size for uploaded images is 5MB.
Dialog to configure the theme of the Data Marketplace¶
Click on Apply theme to persist the changes when the Theme settings popup is closed. The changes will only be applied to the current tab of your browser and the example card in the pill. Other tabs or browsers will ignore these changes. Additionally, it is possible to Import or Export the value of the current theme or restore the default values to restart the theme customization.
If you want to persist your changes, click the Save button. You need to restart the Data Marketplace for your changes to take effect.
You can always go back to the original settings by clicking the Restore defaults button.
If you want to save your settings to a file, use the Export button. To load the settings from a file, use the Import button.
The Enable button status of a theme allows it to be used by every user. A theme with Disable status can be modified but cannot be applied by any user.
Note
If a user has activate locally a theme that is deleted or disabled the user’s theme become the default theme.
Note
This customization is for interface rebranding for internal use only, without redistribution rights.
Language and Region¶
In the User interface section you configure the following settings:
Default language. You can specify the default language used to display the Data Marketplace user interface. You can choose among the enabled languages added in the Available languages table and English (preinstalled). The language selected as default cannot be removed nor disabled.
Available languages. By clicking Add language button, a pop up is opened where you can add a language that will be available to display the user interface. You should select the desired language, attach its language file (customLang_<language_tag>.properties. E.g., customLang_en-US.properties for US English) in the drag and drop file upload zone and click on Add button. The language will be added to the table.
Add language for user interface popup¶
The Available languages table has the following columns:
Language. Generated automatically based on the language tag.
Language tag. Language tags are codes defined by the IETF BCP 47 standard. For the English language, the code is en. You can also specify the language used in a country. For example, en-US for the English of the United States. Other examples are zh for Chinese, es for Spanish, ja for Japanese or ar for Arabic.
Enabled. If disabled, this language could not be used to display the user interface.
Click
to download the language file for a specific language.
Click
to upload a new language file for a specific language.
Click
to remove a language from the table.
User interface section in the Language and region tab¶
In the Metadata section you can configure the following settings:
Enable field descriptions in the user’s language. When enabled, field descriptions will be shown in the language selected by the user in Language and region settings. If a field does not have a description in the user’s language, the default description will be displayed.
You can define which languages will be available for field descriptions. These languages will be shown to the user when choosing its language and will be the languages available to create field descriptions.
Available languages. By clicking Add language button, a pop up is opened where you can add a language that will be available to display field descriptions in the user’s language. You should select the desired language and click on Add button. The language will be added to the table.
The Available languages table has the following columns:
Language. Generated automatically based on the language tag.
Language tag. Language tags are codes defined by the IETF BCP 47 standard. For the English language, the code is en. You can also specify the language used in a country. For example, en-US for the English of the United States. Other examples are zh for Chinese, es for Spanish, ja for Japanese or ar for Arabic.
Enabled. If disabled, this language could not be selected by the users and new field descriptions can not be created.
Click
to remove the language from the table.
Warning
Deleting a language will also delete all the descriptions created in that language. If you do not want the language to be shown but also want to keep the existing descriptions, disable the language instead of deleting it.
Metadata section in the Language and region tab¶
In the Query results section you can specify the default configuration to internationalize results in Data Marketplace:
Internationalize. If enabled, query results will be internationalized by default.
Locale. If Internationalize is enabled, this option defines the default locale to be used when internationalizing query results.
This default configuration applies for all users who have not configured specific settings to internationalize results in Language and region settings.
Query results section in the Language and region tab¶
