What Is New in Denodo 7.0 Update 20190903¶
This page lists the main enhancements added in Denodo 7.0 update 20190903. For a full list of enhancements and bug fixes, check the RELEASE NOTES of the update, inside the zip file of the update or in the Support Site.
- This update includes several performance improvements that reduce the execution time, CPU usage and memory usage of several operations:
- Calculate the lineage and the tree view
- Obtain the dependencies of the views
- Generate the VQL of the views
- Update the configuration of a data source
- Generating the SQL queries to executed on a database.
- Optimizing the execution plan of a query.
In views with several hundreds of columns, these improvements provide a tenfold reduction in the execution time of these operations.
To benefit from these enhancements, you need to re-import the VQL of entire Server.
- The query optimizer can now push down the operations GROUP BY and JOIN under an extended UNION (i.e. a UNION of views that have a different schema).
- Added support to use boolean expressions in the SELECT clause of the queries.
Virtual DataPort (Data Sources)
- Added support for new databases: Cassandra, Amazon Athena and Elasticsearch 6.7 (Elasticsearch 6.5 was already supported). The appendix Supported JDBC Data Sources of the Administration Guide lists all the officially supported databases.
- The data sources DF, JSON and XML now support mutual authentication. That is, using a private key for authentication.
- Denodo can now gather statistics from the system tables of DB2 for z/OS (list of databases from which Denodo can gather the statistics of tables/views from their system tables).
Virtual DataPort (Cache Engine)
- Data sources using pass-through credentials can now be used as cache database. This helps maximize the number of queries that the execution engine can execute in the same data source. Before using this feature, read the appendix Considerations When Configuring Data Sources with Pass-Through Credentials of the Virtual DataPort Administration Guide.
Virtual DataPort (Administration Tool)
- Performance improvements in the communications between Virtual DataPort and the administration tool. These improvements are noticeable in WAN environments and slow connections. Enable this option Use WAN optimized communications in the dialog of the menu Tools > Admin Tool preferences (tab Connection).
- Views are opened faster because now, the tool loads some information of the views on demand. E.g. the tool only requests the information about the tree view if the user opens that dialog.
- Users no longer need to modify the Windows registry to use single sign-on. With previous updates, the user has to modify the registry. To benefit from this feature, you need to update Virtual DataPort and the administration tool.
- “Execute” dialog and VQL Shell: the options “Limit rows” and “Stop query when the limit is reached” have been renamed to “Display rows” and “Retrieve all rows” respectively to make clearer what these options do.
Virtual DataPort (JDBC Driver)
- The JDBC driver can now be loaded from client applications that support Java 9, 10 or 11.
- Users no longer need to modify the Windows registry to connect to Denodo with single sign-on. With previous updates, you have to modify the registry. To benefit from this feature, you need to do the following:
- Update Virtual DataPort and use the latest JDBC driver.
- Add some libraries to the classpath of the client application. See more details in the section Connecting to Virtual DataPort Using Kerberos Authentication of the Developer Guide.
Virtual DataPort (Operations and Management)
Now, you can automatically store the execution trace of the queries that fail or all the queries.
In this update, Virtual DataPort and the administration tool - not Scheduler - have changed the way they store the Kerberos debug information in order to facilitate gathering this information. Now, this information is stored in the same file as other logging information (“vdp.log” and “vdp-admin.log”). The benefits are that you do not need to launch the applications differently to obtain this information and each Kerberos message has a timestamp.
There is an additional step when obtaining the Kerberos debug information: adding a line to the file “log4j2.xml”. The sections linked above explain how to do this.
This change affects any log messages that with previous updates, Virtual DataPort printed in the standard output. For example, if you add the system property
-Djavax.net.debug=allto Virtual DataPort, the output will now be stored in the log file of the component where you added this property (e.g. for Virtual DataPort, in “vdp.log”) and you need to change the “log4j2.xml” file in the same way you do to enable the Kerberos debug mode.
- The Data Catalog has new privileges that allow for a more fine-grained control of what users can do:
|Tasks||Roles Required in Virtual DataPort To Perform These Tasks|
|Assign categories, tags and custom properties groups to views and web services.||data_catalog_classifier|
|Edit views, web services, databases, tags, categories, custom properties groups and custom properties.||data_catalog_editor|
|Can do the same as a user with the roles “data_catalog_editor” and “data_catalog_classifier”.||data_catalog_manager|
|Configure the personalization options and content search.||data_catalog_content_admin|
|Can perform any action.||data_catalog_admin|
See more details about these roles in the section Administration of the Data Catalog Guide.
- Better response times of the dialogs “Element Management”, “Tags” and “Content Search Configuration”.
- Performance improvements in the metadata search and in the metadata synchronization with Virtual DataPort. This is noticeable on servers with a lot of elements.
- The option Compute usage statistics now updates the data about usage before calculating the statistics.
- You can now export and import the metadata of the Data Catalog (categories, tags, settings, etc.) programmatically. You can use a script or use the REST API of the Data Catalog. The goal is to facilitate the promotion of the metadata across environments.
- Added support to store the metadata of the Data Catalog in the PostgreSQL edition of Amazon Aurora.
Enhancements that Apply to Several Components
- If you enabled SSL/TLS on any of the modules of Denodo, you can now store the password of the keystore encrypted. See how to do it in the sections Enabling SSL/TLS in Denodo Platform Servers, Enabling HTTPS in the Embedded Apache Tomcat and Enabling SSL/TLS in Solution Manager Servers.
- You can now configure all the components to store the logs on Amazon S3.