What Is New in Denodo 8.0u20210715¶
This page lists the main enhancements added in the update 8.0 20210715 of the Denodo Platform and Solution Manager.
We also published these videos that explain the enhancements included in this update:
For the full list of enhancements and bug fixes, read the Release Notes in the Support Site, in the page of this update.
Data Catalog
This update includes a new version of the Data Catalog with great improvements: a new user interface, recommendations of views and web services, collaboration between users, and more.
Important
Before installing this update, export everything from the Data Catalog. After installing the update, import back everything. The section Upgrading from 8.0GA or 8.0u20210209 explains this in more detail.
If you have the AI & Recommendations FeaturePack, the Data Catalog now displays personalized recommendations to the users, based on their previous activity.
The language files for Data Catalog in Japanese, Simplified Chinese and French are now available for download. With this file, the administrator can change the language of Data Catalog. The language file in Simplified Chinese is also available for Design Studio. The page Language of the Web Administration Tools has the link to download these files and explains how to use them. This page also explains how the administrator can translate the user interface of this application and others to any language.
Added support to store the metadata of the Data Catalog in Oracle 18c and 19c. See the list of databases you can use to store the metadata.
The REST API of Data Catalog now has operations to query the search feature so you can integrate it in to another application. To learn how to invoke this API, open the visualizer of the API: https://denodo-server.acme.com:9443/denodo-data-catalog/swagger-ui.html#/search.
Common to All Components
We now provide support to run the Denodo Platform and the Solution Manager with the Red Hat build of OpenJDK version 11 (see appendix Supported Java Development Kits (JDK) of the Installation Guide).
Virtual DataPort
Automatic recommendation of summaries: Denodo Platform 8.0 has a new feature called Smart Query Acceleration that relies on a new type of view called Summary. This FeaturePack includes an AI-driven assistant that, using artificial intelligence, analyzes the past queries and recommends new summaries for query acceleration.
You can now load incrementally the cache of views, summary views, remote tables and temporary tables. That is, the execution engine will obtain the data from the sources and in the target table (cache table, table of the summary view,…), it will insert the new rows and update the rows that already exist. In some databases, this is known as UPSERT.
Examples:
Loading the cache of the view “sales” incrementally¶-- With "matching_pk", the cache engine uses the values of the primary key to -- identify which rows of the cache table have to be inserted and which -- ones have to be updated. SELECT * FROM sales WHERE sale_date > '2020-08-01' CONTEXT('cache_preload'='true', 'cache_invalidate'='matching_pk')
Updating the content of a summary view, a remote table or a temporary table¶-- This query uses the fields of the primary key of the view to detect -- rows that need to be updated and not inserted INSERT INTO customers_2020 ON DUPLICATE KEY UPDATE SELECT * FROM all_customers WHERE entry_date > '2020-01-01'
Updating the content of a summary view, a remote table or a temporary table¶-- This query uses the field "customer_id" to detect duplicate rows INSERT INTO customers_2020 ON DUPLICATE KEY (customer_id) UPDATE SELECT * FROM all_customers WHERE entry_date > '2020-01-01'
New function LOCALTIME: it returns the current time (hour, minute and second). That is, a value of type
time
. Previously, to obtain the current time of typetimestamp
, you had to execute this:CAST(CURRENT_TIMESTAMP AS TIME WITHOUT TIME ZONE)
We recommend using this new function because Denodo can delegate it to more databases than the expression above.
This complements the functions CURRENT_DATE (localdate), LOCALTIMESTAMP (timestamp) and CURRENT_TIMESTAMP (timestamptz).
Note that now, “LOCALTIME” is a reserved word so it cannot be used as a name of view, field, database, etc.
Performance improvements in several areas:
When a query uses a function that is not delegable to a database and this function does not reference any fields, the Execution Engine will evaluate the function and delegate the result to the database. This improves the delegation of some queries and may improve the execution time significantly.
The process of automatic simplification of queries is now faster when creating remote tables, and when non-administrator users execute queries that involve views with column privileges or row restrictions.
The Cost-Based Optimizer is now more efficient when selecting the execution plan for queries that involve large number of JOIN operations.
Virtual DataPort: Data Sources
Added support for a new database: SAP HANA 2.0 (also, as cache database). The data source can retrieve the statistics querying the system tables of this database (see Statistics that are gathered from the system tables of each vendor). The Cost-based Optimizer, if enabled, uses these statistics to calculate an optimal execution plan.
A data source can now obtain the statistics from the system tables of the following databases:
Hive
Google BigQuery
Oracle Essbase: in Denodo 8.0 GA (no updates), to do introspection of Essbase, the user account required the privilege “manager (database)” in Essbase. This is no longer necessary. In the previous update (8.0u20210209) this feature was available but disabled by default; now it is enabled by default.
The multidimensional data sources that connect to SAP can now obtain an SAP logon ticket on behalf of the user that is executing the query. The goal is to execute the query in SAP on behalf of the final user instead of with a service account.
Virtual DataPort: Cache Engine
Added support for new cache databases:
Google BigQuery
SAP HANA 2.0
Virtual DataPort: Web Services Published by Denodo
The index page of the RESTful web service now lists the databases of Virtual DataPort. Before, it only showed a welcome message. That is, in http://denodo-server.acme.com:9090/denodo-restfulws/, you will see the list of databases.
The REST web services now provide an OpenAPI document version 3.0. These documents describe the REST web services and make it easier to develop applications that interact with these web services. The REST web services already provided an OpenAPI 2.0 document. See more in the section OpenAPI / Swagger of the Administration Guide.
Important
We strongly recommend increasing the amount of memory allocated to the Tomcat of Denodo Platform. To be on the safe side, we suggest doubling the amount of memory. This is necessary because the REST web services of Denodo, to publish the OpenAPI 3.0 specification, now use more memory.
Virtual DataPort: Design Studio
Following the roadmap, more features of the Administration Tool have been ported to the Design Studio:
Users and roles: you can now manage users and roles, grant them privileges, roles, etc.
Version Control System integration (VCS): users can now do check in and check out when the VCS is Subversion or Microsoft TFS. With the previous update, you already can do this with Git.
Virtual DataPort: Operations and Security
You can now configure the JDBC data sources to retrieve the credentials from these password vaults:
AWS Secrets Manager
Azure Key Vault
CyberArk Vault (already available in the previous update)
HashiCorp Vault
The main benefit of using a password vault is that you avoid storing in the Denodo the service accounts of your JDBC data sources. In addition, it is easier to rotate the credentials of the service accounts because Denodo retrieves them automatically from the vault whenever is necessary. See more about this in Credentials Vault.
Denodo Monitor:
The Denodo Monitor can now store logs the in AWS S3.
When you configure Denodo Monitor to store logging information in Oracle or SQL Server, the Denodo Monitor can use Kerberos authentication to connect to these databases.
See Configuring the Denodo Monitor to see how to enable these features.
For security purposes, the web container of Denodo (Apache Tomcat) now refuses HTTP requests with the method OPTIONS. With earlier updates the method OPTIONS was allowed. The reason for this change was that the method OPTIONS provides information that may be used by an attacker to get additional information about this server. However, the CORS mechanism relies on the server (i.e. the web container of Denodo) to accept the method OPTIONS and provide certain information.
If you are using CORS, follow the steps described in the section Initial Set-Up of CORS of the Administration Guide.
OData Service
The OData service now executes less metadata queries. This improves the performance of the requests to this service.
Scheduler
There is a new type of job, to load and refresh Virtual DataPort “summaries”.
Applications can now connect to the REST API of Scheduler using HTTP Basic authentication.
When Scheduler is configured to store the metadata in Oracle and SQL Server, you can now configure this connection to use Kerberos authentication.
Solution Manager
The REST API of the Solution Manager has new operations:
Update the AWS credentials of the Solution Manager (see Update AWS Global Credentials and Update AWS Environment Credentials).
Import and export settings (see Export Solution Manager Catalog and Configuration and Import Solution Manager Catalog and Configuration).
Release the license usage of a specific server (see Free License Usage by Server Id).
Start, stop and restart clusters that were created in automated cloud mode (see Start Automated Mode Cluster and Stop Automated Mode Cluster)
You can now configure graphically the monitoring features of the Solution Manager. Before, you had to modify the configuration files of the Solution Manager.
Documentation Release Notes
If you enabled SSL in your installation of Denodo (in Virtual DataPort), we now recommend adding the parameter ssl=true to the connection URL. That is, the connection URL defined in the applications that connect to Denodo using the JDBC driver. E.g.
jdbc:denodo://denodo-server.acme.com:9999/customer360?ssl=true&queryTimeout=100000&chunkTimeout=1000&userAgent=myApplication
By setting this property to “true”, the driver only tries to establish SSL connections. Without this property, the driver tries to establish a non-SSL connection first and if SSL is enabled on the Server, the connection fails and immediately, the driver tries to establish an SSL connection.
We also recommend enabling the same option in the Administration Tool. To do this, click the menu Tools > Admin Tool Preferences. Then, click Connection and select Use only SSL communications.
Setting this to “true” is particularly beneficial in cloud environments because the connection between the client applications and Denodo is not as fast as within a local network. This parameter was already available since 8.0 GA and all the updates of 7.0. Now we recommend to use it whenever possible.
Information about Deprecated Features
No features have been marked as deprecated with the release of this update.
See also: Features Deprecated in Denodo Platform 8.0.