What Is New in Denodo 8.0 Update 20210715

This page lists the main enhancements added in the update 8.0 20210715.

We also published these videos that explain the enhancements included in this update:

For a full list of enhancements and bug fixes, read the Release Notes of the update, in the Support Site or inside the zip file of the update.


Data Catalog

This update includes a new version of the Data Catalog with great improvements: a new user interface, recommendations of views and web services, collaboration between users, and more.

Important

Before installing this update, export everything from the Data Catalog. After installing the update, import back everything. The section Upgrading from 8.0GA or 8.0u20210209 explains this in more detail.

If you have the AI & Recommendations Feature Pack, the Data Catalog now displays personalized recommendations to the users, based on their previous activity.

  • Added support to store the metadata of the Data Catalog in Oracle 18c and 19c. See the list of databases you can use to store the metadata.

  • The REST API of Data Catalog now has operations to query the search feature so you can integrate it in to another application. To learn how to invoke this API, open the visualizer of the API: https://denodo-server.acme.com:9443/denodo-data-catalog/swagger-ui.html#/search.

Common to All Components

We now provide support to run the Denodo Platform and the Solution Manager with the Red Hat build of OpenJDK version 11 (see appendix Supported Java Runtime Environments (JRE) of the Installation Guide).

Virtual DataPort

  • Automatic recommendation of summaries: Denodo Platform 8.0 has a new feature called Smart Query Acceleration that relies on a new type of view called Summary. This Feature Pack includes an AI-driven assistant that, using artificial intelligence, analyzes the past queries and recommends new summaries for query acceleration.

  • You can now load incrementally the cache of views, summary views, remote tables and temporary tables. That is, the execution engine will obtain the data from the sources and in the target table (cache table, table of the summary view,…), it will insert the new rows and update the rows that already exist. In some databases, this is known as UPSERT.

    Examples:

    Loading the cache of the view “sales” incrementally
    -- With "maching_pk", the cache engine uses the values of the primary key to
    -- identify which rows of the cache table have to be inserted and which
    -- ones have to be updated.
    
    SELECT *
    FROM sales
    WHERE sale_date > '2020-08-01'
    CONTEXT('cache_preload'='true', 'cache_invalidate'='matching_pk')
    
    Updating the content of a summary view, a remote table or a temporary table
    -- This query uses the fields of the primary key of the view to detect
    -- rows that need to be updated and not inserted
    
    INSERT INTO customers_2020
    ON DUPLICATE KEY UPDATE
    SELECT * FROM all_customers WHERE entry_date > '2020-01-01'
    
    Updating the content of a summary view, a remote table or a temporary table
    -- This query uses the field "customer_id" to detect duplicate rows
    
    INSERT INTO customers_2020
    ON DUPLICATE KEY (customer_id) UPDATE
    SELECT * FROM all_customers WHERE entry_date > '2020-01-01'
    
  • New function LOCALTIME: it returns the current time (hour, minute and second). That is, a value of type time. Previously, to obtain the current time of type timestamp, you had to execute this:

    CAST(CURRENT_TIMESTAMP AS TIME WITHOUT TIME ZONE)
    

    We recommend using this new function because Denodo can delegate it to more databases than the expression above.

    This complements the functions CURRENT_DATE (localdate), LOCALTIMESTAMP (timestamp) and CURRENT_TIMESTAMP (timestamptz).

  • Performance improvements in several areas:

    • When a query uses a function that is not delegable to a database and this function does not reference any fields, the Execution Engine will evaluate the function and delegate the result to the database. This improves the delegation of some queries and may improve the execution time significantly.

    • The process of automatic simplification of queries is now faster when creating remote tables, and when non-administrator users execute queries that involve views with column privileges or row restrictions.

    • The Cost-Based Optimizer is now more efficient when selecting the execution plan for queries that involve large number of JOIN operations.

Virtual DataPort: Data Sources

  • Added support for a new database: SAP HANA 2.0 (also, as cache database). The data source can retrieve the statistics querying the system tables of this database (see Statistics that are gathered from the system tables of each vendor). The Cost-based Optimizer, if enabled, uses these statistics to calculate an optimal execution plan.

  • A data source can now obtain the statistics from the system tables of the following databases:

    1. Hive

    2. Google BigQuery

  • Oracle Essbase: in Denodo 8.0 GA (no updates), to do introspection of Essbase, the user account required the privilege “manager (database)” in Essbase. This is no longer necessary. In the previous update (8.0u20210209) this feature was available but disabled by default; now it is enabled by default.

  • The multidimensional data sources that connect to SAP can now obtain an SAP logon ticket on behalf of the user that is executing the query. The goal is to execute the query in SAP on behalf of the final user instead of with a service account.

Virtual DataPort: Cache Engine

Added support for new cache databases:

  • Google BigQuery

  • SAP HANA 2.0

Virtual DataPort: Web Services Published by Denodo

  • The index page of the RESTful web service now lists the databases of Virtual DataPort. Before, it only showed a welcome message. That is, in http://denodo-server.acme.com:9090/denodo-restfulws/, you will see the list of databases.

  • The REST web services now provide an OpenAPI document version 3.0. These documents describe the REST web services and make it easier to develop applications that interact with these web services. The REST web services already provided an OpenAPI 2.0 document. See more in the section OpenAPI / Swagger of the Administration Guide.

Virtual DataPort: Design Studio

Following the roadmap, more features of the Administration Tool have been ported to the Design Studio:

  • Users and roles: you can now manage users and roles, grant them privileges, roles, etc.

  • Version Control System integration (VCS): users can now do check in and check out when the VCS is Subversion or Microsoft TFS. With the previous update, you already can do this with Git.

Virtual DataPort: Operations and Security

  • You can now configure the JDBC data sources to retrieve the credentials from these password vaults:

    • AWS Secrets Manager

    • Azure Key Vault

    • CyberArk Vault (already available in the previous update)

    • HashiCorp Vault

    The main benefit of using a password vault is that you avoid storing in the Denodo the service accounts of your JDBC data sources. In addition, it is easier to rotate the credentials of the service accounts because Denodo retrieves them automatically from the vault whenever is necessary. See more about this in Credentials Vault.

  • Denodo Monitor:

    • The Denodo Monitor can now store logs the in AWS S3.

    • When you configure Denodo Monitor to store logging information in Oracle or SQL Server, the Denodo Monitor can use Kerberos authentication to connect to these databases.

    See Configuring the Denodo Monitor to see how to enable these features.

  • For security purposes, the web container of Denodo (Apache Tomcat) now refuses HTTP requests with the method OPTIONS. With earlier updates the method OPTIONS was allowed. The reason for this change was that the method OPTIONS provides information that may be used by an attacker to get additional information about this server. However, the CORS mechanism relies on the server (i.e. the web container of Denodo) to accept the method OPTIONS and provide certain information.

    If you are using CORS, follow the steps described in the section Initial Set-Up of CORS of the Administration Guide.

OData Service

The OData service now executes less metadata queries. This improves the performance of the requests to this service.

Scheduler

  • There is a new type of job, to load and refresh Virtual DataPort “summaries”.

  • Applications can now connect to the REST API of Scheduler using HTTP Basic authentication.

  • When Scheduler is configured to store the metadata in Oracle and SQL Server, you can now configure this connection to use Kerberos authentication.

Solution Manager

Documentation Release Notes

  • If you enabled SSL in your installation of Denodo (in Virtual DataPort), we now recommend adding the parameter ssl=true to the connection URL. That is, the connection URL defined in the applications that connect to Denodo using the JDBC driver. E.g.

    jdbc:denodo://denodo-server.acme.com:9999/customer360?ssl=true&queryTimeout=100000&chunkTimeout=1000&userAgent=myApplication
    

    By setting this property to “true”, the driver only tries to establish SSL connections. Without this property, the driver tries to establish a non-SSL connection first and if SSL is enabled on the Server, the connection fails and immediately, the driver tries to establish an SSL connection.

    We also recommend enabling the same option in the Administration Tool. To do this, click the menu Tools > Admin Tool Preferences. Then, click Connection and select Use only SSL communications.

    Setting this to “true” is particularly beneficial in cloud environments because the connection between the client applications and Denodo is not as fast as within a local network. This parameter was already available since 8.0 GA and all the updates of 7.0. Now we recommend to use it whenever possible.

Information about Deprecated Features

No features have been marked as deprecated with the release of this update.

See also: Features Deprecated in Denodo Platform 8.0.