USER MANUALS

What Is New in Denodo 8.0 Update 8.0u20230301

This page lists the main enhancements of the update 8.0u20230301 of the Denodo Platform and the Solution Manager.

We also published these videos that explain the enhancements included in this update:

For a full list of enhancements and bug fixes, read the Release Notes of the update, in the Support Site or inside the zip file of the update.


Virtual DataPort

  • Virtual DataPort now has a procedural language to create stored procedures that invoke VQL statements and use elements such as conditions and loops. These are called VQL procedures. For example:

    CREATE OR REPLACE VQL PROCEDURE procedureUsingCursors
(idInternet OUT INTEGER, ttime OUT TIMESTAMP, idtax OUT VARCHAR)
AS (
  CURSOR cursorData IS 'SELECT iinc_id, ttime, taxid FROM internet_inc WHERE specific_field1 > 1';
  rinternet cursorData%ROWTYPE;
)
BEGIN
    OPEN cursorData;
    LOOP
      FETCH cursorData INTO rinternet;
      RETURN ROW ( idInternet, ttime, idtax) VALUES (rinternet.iinc_id, rinternet.ttime, rinternet.taxid);
      EXIT WHEN cursorData%NOTFOUND;
    END LOOP;
    CLOSE cursorData;
END
;

    VQL procedures are an alternative to developing stored procedures with the Java API of Denodo.

    Main benefits of these new procedures compared to Java stored procedures:

    • The SQL developers do not need to know Java to develop stored procedures.

    • VQL procedures are easier to modify than Java procedures: you do not need to recompile a Java class, package it in a .jar file and import it again; you can modify the procedure from the Design Studio, the Administration Tool or any client application.

    • It is easier for other users to review your procedure because they do not need to download the .jar file with the extension, open it and extract the Java class in another editor.

    Developing stored procedures with the Java API is still fully supported and has not changed.

  • This update includes embedded Massively Parallel Processing (MPP) capabilities to improve the performance on environments containing data in an object storage:

    • Added support for a special data source that allows to explore an object storage like Amazon S3 or HDFS and create base views over data, using Parquet format.

    • Improves the performance accessing these data by leveraging on the MPP capabilities of a customized PrestoDB cluster.

    • We are also going to release a new version of this utility that deploys a Presto cluster that has been customized to work with Denodo. This utility creates in Virtual DataPort a special data source (called “embedded_mpp”) that points to this PrestoDB.

    • The query optimizer now applies new acceleration techniques that have been designed for queries accessing this kind of data.

    These features require the subscription bundle Enterprise Plus.

  • Subqueries: before this update, you could use subqueries in the FROM and the WHERE clauses of the queries. Now, you can also use them in the SELECT clause. For example:

    SELECT name, (SELECT AVG(price) FROM product) AS average,
FROM product;

Virtual DataPort (Connectivity)

There are several improvements in the JDBC data sources:

  • Added pass-through credentials of OAuth credentials. You can configure a JDBC data source so when a client application queries a view that uses this data source, the data source uses the credentials of the user to connect to this database. Before this update, you could do this for databases that accept username/password authentication and Kerberos authentication. Now, you can do this for data sources that accept OAuth authentication. See JDBC Sources - Connecting to a JDBC Source with OAuth Authentication.

  • New JDBC adapters:

    • AlloyDB for PostgreSQL (also for the Cache Engine)

    • Exasol

    • MySQL 8 (also for the Cache Engine)

    • Spark SQL 3.x (also for the Cache Engine)

  • JDBC data sources: the PostgreSQL adapter now supports passing-through Kerberos credentials.

  • For Hadoop sources: added support to use Apache Knox when doing bulk data loads when the file system is HDFS.

Virtual DataPort (Cache Engine)

The Cache Engine creates tables in a database when enabling the cache of a view, to create a remote table and a summary table and to do a data movement. You can now customize the SQL statements the Cache Engine executes to create these tables (see Cache Table Creation Templates - VQL Guide).

Design Studio

The Design Studio now has a wizard to list the functions of Virtual DataPort you can use in queries. To open it, click the menu Help > Functions list.

Virtual DataPort (Operations and Security)

  • Added support to import and assign tags defined in Collibra: in the previous update (8.0u20220815), we added a stored procedure to obtain the tags assigned to views and their columns, from Collibra. This procedure returns the VQL statements that assigns the tags to the views and columns. We also added a procedure to do it from any catalog.

    This update includes a wizard to this graphically with Design Studio and the Administration Tool. To open it wizard, click the menu Administration > Semantic and Governance > Import tags from external catalog.

    In the pill CONFIGURATION of this wizard, you configure the connection to Collibra and in EXTERNAL CATALOG, where you want to obtain the tags. Once you review the tags obtained from Collibra, click Accept changes to apply the changes in the tags assigned to views and their columns.

Data Catalog

  • Data Catalog now has a “VQL Shell” similar to the one in Design Studio and the Administration Tool. With this, users can execute any query in Virtual DataPort without having to define a My query or create a view (see VQL Shell - Data Catalog Guide).

  • Added support to store the metadata of Data Catalog in PostgreSQL 13, and SQL Server 2016, 2017 and 2019.

Scheduler

Solution Manager

  • With this update, the Solution Manager now deploys revisions and does rollbacks much faster (faster by orders of magnitude)

  • Creating a revision is easier thanks to the new Differences Wizard that allows you to compare the metadata of two Virtual DataPort servers. This wizard helps you identify what elements you have to add to a revision.

  • REST API of Solution Manager:

    • Added operation to download the VQL of a revision.

    • Added operation to start and stop monitoring an environment.

Information about Deprecated Features

  • Version Control Systems Integration: the integration with Subversion and Microsoft Team Foundation Server (TFS) is now deprecated. The integration with Git, including Git repositories managed by Microsoft Team Foundation Server, is still fully supported.

  • The function IS_PROJECTED_FIELD of Virtual DataPort is now deprecated.

See also: Features Deprecated in Denodo Platform 8.0.

Add feedback