What Is New in Denodo 8.0u20230914¶
This page lists the main enhancements of the update 8.0u20230914 of the Denodo Platform and the Solution Manager.
We also published these videos that explain the enhancements included in this update:
For a full list of enhancements and bug fixes, read the Release Notes of the update, in the Support Site.
Virtual DataPort
New feature Import Model. This feature reads a data model created with an application like ER/Studio Data Architect, ERwin Data Modeler, etc., and creates the equivalent interface views and associations in Virtual DataPort.
To use this, log in to Design Studio and click the menu Tools > Import model.
We already provided this functionality through a Desktop application called Model Bridge that you could download from the Support Site. This update integrates this functionality in to Design Studio.
New FinOps capabilities so administrators can monitor the data flows between their cloud providers:
In the configuration of a data source, you can select if the data source connects to an on-premise system or it is the cloud. If it is in the cloud, you can select the vendor and the region (e.g. AWS us-east1). This is for data sources JDBC, delimited files, JSON, XML and custom.
Applications that connect to Denodo using the JDBC driver can indicate the cloud provider and region they use as a property of the connection.
All this information is logged so administrators can use it to get insights about the usage of cloud data in the organizations, what regions and try to get pricing estimates.
See “Virtual DataPort Query Data Sources Monitor” in the page Virtual DataPort Monitors. We also published a video that explains these enhancements.
Resource Manager:
You can now configure the restriction of a plan to be applied only when the CPU consumption reaches a certain percentage during several consecutive measures.
New restrictions available:
Set the maximum number of concurrent queries per user.
Stop the queries that use more memory than a certain amount in megabytes.
In the definition of a restriction, when selecting Add variables to the query context, you can now use SQL functions whose result is a “text” value. Before you could only use literals.
In 8.0u20230301, we added a feature called VQL procedures, which allows you to create stored procedures using a procedural language. This language has a new command:
EXECUTE <DDL command:literal>that allows you to execute DDL statements like “CREATE VIEW”, “DROP VIEW”, etc. (see the page VQL procedures).
Virtual DataPort - Data Sources
New JDBC adapters:
Alibaba AnalyticDB for MySQL
Alibaba AnalyticDB for PostgreSQL
Alibaba ApsaraDB PolarDB MySQL
Alibaba ApsaraDB PolarDB PostgreSQL
Alibaba ApsaraDB RDS for MySQL
Alibaba ApsaraDB RDS for PostgreSQL
Alibaba ApsaraDB RDS for SQL Server
Teradata 17 (also for the Cache Engine)
Denodo Embedded MPP capabilities:
When you create base views over elements stored in the MPP database, you can now configure the number of files the data source will process to generate the schema of the base view (i.e. its fields). This feature makes it easier to deal with schema creep.
You can now do source refresh of JDBC base views that point to an object stored in the MPP database.
The following JDBC adapters can now read compound values:
Athena
Google BigQuery
Impala
Support for introspecting Iceberg tables in Presto and the Denodo Embedded MPP.
New stored procedure GET_USER_ACCOUNTS: returns the user accounts created in Virtual DataPort.
Virtual DataPort - Execution Engine
Dealing with very complex views that have data movement settings now consumes less memory.
Reduce the memory the query optimizer uses by removing from the nodes of the execution plan, the fields that the query does not require, in the first phase of the optimization of a query,
Virtual DataPort - Design Studio
Design Studio now stores in the user’s browser several settings, which makes it more user friendly. Among others:
The content of the VQL Shell, including the Execution logs so users can see their own queries the next time they log in.
VQL panel of elements in Design Studio: store in the users’ browser the options selected by the user the last time the user opened this panel.
Dialogs “Import”, “Export” and “Export Database” of the menu “File”: remember the latest selected options.
…
This feature uses the local storage of the browser of each user to store their own information. The administrator of Design Studio can disable this for all users, in the Design Studio Preferences.
Regarding the Execute and download feature of the VQL Shell and the Execution panel:
This operation is now more efficient by orders of magnitude thus avoiding any possible memory issues when using it.
By default, this feature now only returns the 500,000 rows. The administrator of Design Studio can remove this limit or modify it in the Design Studio Preferences
The Server Explorer now displays the columns of the views.
Virtual DataPort - JDBC Driver
When executing INSERT, UPDATE or DELETE statements in batch, the values of the batch are sent more efficiently to the server, which causes these statements to run much faster. That is, for applications that use java.sql.PreparedStatement#addBatch().
To take advantage of this, this update has to be installed in the server, and also use the JDBC driver of this update.
This also benefits the applications that connect to the ODBC interface of Virtual DataPort.
Virtual DataPort - ODBC Driver
Virtual DataPort - Operations and Security
The Denodo Monitor now generates a file called “vdpquerydatasources” that stores the details regarding the data sources that the execution engines accesses during the execution of a query: name of the data source, queries executed on each data source, number of rows the query returned, etc.
To take advantage of this feature:
In the Solution Manager: you do not have to do anything.
In the installation of the Denodo Platform: you do have to unzip the new
<DENODO_HOME>/tools/monitor/denodo-monitor.zipand follow the steps of the page Configuring the Denodo Monitor.
Data Catalog
Assisted Queries: with this feature, you can type a query in natural language (e.g. “tell me the top-10 customer by country”) to obtain the data you need, without having to know SQL or type complex SQL queries. Using Artificial Intelligence/Large Language Models (LLMs), this feature generates a SQL query that returns the data you need, based on the view’s metadata and your query.
To use it, open the view and click the new tab Assisted Queries.
See more about this in Assisted Queries.
New workflow capabilities: users can now initiate cases to request changes that will allow administrators and data stewards to modify or provide new elements as requested by the users.
The main goal of this enhancement is to request: access requests, data quality issues, metadata changes and general questions.
For more information, see Workflow Requests. We also published a video explaining this feature.
You can now search in the metadata (names of views, their columns, descriptions, etc.) using more advanced search terms like wildcards, fuzzy searches (the result includes results that are similar to the words you search), etc.
To use this, the administrator of Data Catalog has to enable Metadata Search first.
Scheduler
The administrator can now configure a template in HTML to generate the emails sent by Scheduler jobs. This page explains how to enable this feature.
New user privilege Read. Users that have this privilege can view a job, but not execute it. To execute a job, they require the privilege Execute. Users with the privilege Execute can view and also execute a job. See Privileges in Scheduler.
Solution Manager
The automated mode now creates cluster faster by not waiting for the Virtual DataPort nodes to be completely created.
Enhancements that Apply to Several Modules
Starting with this update, all the modules of the Denodo Platform and the Solution Manager enforce password policies. That is, the new passwords need to be eight characters or longer, use uppercase and lowercase characters, etc.
See more about this in the page Password Policies in the Denodo Platform and Solution Manager.
Note
For administrators that recently installed this update: everything will keep working for existing user accounts. These policies only affect new accounts or to users that try to change the password.
We also published a video that explains this enhancement.
Data Catalog and Design Studio now have a mechanism to prevent load balancers from closing the connection between the users’ browser and Data Catalog or Design Studio. This feature is disabled by default.
The page Prevent Load Balancers From Closing Connections of Data Catalog and Design Studio explains how to enable this.
Accessibility improvements in Data Catalog, Design Studio and Scheduler in areas like notifications they display, and aria-labels.
Documentation Release Notes
The zip files of the updates no longer include the RELEASE NOTES. Instead, read them in the description of the update, in the Support Site.
Information about Deprecated Features
With the release of this update, we have deprecated these features:
The command QUERY WRAPPER, which allows you to query ITPilot wrappers without creating a base view, is now deprecated. Do not use it to query ITPilot wrappers or any other types of wrappers.
As of Oct 09, 2024, the following features are not going to be available in the next major version (Denodo 9):
ITPilot
The Denodo4Eclipse Plugin
This list is not exhaustive; there could be more features that are deprecated that finally, will not be available in Denodo 9.
This features were deprecated with the release of Denodo 8.0.
The page Features Deprecated in Denodo Platform 8.0 lists all the features that are deprecated.