USER MANUALS

What Is New in Denodo 9.2

This page lists the main enhancements of the update 9.2 of the Denodo Platform and the Solution Manager.

We also published these videos that explain the enhancements included in this update:

For the full list of enhancements and bug fixes, read the Release Notes in the Support Site, in the page of this update.

Virtual DataPort

  • The Denodo Assistant is a set of features of the Denodo Platform that use artificial intelligence to make it easier to create and use virtual models. This update adds several features:

    • A feature to make it easier to create join views. In the tab Model of the view, there is a new button (Suggest Joins) to automatically create the join conditions of the view. This feature relies on the metadata of the views that form this view.

    • New feature to add extra instructions in the requests sent to the Large Language Model, when requesting suggestions to improve the metadata of the views (i.e., the description of a view, name of fields of a view and their description). To enable this, open Server configuration and go to DENODO ASSISTANT > General. This feature is also available for the Denodo Assistant of Data Catalog. This is useful to include in these requests the company’s business naming conventions.

    • Several new SQL functions that you can use in your queries: CLASSIFY_AI, ENRICH_AI_BINARY, EXTRACT_AI, SENTIMENT_AI, SUMMARIZE_AI and TRANSLATE_AI. The page Large Language Model Functions describes the requirements to use them.

  • When creating a summary, you can now set a custom load query that will be used to perform incremental loads of the summary view.

  • New SQL functions:

    • INITCAP: it capitalizes the first letter of each word in a given string and converts the other letters to lowercase.

    • ENCODE_BASE64 and DECODE_BASE64: functions to encode/decode text with the Base64 encoding scheme.

Virtual DataPort - Data Sources

  • JDBC adapters:

    • New JDBC adapters:

      • PostgreSQL 14, 15, 16 and 17 (Virtual DataPort already had adapters for earlier versions of PostgreSQL).

      • Microsoft Dataverse

      • Huawei GaussDB

    • The adapter for Amazon Aurora now supports the authentication method AWS IAM credentials.

    • The adapter for Google BigQuery now supports the authentication method Workload Identity Federation.

  • Virtual DataPort can also now connect to Huawei Object Storage Service (OBS). That is, to read files stored there, or when doing bulk data load.

  • The introspection dialog of the Embedded MPP now supports tables in the Delta Lake Universal Format (UniForm).

  • Improvement in the data sources DF (delimited files), JSON and XML that retrieve data from an HTTP server: these data sources can now do pagination with indices in requests of type POST.

Virtual DataPort - Execution Engine

  • To increase the stability of the Virtual DataPort server, the execution engine will limit by default the amount of memory used to calculate the execution plan of a query (see Limit the Maximum Amount of Memory of a Query).

  • If a client application connects to Virtual DataPort using the protocol Arrow Flight SQL and executes a query that is fully delegated to Snowflake, the execution is now much faster. That is because the execution engine can now obtain the result of the query in the Apache Arrow format from Snowflake and passes it to the client application, without the overhead of having to convert the data from one format to another. 9.1 already includes the same feature but for BigQuery.

  • The execution engine now calculates the execution plan of the query faster when the view has cache full and has many subviews.

  • The stored procedures CATALOG_VDP_METADATA_VIEWS and CATALOG_METADATA_VIEWS now use less memory. This is important for Data Catalog and other client applications that use these procedures.

Virtual DataPort - Cache Engine

Virtual DataPort - Version Control System (VCS)

  • New feature: Workspaces. A workspace is protected area of the Denodo server where users can create, edit, remove and test elements without affecting other users. Creating a workspace is like creating a Git branch in a software project. That is, making changes in their own workspace without affecting others. The changes of others do not affect them either. Once they complete the changes, they can push the changes to the repository and merge the changes from that branch into the main branch.

    With this feature, the SQL developers that use Denodo can follow the same workflow and best practices software developers use.

    The video Workspace Support for CI-CD explains this in detail.

Virtual DataPort - Operations and Security

  • Included support to restrict the query acceleration optimizations that involve moving a dataset to a different data source. The Denodo optimizer may decide to automatically copy a dataset of a data source to a temporary table in another data source to speed up specific data operations. This is called data movement.

    The administrators can now limit these data movements to enforce security requirements of organizations that must not move data from a region to another (see Restrict Data Movement Optimizations from Sensitive Data Sources.

  • Improvements in the credentials vault feature:

    • You can now configure LDAP data sources to obtain the credentials from a vault.

    • Each database of Denodo can now have its own “credentials vault” settings. That is, the administrator can configure a database to use different settings, not the global ones defined in Server Configuration. The data sources of that database will use those settings to retrieve the “secrets” from the vault.

Data Catalog

Significant improvements in the user interface of Data Catalog. Among other things, the home page, the summary of elements, the search page, data lineage, the search results, etc.

GraphQL Service

Client applications can now send requests to the GraphQL service that filter data by compound fields like arrays and registers (see Filtering on Complex Fields).

Scheduler

  • New type of job: DAG Cache Management (DAG stands for Direct Acyclic Graph) to load/refresh the cache of views and summaries. When you create this type of job, you only have to select a database. Then, every time Scheduler runs this job, it loads/refreshes the cache of all the views of that database. That is, the views that the database has at that moment. The benefits of this new type of job are:

    • It refreshes the cache “in order”. That is, starting from the base views, and then, the views above. The result is that the views will return data that is “fresher”.

    • It works in parallel. That is, if for example, a database has two views completely unrelated, Scheduler will refresh the cache of both at the same time.

    When you create this type of job, instead of selecting a database, you can also select several databases, or several folders within a database.

    We recommend using this type of job to refresh the cache of the views and summaries of a database. Note that the job Simple Cache Management (previously known as VDPCache is still fully supported).

    The video Automated Dependency Analysis for Replication Tasks explains this in detail.

  • We renamed the types of jobs to better reflect their purpose. Their functionality has not changed. For example, the job VDPCache is now called Simple Cache Management.

Solution Manager

Implemented an audit trail feature within the Solution Manager. This functionality records all user actions, providing a comprehensive audit trail for monitoring and analysis purposes.

Diagnostic & Monitoring Tool

The Diagnostic & Monitoring Tool can now create diagnostics from logs stored in a database, not just from local files.

Enhancements that Apply to Several Modules

Denodo Platform 9.2 introduces a significant enhancement to the user experience by enabling Data Catalog, Design Studio, and Scheduler to dynamically adapt their language based on the user’s browser preferences. This feature allows administrators to configure these web applications for multiple languages so it eliminates the previous limitation where these applications were either in English or in the language selected by the administrator, for all users. See more in Language of Data Catalog, Design Studio and Scheduler.

Information about Deprecated Features

With the release of this update, we have not deprecated any feature. The page Features Deprecated in Denodo Platform lists all the features that are deprecated.

Add feedback