USER MANUALS

What Is New in Denodo 9.5

This page lists the main enhancements of the update 9.5 of the Denodo Platform and the Solution Manager.

We also published these videos that explain the main enhancements included in this update:

  • Trusted Metrics and KPIs: Denodo 9.5 introduces Metric Views, a new semantic-layer object for defining, governing, and reusing business metrics and KPIs.

  • AI-Assisted Data Development: Denodo 9.5 advances the VQL Shell of Design Studio toward a more natural-language-driven development experience.

  • Data Marketplace: overview of updates included in the latest release of the Data Marketplace.

  • Denodo Lakehouse Accelerator: key features related to the new version of the Denodo Lakehouse Accelerator.

For the full list of enhancements and bug fixes, read the Release Notes in the Support Site.

Virtual DataPort

  • New type of view called Metric View. This type of view allows data engineers to declare analytical metrics calculated directly from underlying base or derived views, simplifying the development of enterprise KPIs (see documentation). Data Marketplace has a specific view for metric views.

  • The Denodo Assistant of the VQL Shell has evolved into a Large Language Model (LLM) agent. In previous updates, it was a question-and-answer chatbot. Now, it is an interactive assistant that - using natural language - helps you generate, fix, and explain queries, explore views, and analyze data. To do this, it orchestrates multiple API calls sequentially to fulfill the users’ requests (documentation).

    To enable this feature, first configure the Denodo Assistant and an LLM under Server configuration > Denodo Assistant. Then, you can customize the agent’s behavior in the Chatbot Configuration.

  • You can now use the clauses LIMIT and OFFSET in subqueries and in the definition of a derived view (i.e. in the CREATE VIEW statement).

  • New metadata propagation mechanism: this dynamically calculates and assigns explicit field sizes for most of the text functions (CONCAT, TRIM, LOWER, UPPER…) and CASE statements. For instance, concatenating a 10-character field with a 4-character constant now explicitly propagates a field size of 14. Previously, the length of fields resulting from a text operation was 65,536 by default. This caused some third-party client applications to default to their maximum allowable character lengths when allocating memory for the result set. Now, by providing these precise field lengths in the view metadata, Denodo significantly optimizes memory allocation and data transfer performance for consumer applications.

    This feature is disabled by default. To enable it, see Propagating Properties for Derived Text Fields.

Virtual DataPort - Data Sources

  • New SQL database adapters:

    • Amazon Athena: there was already an adapter for Athena. This new one uses the new Athena JDBC 3.x driver. The existing adapter renamed to Athena 2.x uses the Athena JDBC 2.x driver, and it has been deprecated.

    • Databricks: there was already an adapter for Databricks. This new one uses the new open-source JDBC driver developed by Databricks. Databricks recommends using this new driver because it includes improvements such as support for Unity Catalog, metric views, faster large-result retrieval, etc. The existing adapter has been deprecated because it uses the Simba driver, which Databricks considers legacy.

    • Teradata 20

  • JSON data source: added support for NDJSON. To use it, look for the check box Process newline-delimited JSON (NDJSON) format in the configuration of the JSON data source.

    NDJSON stands for Newline-Delimited JSON. It is a convenient format for storing and streaming structured data where each individual line is a valid JSON object.

  • If you have data sources that store Hadoop properties that are sensitive, you can now store them encrypted. To do it, select these properties and select the new check box Encrypted. This enhancement is for the data sources of type JDBC, Delimited File, XML, and JSON.

Virtual DataPort - Execution Engine

  • Transitive join condition delegation: the execution engine can now calculate transitive equality relationships across complex multi-way outer join operations. This enables accurate branch pruning across partitioned tables even when queries are dynamically generated by drag-and-drop actions in self-service tools.

  • Union pruning: optimized query rewrite behaviors to trigger union branch pruning rules even if a condition sits directly below a correlated subquery definition.

Virtual DataPort - Materialization Engine (previously known as Cache Engine)

Virtual DataPort - Web Services Published by Denodo

  • You can now enable a “readiness endpoint” in the REST web service (i.e. https://denodo-server:9443/denodo-restfulws), to monitor if it is ready to receive requests. Disabled by default; see the documentation to enable it.

Virtual DataPort - Design Studio

  • Revamped the SUMMARY of views and associations to display more clearly the type of view, its database and its folder.

Data Marketplace

  • Denodo Assistant Features of Data Marketplace:

    • The chatbot is now a Large Language Model agent. This allows the chatbot to orchestrate and step through multiple API calls within a single user response.

    • The assistant now pre-processes queries by analyzing metadata to detect ambiguity; if multiple interpretations are possible, it will prompt the user to clarify their intent.

    • Reduced token usage up to 80% by optimizing some prompts.

  • The feature Reference Lineage has been rebranded to 360 Graph to better reflect its capabilities. There is a new option to hide non-highlighted components. Useful for complex views.

  • Several improvements in the user interface of the features Query and Data Preparation.

Lakehouse Accelerator

The REGISTER_EMBEDDED_MPP stored procedure now allows you to specify whether the Denodo Lakehouse Accelerator deployment uses the traditional Java engine or the new C++ engine Denodo Lakehouse Accelerator on Velox.

In both cases, the procedure creates the “embedded_mpp” data source. This data source now automatically includes a specific JDBC adapter - either Lakehouse Accelerator or Lakehouse Accelerator on Velox - instead of the generic Presto adapter. This change allows Denodo to effectively manage the distinct capabilities of each engine and apply specific optimizations, improving its performance particularly when the Lakehouse Accelerator is configured as the Materialization Engine (formerly known as the Cache Engine).

Scheduler

  • In JDBC and VDP data sources, you can now use a Kerberos keytab for authentication.

  • New authentication method for Elasticsearch data sources: API key. This is one of the authentication methods of Elasticsearch Cloud Serverless.

Solution Manager

  • New operations in the REST API of Solution Manager:

    • Start and stop the monitoring of a cluster (see Start, Stop and Status). The REST API already had operations to do this for environments, but not for clusters.

    • Modify the configuration of the metadata database (documentation).

Enhancements that Apply to Several Modules

  • Support for credentials vault:

    • New centralized credentials vault configuration: the administrator can now define the connection to the credentials vault of your organization, at the installation level and then, reuse this configuration in all the modules. This eliminates the need to repeat the same configuration steps for each module.

    • Added support to obtain - from a vault - the credentials to connect to the metadata database. This is for Data Marketplace, Diagnostic & Monitoring Tool, Scheduler, Solution Manager and Virtual DataPort.

  • Added native support for Gemini Enterprise Agent Platform (formerly known as Vertex AI) in Virtual DataPort and Data Marketplace. This allows Denodo Assistant to leverage Gemini as a Large Language Model (LLM) provider for conversational interfaces and an embedding model provider for vector-based search tasks.

Information about Deprecated Features

With the release of this update, we have deprecated these features:

  • In Virtual DataPort:

    • The adapter Amazon Athena 2.x. Consider moving to the new adapter Amazon Athena 3.x.

    • The following adapters have been deprecated because the vendor no longer provides support to these versions. For all these, there is a newer adapter available.

      • Microsoft SQL Server 2012 and 2014

      • MySQL 4 and 5

      • Netezza 4.6, 5 and 6

      • Oracle 8i, Oracle 9i, 10g, 11g, TimesTen 11g, 12c, 12c In-Memory, E-Business Suite 12 and 18c

      • PostgreSQL 8, 9, 10, 11, 12 and 13

      • SAP HANA 1

      • Teradata 12, 13, 14, 15, 16 and 17

      • Vertica 7

    • The Simba-based Databricks adapter is now deprecated and renamed to Databricks (Simba driver) (deprecated). We recommend switching to the new Databricks adapter, which features the official open-source JDBC driver.

    Important

    Follow the steps of How to Modify the JDBC Adapter of an Data Source to update the adapter of your JDBC data sources.

The page Features Deprecated in Denodo Platform lists all the features that are deprecated.

Add feedback