USER MANUALS

Delta Lake

Delta Lake is table format that extends Parquet data files with a file-based transaction log for ACID transactions and scalable metadata handling.

The Denodo Embedded MPP is distributed with a predefined catalog named delta connected to the Embedded Hive Metastore for accessing Delta Lake tables.

delta.properties
connector.name=delta

# Embedded Hive Metastore
hive.metastore.uri=thrift://hive-metastore:9083
hive.metastore-timeout=20s

hive.config.resources=/opt/presto-server/etc/catalog/core-site.xml

Note

Delta is a restricted catalog, so it is not listed on the From MPP Catalogs tab of the Embedded MPP data source.

To query Delta Lake tables you have to create those tables in the Denodo Embedded MPP. You can use the embedded data source in Denodo to graphically explore Delta Lake tables, register them in the Denodo Embedded MPP and create the base views in Denodo. You can check Object Storage Data in Open Table Formats for more details on how to connect an Object Storage graphically.

Explore Delta Lake tables

Explore Delta Lake tables

Important

To graphically explore Delta Lake tables you need the Denodo subscription bundle Enterprise Plus.

Features

The Denodo Embedded MPP provides the following features when treating with Delta Lake tables:

  • Graphically explore Delta Lake datasets, create tables in the MPP and base views in Denodo (From object storage tab of the Embedded MPP data source)

  • Create base views over existing Delta Lake tables in an External Metastore (From MPP Catalogs tab of the Embedded MPP data source). To do this create a new catalog in the Embedded MPP, because the predefined catalog named delta is restricted, so it cannot be accessed from the From MPP Catalogs tab.

  • Querying

    Note

    Delta protocol version (3, 7) is supported from Denodo Embedded MPP 20241007

  • Embedded MPP Acceleration

Limitations

  • Bulk data load

  • Caching: full cache mode

  • Remote tables

  • Inserts: it is not possible to insert data into views created from Delta Lake tables using the From Object Storage tab of the Embedded MPP data source. See Manage Views Created from Object Storage for details.

Supported Operations by Metastore Type

Operation

Embedded Hive Metastore

External Hive Metastore

AWS Glue Data Catalog

Read

Yes

Yes

Yes

Create/Insert

No

No

No

Update

No

No

No

Delete

No

No

No
Add feedback