USER MANUALS

Nessie Catalog

If you are already using Nessie Catalog to manage your Iceberg tables, the Denodo Embedded MPP can integrate with as an external Metastore. This allows you to leverage Nessie for your data lake tables and access them directly from Denodo.

To connect to a Nessie Catalog, you must define a new catalog within your Denodo Embedded MPP configuration.

The recommended method for defining new catalogs is by using the presto.catalog property in your values.yaml file. This approach simplifies management and upgrades. Once configured, this new catalog will be accessible from the From MPP Catalogs tab in the Denodo Embedded MPP data source.

Create Nessie Views From MPP Catalog

Create Nessie Views From MPP Catalog

Here is an example of an Iceberg catalog named nessie configured to connect to a Nessie Catalog within your values.yaml:

  catalog:
    nessie: |-
     connector.name=iceberg
     iceberg.catalog.type=nessie
     iceberg.catalog.warehouse=/tmp
     iceberg.nessie.uri=http://<nessieuri>:19120/api/v1
     iceberg.nessie.auth.type=BASIC
     iceberg.nessie.auth.basic.username=
     iceberg.nessie.auth.basic.password=
     iceberg.nessie.read-timeout-ms=
     hive.s3.path-style-access=true

     hive.pushdown-filter-enabled=true
     hive.parquet-batch-read-optimization-enabled=true

As an alternative to using values.yaml, you can define a new catalog by creating a separate properties file directly in the presto/conf/catalog/ folder of the Denodo Embedded MPP Helm chart (e.g., presto/conf/catalog/nessie.properties). The file name, nessie in this example, will become the catalog name in Embedded MPP.

Properties

Property Name

Description

iceberg.nessie.uri

Nessie API endpoint URI (required). Example: https://localhost:19120/api/v1

iceberg.nessie.auth.type

The authentication type to use. Available values are BASIC or BEARER.

iceberg.nessie.auth.basic.username

The username to use with BASIC authentication. Example: test_user

iceberg.nessie.auth.basic.password

The password to use with BASIC authentication. Example: my$ecretPass

iceberg.nessie.read-timeout-ms

The read timeout in milliseconds for requests to the Nessie server.

Note

Nessie manages the metadata for your Iceberg tables, but the actual data files are typically stored in an Object Storage (like Amazon S3 or Azure Data Lake Storage). Depending on the location of these Iceberg tables, you may need to provide credentials for that specific object storage to the Denodo Embedded MPP cluster. To provide these credentials, use Kubernetes secrets before deploying the MPP cluster.

As an example, if your Iceberg data is stored in AWS S3 and requires explicit credentials (e.g., if you’re not using EKS Pod Identities, IAM Roles for Service Accounts or EC2 instance profiles), you have to create a Kubernetes secret like this:

kubectl create secret generic mpp-credentials
--from-literal=METASTORE_DB_PASSWORD=hive
--from-literal=AWS_ACCESS_KEY_ID=awsaccesskeyid
--from-literal=AWS_SECRET_ACCESS_KEY=awssecretaccesskey

Additionally, you need to enable the security credentials in your values.yaml by setting:

objectStorage:
  aws:
    securityCredentials:
      enabled: true

Supported Operations

The following table summarizes the operations supported by the Denodo Embedded MPP when connecting to a Nessie Catalog:

Operation

Iceberg

Read

Yes

Create/Insert (*)

Yes

Update

No

Merge

No

Delete (**)

Yes

(*) To support write operations in Iceberg catalogs other than the predefined iceberg catalog, you must configure Denodo to recognize these additional catalog names by executing:

SET 'com.denodo.util.jdbc.inspector.impl.PrestoJDBCInspector.iceberg.catalogNames' = 'iceberg, other_iceberg, another_iceberg';

(**) Delete operations are supported from Denodo 9.2.0. For Iceberg V1 tables, Denodo can only delete data in one or more entire partitions. All columns used in the WHERE clause must all be identity transformed partition columns of the target table.

Add feedback