USER MANUALS

AWS Glue Data Catalog

In case that you already have a AWS Glue Data Catalog containing table definitions you want to access from the Denodo Embedded MPP, you can use the AWS Glue Data Catalog as an external Metastore.

To do this, you can manually define a new catalog by creating the properties file in presto/conf/catalog/, e.g., presto/conf/catalog/glue_hive.properties. The file name, glue_hive, would be the catalog name.

Hive Tables

Hive catalog to read Parquet files from AWS Glue Data Catalog
connector.name=hive-hadoop2

hive.metastore=glue

# AWS region of the Glue Catalog
hive.metastore.glue.region=

# The ID of the Glue Catalog in which the metadata database resides
hive.metastore.glue.catalogid=

# Access Key and Secret Key for Glue
# Credentials and core-site.xml are not required when the MPP
# runs in EKS because it will use the IAM Roles for Service Accounts or the IAM EC2 instance profile,
# whichever is configured in EKS
hive.metastore.glue.aws-access-key=
hive.metastore.glue.aws-secret-key=
hive.config.resources=/opt/presto-server/etc/catalog/core-site.xml


# For Bulk Data load
hive.allow-drop-table=true
hive.non-managed-table-writes-enabled=true

hive.parquet.use-column-names=true

Delta Lake Tables

Delta catalog to read Delta Lake tables from AWS Glue Data Catalog
connector.name=delta

hive.metastore=glue

# AWS region of the Glue Catalog
hive.metastore.glue.region=

# The ID of the Glue Catalog in which the metadata database resides
hive.metastore.glue.catalogid=

# Access Key and Secret Key for Glue
# Credentials and core-site.xml are not required when the MPP
# runs in EKS because it will use the IAM Roles for Service Accounts or the IAM EC2 instance profile,
# whichever is configured in EKS
hive.metastore.glue.aws-access-key=
hive.metastore.glue.aws-secret-key=
hive.config.resources=/opt/presto-server/etc/catalog/core-site.xml
#

hive.parquet.use-column-names=true

Iceberg Tables

Iceberg catalog to read Iceberg tables from AWS Glue Data Catalog
connector.name=iceberg

iceberg.catalog.type=HIVE

hive.metastore=glue

# AWS region of the Glue Catalog
hive.metastore.glue.region=

# The ID of the Glue Catalog in which the metadata database resides
hive.metastore.glue.catalogid=

# Access Key and Secret Key for Glue
# Credentials and core-site.xml are not required when the MPP
# runs in EKS because it will use the IAM Roles for Service Accounts or the IAM EC2 instance profile,
# whichever is configured in EKS
hive.metastore.glue.aws-access-key=
hive.metastore.glue.aws-secret-key=
hive.config.resources=/opt/presto-server/etc/catalog/core-site.xml
#

hive.parquet.use-column-names=true

AWS Privileges for AWS Glue Data Catalog

The AWS privileges required by the IAM role of the Denodo Embedded MPP to access the AWS Glue Data Catalog are:

  • Reading from AWS Glue:

    • glue:GetDatabases

    • glue:GetDatabase

    • glue:GetTables

    • glue:GetTable

    • glue:GetPartitions

    • glue:GetPartition

    • glue:BatchGetPartition

  • Writing to AWS Glue. Same as for reading and also:

    • glue:CreateTable

    • glue:DeleteTable

    • glue:UpdateTable

    • glue:BatchCreatePartition

    • glue:UpdatePartition

    • glue:DeletePartition

Add feedback