Azure Data Lake Gen 2¶

Before deploying the Denodo Lakehouse Accelerator (formerly known as Denodo Embedded MPP) on Azure Kubernetes Service check Denodo Lakehouse Accelerator Azure Checklist to make sure you have everything you need.

There are three options to deploy a Denodo Lakehouse Accelerator that will access Data Lake Storage gen2 datasets:

1. The recommended one: Add MSI Tenant ID and Client ID in the kubectl create secret command.

   Used when the Denodo Lakehouse Accelerator will run in Azure Kubernetes Service and will access Data Lake Storage gen2 using Azure Managed Identities.

   You have to provide the following properties to the kubectl create secret command before the Lakehouse Accelerator is deployed:

   ENV Variable	Description
   AZURE_OAUTH_MSI_TENANT_ID	MSI Tenant ID
   AZURE_OAUTH_CLIENT_ID	Client ID

   kubectl create secret generic mpp-credentials
   --from-literal=METASTORE_DB_PASSWORD=hive
   --from-literal=AZURE_OAUTH_MSI_TENANT_ID=azureoauthmsitenantid
   --from-literal=AZURE_OAUTH_CLIENT_ID=azureoauthclientid
   

   It’s also necessary to set true the objectStorage.azure.managedIdentities.enabled property in values.yaml

   Run helm install sentence

   helm install lakehouseaccelerator lakehouseaccelerator/
   

2. Provide OAuth2 client credentials to the kubectl create secret command.

   You have to provide the following properties to the kubectl create secret command before the Lakehouse Accelerator is deployed:

   ENV Variable	Description
   AZURE_OAUTH_DIRECTORY_ID(*)	Microsoft Directory ID or Tenant ID
   AZURE_OAUTH_CLIENT_ID	Client ID
   AZURE_OAUTH_CLIENT_SECRET	Client Secret

   (*) Note that AZURE_OAUTH_DIRECTORY_ID is https://login.microsoftonline.com/${env.AZURE_OAUTH_DIRECTORY_ID}/oauth2/token

   kubectl create secret generic mpp-credentials
   --from-literal=METASTORE_DB_PASSWORD=hive
   --from-literal=AZURE_OAUTH_DIRECTORY_ID=azureoauthdirectoryid
   --from-literal=AZURE_OAUTH_CLIENT_ID=azureoauthclientid
   --from-literal=AZURE_OAUTH_CLIENT_SECRET=azureoauthclientsecret
   

   It’s also necessary to set true the objectStorage.azure.oauth2ClientCredentials.enabled property in values.yaml

   Run helm install sentence

   helm install lakehouseaccelerator lakehouseaccelerator/
   

3. Provide the Azure credentials for the Shared Key authentication method to the kubectl create secret command before the Lakehouse Accelerator is deployed:

   ENV Variable	Description
   ABFS_STORAGE_KEY	Azure Data Lake Gen 2 Storage Key

   kubectl create secret generic mpp-credentials
   --from-literal=METASTORE_DB_PASSWORD=hive
   --from-literal=ABFS_STORAGE_KEY=abfsstoragekey
   

   It’s also necessary to set true the objectStorage.azure.sharedKey.enabled and add the Azure Data Lake Gen 2 Storage Account in the objectStorage.azure.sharedKey.account properties in values.yaml.

   Run helm install sentence

   helm install lakehouseaccelerator lakehouseaccelerator/