USER MANUALS

Azure Data Lake Gen 1

Warning

Azure Data Lake Storage Gen1 has been retired on February 29, 2024. If you use Azure Data Lake Storage Gen1, be sure to migrate to Azure Data Lake Storage Gen2.

Before deploying the Denodo Embedded MPP on Azure Kubernetes Service check Denodo Embedded MPP Azure Checklist to make sure you have everything you need.

There are two options to deploy a Denodo Embedded MPP that will access Data Lake Storage gen1 datasets:

  1. The recommended one: provide no credentials to the cluster.sh deploy command.

    cluster.sh deploy --credstore-password xxx
    

    Used when the Denodo Embedded MPP will run in Azure Kubernetes Service and will access Data Lake Storage gen1 using Azure Managed Identities.

    For this you need to add the following properties to the presto/conf/catalog/core-site.xml and hive-metastore/conf/core-site.xml, before the Embedded MPP is deployed:

    core-site.xml using Azure Managed Identities
    <property>
      <name>fs.adl.oauth2.access.token.provider.type</name>
      <value>Msi</value>
    </property>
    
    <property>
      <name>fs.adl.oauth2.msi.port</name>
      <value>PORT NUMBER for the REST endpoint of the token service
             exposed to localhost by the identity extension in the VM,
             (if different from the default of 50342)
      </value>
    </property>
    
  2. Provide the OAuth2 client credentials to the cluster.sh deploy command:

    cluster.sh deploy --adl-client-id www --adl-client-secret xxx --adl-tenant-id yyyy --credstore-password zzz
    
    • --adl-client-id: Client id

    • --adl-client-secret: OAuth2 refresh token from the Azure Active Directory service associated with the client id. If this secret is not specified in the command line, cluster.sh deploy will prompt for it, keeping secrets out of the bash history

    • --adl-tenant-id: id of the application’s Azure Active Directory tenant.

Add feedback