Introduction
The “Import Tags from External Catalog” functionality in Denodo Virtual DataPort (VDP) allows users to import the tags from an instance of Microsoft Purview and add them to the corresponding views and columns in VDP as tags. Tags are retrieved from Purview assets that have been previously imported from VDP using the Denodo Governance bridge for Microsoft Purview.
To use this functionality, users will need to create a JSON data source to retrieve the tags in Microsoft Purview. A derived view will also be needed in order to transform the data obtained from Purview and use the “Import Tags from External Catalog” functionality.
NOTE: This feature is only available with Denodo Enterprise Plus. For more information see Denodo Platform - Subscription Bundles.
Configuration
Create a JSON data source
1. In order to create a new JSON data source to obtain Microsoft Purview tags, right-click on the Server Explorer and click on New > Data source > JSON.
2. Configure the HTTP path with the following values:
Default HTTP method: POST
Base URL: https://<Domain>.purview.azure.com/datamap/api/search/query?api-version=2023-09-01
Post body:
\{ "keywords": null, "limit": 1000, "filter": \{ "or": [ \{ "assetType": "Denodo Platform Database" \} ] \} \} |
3. Configure authentication with the following values
Authentication: OAuth 2.0
Authentication grant: Client Credentials grant
Client identifier: client ID of the Microsoft Purview instance.
Client secret: client Secret of the Microsoft Purview instance.
Select Send client credentials using the HTTP Basic authentication scheme.
4. Use the Launch the OAUTH 2.0 credentials Wizard to help you obtain these credentials functionality to obtain the access token with the following values:
Token endpoint URL
https://login.microsoftonline.com/<TENANT_ID>/oauth2/token
Request sign in method: "Authorization" request header
Add a resource parameter in Extra parameters of the refresh token requests:
5. Save the data source.
Create a base view
Create the base view setting Tuple root to /JSONFile/value
Create a selection/projection view
Execute the following VQL command in a VQL Shell, properly setting the name of the base view created in the previous step in the FROM clause (in this example tags_from_purview):
CREATE OR REPLACE VIEW p_tags_from_purview AS SELECT tags_from_purview.qualifiedname AS qualifiedname, tags_from_purview.entitytype AS entitytype, tags_from_purview.assettype AS assettype, tags_from_purview.name AS name, tags_from_purview.tag AS tag, NULL AS tag_description, case WHEN (tags_from_purview.entitytype = 'denodo_db') THEN tags_from_purview.name WHEN (tags_from_purview.entitytype = 'denodo_datasource') THEN substring(qualifiedname, (instr(qualifiedname, '/database/')+10), instr(qualifiedname, '/datasource/')) WHEN (tags_from_purview.entitytype = 'denodo_view') THEN substring(qualifiedname, (instr(qualifiedname, '/database/')+10), instr(qualifiedname, '/view/')) WHEN (tags_from_purview.entitytype = 'denodo_column') THEN substring(qualifiedname, (instr(qualifiedname, '/database/')+10), instr(qualifiedname, '/column/')) END AS database_name, case WHEN (tags_from_purview.entitytype = 'denodo_view') THEN tags_from_purview.name WHEN (tags_from_purview.entitytype = 'denodo_view') THEN tags_from_purview.name WHEN (tags_from_purview.entitytype = 'denodo_column') THEN substring(qualifiedname, (instr(qualifiedname, '/column/')+8), ((instr(qualifiedname, '/column/')+8)+instr(substring(qualifiedname, (instr(qualifiedname, '/column/')+8)), '.'))) END AS view_name, case WHEN (tags_from_purview.entitytype = 'denodo_column') THEN tags_from_purview.name END AS column_name, case WHEN (tags_from_purview.entitytype = 'denodo_view') THEN 'view' WHEN (tags_from_purview.entitytype = 'denodo_column') THEN 'column' WHEN (tags_from_purview.entitytype = 'denodo_db') THEN 'database' END AS element_type FROM tags_from_purview; |
Create a flatten view
Execute the following VQL command in a VQL Shell in order to create the derived view that flattens the data and adopts the final structure to be used to import tags from Microsoft Purview:
CREATE OR REPLACE VIEW flatten_tags_from_purview AS SELECT qualifiedname AS qualifiedname, entitytype AS entitytype, assettype AS assettype, name AS name, tag_description AS tag_description, database_name AS database_name, view_name AS view_name, column_name AS column_name, element_type AS element_type, field_0 AS tag FROM FLATTEN p_tags_from_purview AS v ( v.tag); |
We can see the results of an execution of the derived view in the following image:
Import Tags
Configuration
In order to use the Import Tags from External Catalog functionality in Denodo Design Studio, users need to go to Administration > Semantic and governance > Import Tags from External Catalog. Then configure the database name and the view name that corresponds to the created derived view, test the connection and save.
Run the import
In the EXTERNAL CATALOG tab, Generic should be selected as the Catalog value.
After executing, tags should be correctly retrieved. Clicking on “Accept changes” will apply the new tags on Denodo view columns.
References
The information provided in the Denodo Knowledge Base is intended to assist our users in advanced uses of Denodo. Please note that the results from the application of processes and configurations detailed in these documents may vary depending on your specific environment. Use them at your own discretion.
For an official guide of supported features, please refer to the User Manuals. For questions on critical systems or complex environments we recommend you to contact your Denodo Customer Success Manager.
Questions
