Overview
Collibra offers the Collibra Data Intelligence Cloud which is the system of record for data. Collibra delivers an end-to-end, integrated Data Intelligence platform that’s purpose-built to automate data workflows and deliver trusted data insights to users.
The Denodo Governance Bridge is a DenodoConnect Enterprise component that allows the synchronization of Denodo metadata and lineage into data governance tools in order to support the inclusion of Denodo virtual databases in enterprise-wide data governance initiatives. Among these data governance tools it has support for Collibra Data Intelligence Platform. You can download this DenodoConnect Enterprise component from the DenodoConnects section of the support site.
In addition, registering Denodo as a data source in Collibra is available by adding a JDBC connection (using the Denodo JDBC driver). This connection also allows you to collect metadata from Denodo and add it into Collibra via Edge.
The two synchronization methods mentioned, via the Denodo Governance Bridge and via Edge, are complementary. Denodo Governance Bridge can include more complete and accurate Denodo metadata. Whereas the Edge connection allows you to include the Edge capabilities such as sampling.
Denodo Governance Bridge for Collibra
The Denodo Governance Bridge for Collibra is a Spring Boot application that retrieves, maps and ingests metadata from Denodo Platform to Collibra. It also allows the creation of technical lineage metadata that can be loaded into Collibra via the Lineage Harvester or via Edge.
Prerequisites to run the Denodo Governance Bridge for Collibra
- Java 8 or later
- The environment variables JAVA_HOME and PATH correctly configured
It is also important to highlight that if there are Unicode characters, for example space characters, in the names of the elements to be synchronized or for which you are going to create technical lineage, you must have Unicode support enabled in the Denodo Virtual DataPort server to achieve a correct synchronization in Collibra.
Configuring Collibra
As the Denodo Governance Bridge for Collibra user manual mentions, in order to use the application, the Collibra instance must be customized beforehand.
You need to create or review if it already exists:
- A domain (it must be part of a community) with the type Physical Data Dictionary. This is where the Denodo integration assets are upserted.
The name does not necessarily have to be “Denodo Data Dictionary”.
Example:
- A scope. A scope in Collibra is a selection of communities and domains that you can refer to the assignments created during the set up configuration. You have to create a scope and it must be associated with the Community/Domain that you are going to use in your synchronization. In order to make an example, we will use the scope "Denodo Scope".
- Custom Asset types to map Denodo elements:
Asset Type |
Description |
Parent Asset Type |
Denodo Folder |
Represents a Denodo folder |
BI Folder |
Denodo Data Source |
Represents a Denodo data source |
BI Data Source |
- Attribute Types that are going to be used to model derived views, associations and complete views information:
Attribute Type |
Description |
Kind |
Dependency Type |
The dependency (e.g. Union) between two views |
Text |
Multiplicity |
The multiplicity of the column-to-column associations |
Text |
Left Role Description |
The left role description of the column-to-column associations |
Text |
Right Role Description |
The right role description of the column-to-column associations |
Text |
Cache Status |
The cache status of the respective view |
Text |
View Status |
The status of the respective view (available since Denodo 8.0) |
Text |
- A Relation Type to describe how Schema and Denodo Folder relate to each other:
Head |
Role |
Co-Role |
Tail |
Schema |
contains |
is part of |
Denodo Folder |
- Complex Relation Types:
Head |
Leg 1 |
Leg 2 |
Attributes |
Description |
Denodo View Dependency |
Name |
Name |
Dependency Type |
Complex relation type used to provide the Denodo views lineage |
source |
target |
|||
Asset Type |
Asset type |
|||
Table (1:N) |
Table (1:1) |
|||
Denodo Column Association |
Name |
Name |
Multiplicity Left Role Description Right Role Description |
Complex relation type used to represent the column-to- column associations |
source |
target |
|||
Asset Type |
Asset Type |
|||
Column (1:1) |
Column (1:1) |
- Assignments. An assignment is a collection of components that is assigned to an asset type. You need to apply these assignments to a scope (scoped assignment). A scoped assignment only applies to assets that are located in a domain or community that belongs to the scope, therefore, these assignments should be applied in the created scope.
You should assign the custom scope, in our example Denodo Scope, to the asset types Schema, Denodo Folder, Table, Database View and Column. After that, you should add the following characteristics in the custom scope in the corresponding types:
Asset Type |
Add |
Schema |
Relations: contains Denodo Folder |
Denodo Folder |
Relations: is part of Schema |
Table |
Attributes: URL, Cache Status, View Status Complex Relations: Denodo View Dependency |
Database View |
Attributes: URL, Cache Status, View Status Complex Relations: Denodo View Dependency |
Column |
Complex Relations: Denodo Column Association |
It is important to note that even once you have correctly created the metamodel in Collibra, you should verify that the corresponding asset layouts include the attributes or relations you want to see.
Configuring the Denodo Governance Bridge
The Denodo Governance Bridge distribution consists of three folders containing the distribution for each module. The Denodo Governance Bridge for Collibra is available in the denodo-collibra-governance-bridge-<version> folder.
In order to configure the Denodo Governance Bridge for Collibra, there is an application.properties file under the denodo-collibra-governace-bridge-<version>/conf folder. It contains properties that are used to communicate with the Denodo and Collibra Platform instances and other internal application properties. Check their meaning in the Configuration of the Denodo Governance Bridge for Collibra section of the Denodo Governance Bridge for Collibra - User Manual and set the appropriate values.
It is important to highlight the importance of establishing the Collibra custom constants used to define the custom types of assets, attributes, relations and complex relations required by the integration (see the Configuring Collibra section for more detailed information regarding these custom elements). You should update the IDs with the values for these custom types. They are available in the Settings > Operating Model section of the Collibra instance. Note that you must add the Resource ID column that is hidden by default.
General view
Physical Data Dictionary and Assets
In the Dashboard you can access your domain.
The assets created after running the integration are available clicking on the domain name, Denodo DD in this example:
You can also see in the Catalog section the table type assets created. Clicking on the Data Dictionary section and filtering by community name and domain name:
Table
When you open an asset with Table type, you can see its information, including the URL of the view in the Denodo RESTful Web service and in the Denodo Data Catalog.
Column
When you open an asset with Column type you can see its metadata information and other details:
Diagram
In the Diagram section of a Table or a Column, you can see lineage information. Note that the technical lineage has its particular section and to be able to display this data you should generate the technical lineage with the Denodo Governance Bridge for Collibra and load into Collibra via the Lineage Harvester or via Edge. For more detailed information, please refer to the Denodo technical lineage section of the Denodo Governance Bridge for Collibra - User Manual.
Diagram example:
Note that the exploration to see the lineage data is element by element, but the Preview option, that opens the right side panel, is very helpful to get information about the assets and then, based on this, explore the ones you want to get more data on. For example, if you have this diagram:
As you can see in the image above, clicking on the Denodo View Relation, you can know that the sources are books and booksales, therefore, you can explore this asset if you want to see the complete Denodo View Relation.
Technical lineage
Although the technical lineage is not created as a result of a synchronization, Denodo Governance Bridge for Collibra offers the possibility to generate technical lineage that can then be loaded into Collibra. Refer to the Denodo technical lineage section of the Denodo Governance Bridge for Collibra - User Manual for further details.
Technical lineage example:
References
Denodo Support Site: DenodoConnects
DenodoConnect Components: Denodo Governance Bridge for Collibra - User Manual
The information provided in the Denodo Knowledge Base is intended to assist our users in advanced uses of Denodo. Please note that the results from the application of processes and configurations detailed in these documents may vary depending on your specific environment. Use them at your own discretion.
For an official guide of supported features, please refer to the User Manuals. For questions on critical systems or complex environments we recommend you to contact your Denodo Customer Success Manager.