Overview

Collibra offers the Collibra Data Intelligence Cloud which is the system of record for data. Collibra delivers an end-to-end, integrated Data Intelligence platform that’s purpose-built to automate data workflows and deliver trusted data insights to users.

The Denodo Governance Bridge is a DenodoConnect Enterprise component that allows the synchronization of Denodo metadata and lineage into data governance tools in order to support the inclusion of Denodo virtual databases in enterprise-wide data governance initiatives. Among these data governance tools it has support for Collibra Data Intelligence Platform. You can download this DenodoConnect Enterprise component from the DenodoConnects section of the support site.

In addition, registering Denodo as a data source in Collibra is available by adding a JDBC connection (using the Denodo JDBC driver). This connection also allows you to collect metadata from Denodo and add it into Collibra via Edge.

The two synchronization methods mentioned, via the Denodo Governance Bridge and via Edge, are complementary. Denodo Governance Bridge can include more complete and accurate Denodo metadata. Whereas the Edge connection allows you to include the Edge capabilities such as sampling.

Denodo Governance Bridge for Collibra

The Denodo Governance Bridge for Collibra is a Spring Boot application that retrieves, maps and ingests metadata from Denodo Platform to Collibra. It also allows the creation of technical lineage metadata that can be loaded into Collibra via the Lineage Harvester or via Edge.

Prerequisites to run the Denodo Governance Bridge for Collibra

• Java 8 or later
• The environment variables JAVA_HOME and PATH correctly configured

It is also important to highlight that if there are Unicode characters, for example space characters, in the names of the elements to be synchronized or for which you are going to create technical lineage, you must have Unicode support enabled in the Denodo Virtual DataPort server to achieve a correct synchronization in Collibra.

Configuring Collibra

As the Denodo Governance Bridge for Collibra user manual mentions, in order to use the application, the Collibra instance must be customized beforehand.

You need to create or review if it already exists:

• A domain (it must be part of a community) with the type Physical Data Dictionary. This is where the Denodo integration assets are upserted.

The  name does not necessarily have to be “Denodo Data Dictionary”.

Example:

• A scope. A scope in Collibra is a selection of communities and domains that you can refer to the assignments created during the set up configuration. You have to create a scope and it must be associated with the Community/Domain that you are going to use in your synchronization. In order to make an example, we will use the scope "Denodo Scope".

• Custom Asset types to map Denodo elements:

• Attribute Types that are going to be used to model derived views, associations and complete views information:

• A Relation Type to describe how Schema and Denodo Folder relate to each other:

• Complex Relation Types:

• Assignments. An assignment is a collection of components that is assigned to an asset type. You need to apply these assignments to a scope (scoped assignment). A scoped assignment only applies to assets that are located in a domain or community that belongs to the scope, therefore, these assignments should be applied in the created scope.

You should assign the custom scope, in our example Denodo Scope, to the asset types Schema, Denodo Folder, Table, Database View and Column. After that, you should add the following characteristics in the custom scope in the corresponding types:

It is important to note that even once you have correctly created the metamodel in Collibra, you should verify that the corresponding asset layouts include the attributes or relations you want to see.

Configuring the Denodo Governance Bridge

The Denodo Governance Bridge distribution consists of three folders containing the distribution for each module. The Denodo Governance Bridge for Collibra is available in the denodo-collibra-governance-bridge-<version>  folder.

In order to configure the Denodo Governance Bridge for Collibra, there is an application.properties file under the denodo-collibra-governace-bridge-<version>/conf folder. It contains properties that are used to communicate with the Denodo and Collibra Platform instances and other internal application properties. Check their meaning in the Configuration of the Denodo Governance Bridge for Collibra section of the Denodo Governance Bridge for Collibra - User Manual  and set the appropriate values.

It is important to highlight the importance of establishing the Collibra custom constants used to define the custom types of assets, attributes, relations and complex relations required by the integration (see the Configuring Collibra section for more detailed information regarding these custom elements). You should update the IDs with the values for these custom types. They are available in the Settings > Operating Model section of the Collibra instance. Note that you must add the Resource ID column that is hidden by default.

General view

Physical Data Dictionary and Assets

In the Dashboard you can access your domain.

The assets created after running the integration are available clicking on the domain name, Denodo DD in this example:

You can also see in the Catalog section the table type assets created. Clicking on the Data Dictionary section and filtering by community name and domain name:

Table

When you open an asset with Table type, you can see its information, including the URL of the view in the Denodo RESTful Web service and in the Denodo Data Catalog.

Column

When you open an asset with Column type you can see its metadata information and other details:

Diagram

In the Diagram section of a Table or a Column, you can see lineage information. Note that the technical lineage has its particular section and to be able to display this data you should generate the technical lineage with the Denodo Governance Bridge for Collibra and load into Collibra via the Lineage Harvester or via Edge. For more detailed information, please refer to the Denodo technical lineage section of the Denodo Governance Bridge for Collibra - User Manual.

Diagram example:

Note that the exploration to see the lineage data is element by element, but the Preview option, that opens the right side panel, is very helpful to get information about the assets and then, based on this, explore the ones you want to get more data on. For example, if you have this diagram:

As you can see in the image above, clicking on the Denodo View Relation, you can know that the sources are books and booksales, therefore, you can explore this asset if you want to see the complete Denodo View Relation.

Technical lineage

Although the technical lineage is not created as a result of a synchronization, Denodo Governance Bridge for Collibra offers the possibility to generate technical lineage that can then be loaded into Collibra. Refer to the Denodo technical lineage section of the Denodo Governance Bridge for Collibra - User Manual for further details.

Technical lineage example:

References

Denodo Support Site: DenodoConnects

DenodoConnect Components: Denodo Governance Bridge for Collibra - User Manual