The Denodo Data Marketplace (also known as Denodo Data Catalog) is a web based self service tool included in Denodo Platform that lets both technical and business users query, search and browse information and metadata stored in a Virtual DataPort server. With this tool, users can generate new knowledge and pave the way to make better decisions.

Before starting this tutorial, follow the subsequent instructions:

1. Launch the resources needed (check how in the Installation & Bootstrapping tutorial).
2. Log in Denodo Design Studio (user/password: admin/admin)
3. Import this VQL to Denodo by clicking into File > Import. Drag ‘n' drop the file, select Use custom password for sensitive data decryption, and enter denodo in the Password field.

Note for Agora users: If you have followed the tutorial Agora Data Integration and using Denodo Platform in Agora then you need to import this VQL instead.

4. Click on File > Refresh

If all steps have been executed correctly, you should observe the following in the Design Studio's elements tree:

Great! Now it's time to start the tutorial!

1. For starting this web tool in a local installation, you have to open the Denodo Platform Control Center, and start the Data Marketplace. Once it changes the status to "Running", click the Data Marketplace link to open the Web tool (by default: https://127.0.0.1:9090/denodo-data-catalog).
   
2. Now login to the Data Marketplace using the standard login details (admin/admin):

The first time you login to the Data Marketplace, you will notice the Metadata synchronization popup window. This needs to be run when you open the Data Marketplace for the first time, in order to ensure that the Data Marketplace reflects the latest state of the Denodo Virtual DataPort server you are connected to.

Run the synchronization as follows:

1. Click the Synchronize the metadata now link.
2. Click Continue on each Synchronization step.
3. Once the synchronization is complete, you should now see the views and web services available in the tree view when navigating to the Browse > Database / Folders page and expanding the tutorial database

Great! Now let's continue with the next setup.

In this section we will explore the features of the Data Marketplace Content Search. With this feature you can use Denodo Scheduler to index the content of your views using either ElasticSearch or the Denodo Scheduler Index Server. You can then allow your users to perform Google-like searches on them, and to customize how they see the search results.

We are going to index the fields of the bv_crm_client view to allow more rapid discovery of client details.

Our first step is to configure an Index. Let's see how to do that using Denodo Scheduler

Creating an Index in the Denodo Scheduler

Start the Scheduler Server and the Scheduler Index Server (you can follow the latest section of the Caching tutorial to know how to start Denodo Scheduler). Once these are all running, open the Scheduler Administration Tool for creating an Index.

Create the Index following these steps:

1. In the login screen of the Scheduler Administration Tool, provide the login details admin / admin and URI of the Scheduler Server.

2. In the Denodo Scheduler we need to create a new job to create and maintain the Index. Click Add Job > Data Indexer

3. Give the Job a suitable name, in this case index_clients.

4. Choose the following settings under the Configuration tab, while leaving the rest to default/blank:

• Data Source: VDP
• View: tutorial.bv_crm_client
• Indexing process name: tutorial.bv_crm_client

5. Under the Exporters section, click Add Exporter > Scheduler-Index and choose the following settings while leaving the rest to default/blank:

• Data Source: Scheduler-Index
• Index name: ix_client

6. Save the Scheduler Job.
7. Once the job is saved, you can execute the job by clicking three dots and then the Start option.

8. The job will execute and once successfully complete, the Result status will change to COMPLETE, indicating that the Index has been populated.

Configuring the Index in the Data Marketplace

We now need to configure the newly created Index in the Data Marketplace, in order to ensure that the Data Marketplace includes the Index as part of the searchable content.

1. Open the Data Marketplace and navigate to Administration > Set-up and management.
2. In the Administration window, click on Search Engines option

3. Click on + Add server option under Index Servers tab.

4. Add the details as follows to the Add New Index Server screen.

• Name: TutorialIndex
• Description: Tutorial Index
• Type: Scheduler Index
• Host: sched (or localhost if you are using a local installation of Denodo)
• Port: 9000
• Login: admin
• Password: admin

5. Click Ok.

6. Go to the Content Search tab, Click the Pencil Icon.

7. In the Search index path screen, add the following details:

• Index Type: Scheduler Index
• Index Server: TutorialIndex
• Index Name: ix_client

8. Click Ok.

9. The Index will display a green checkmark under the Configured column to indicate that the Index was added successfully.

Done! In the Using Denodo Data Marketplace tutorial you will see the new Index in action. For now, let's continue setting up other configurations of the Data Marketplace.

In this section, we will explore the feature of creating Property Groups. With this feature, we can use the Denodo Data Marketplace to define custom properties; add them to databases, views and web services; and give them a value on each element in order to improve the available information on them.

In our example, we are going to create a new property group containing custom properties to identify the data owner of a view, and, as an example, assign it to the bv_crm_client view.

Creating a Property Group

Our first step is to create a property group to specify information for Data Ownership

1. Open the Data Marketplace and navigate to the Administration > Set-up and management.
2. In the Administration window, click on the Property Groups option.
3. In the Property group management page, click on the + Add Property Group button

4. In the pop-up page that appears, specify the following details:
   
   Name: Data Ownership
   Description: This property group is to specify the data owner information of the data asset
   Show in: Summary tab

5. Click Ok

Adding a Custom Property to the Property Group

Now that we have created the Property Group, it's time to add some Custom Properties to it to specify what kind of information we want to define to enhance the information of the views.

1. In the Property group management page, click on the three dots button located at the rightmost section of the Data Ownership property group, and select Manage Properties option.

2. In the next page, click the + Add Property button.

3. In the pop-up page that appears, specify the following details:
   
   Name: Data Owner
   Description: The name of the data owner
   Property type: Text

4. Click Ok
5. Once we are back to the Properties Management page, we can add more custom properties to the property group, but for now, we will just add one property for this tutorial

Assigning the Property Group to a View

We can now assign our newly created Property Group with the relevant Custom Property to our bv_crm_client view to enhance it with the Data Ownership information.

1. Navigate back to the Administration > Set-up and management > Property Group page.
2. Click on the three dots button located at the rightmost section of the Data Ownership property group, and select Assign Property Group option.

3. In the Assign property group page, select the bv_crm_client view, and drag it towards the Views section.

4. In the pop-up page that appears, we can now specify the value of the Data Owner Name property, which we configured earlier.
   
   Specify Jane Smith in the text box and click Ok.

5. Now, navigate to the Browse > Databases /Folders page, and click on the bv_crm_client view to open it.

6. In the Summary tab of the view, we can now see the Data Ownership information of this view

Great! Let's continue setting up other configurations of the Data Marketplace.

In this section, we will explore how to configure one of the best features of Data Marketplace: the LLM Configuration. This feature lets you explain your data needs in natural language via the Natural language query input. Providing more details not only refines the search results but also elevates the quality of the displayed information. This extends also to the metadata of the view.

In this tutorial we will set up the required configuration to start using the LLM Configuration feature.

Get an OpenAI API Secret Key

The Data Marketplace provides 3 options on the API to use for the LLM services so you need the credentials of the service in order to configure it in the Data Marketplace. To proceed with this part of the tutorial, you would need access to available LLM such as OPEN AI, Amazon Bedrock, Azure Open AI service and Custom.

Configure the LLM Configuration feature in the Data Marketplace

1. Login to the Data Marketplace and navigate to the Administration > Set-up and management, click on Server, and go to the LLM Configuration tab.

2. Specify the LLM information, in our example, we will use OpenAI:
   
   Provider: OpenAI
   API key: the secret key obtained from LLM provider
   Organization ID: the service account ID from LLM provider
   Model: select one of the supported models or enter the name of a different one clicking on the pencil button
   
3. Click on Save. Finally, you can click on the Test Configuration button to check that it has been configured correctly.

4. Now, let's return to the Administration > Set-up and management, click on Personalization, and open the Denodo Assistant tab.

5. Select Enable query generation option and leave other values as default. Save the settings.

Done! You will see the LLM in action in the "Assisted Query (Using AI)" section of the Using Denodo Data Marketplace tutorial.

To provide workflow capabilities in the Data Marketplace, it has introduced some types of requests that will allow users to initiate cases on elements that will be attended by an administrator from a data stewardship perspective.

Most frequent types of requests: access requests, report data quality issues, request changes and ask general questions.

Setup a user with restricted access to the tutorial views

By default, users don't have privileges to create these requests, so let's configure a role to have access to this functionality:

1. Ensure you are logged in using an admin user and navigate to the Administration > Set-up and management > Server > Permissions > Request page.
2. Search for the restricted role (already created), and assign permission to create access, change, data quality and question requests and save the changes.

In this tutorial, we have configured various Data Marketplace features which we are going to explore in the next. Head on to the next tutorial to see how these features work.

Thanks!