USER MANUALS

Custom Sources

Custom data sources, also called Custom wrappers, are ad-hoc wrappers developed in Java to access any source that Virtual DataPort does not have support for.

These wrappers are developed to expose the metadata and query capabilities of any kind of application or proprietary data repository, to the Denodo Platform.

The section Developing Custom Wrappers of the Developer Guide explains how to develop them.

The Denodo Platform includes the source code of a sample custom source that lists the files of a directory of the host in which Virtual DataPort runs. The example is located at <DENODO_HOME>/samples/vdp/customWrappers.

To compile the classes of the example, execute the script scripts/compile_listdir_customwrapper, which generates the jar file target/jars/denodo-demo-listdir-customwrapper.jar.

Note

Before creating the sample custom data source in Virtual DataPort, you have to import the jar that contains its classes. To do this, click Extension management on the menu File. Then, click on Import, select the file <DENODO_HOME>/samples/vdp/customWrappers/target/jars/denodo-demo-listdir-customwrapper.jar and click Close. When doing this, the contents of the jar are sent to the Server.

To create a Custom data source, first you need to upload to Virtual DataPort the jar file(s) that contains the extension. To do this, follow these steps:

  1. Click the menu File > Extension management.

  2. In the tab Extensions, click Import and select the jar file(s) of the extension and click Ok. This will upload the jar files

Once you upload a jar file, you can create several custom data sources over the same jar; you do not need to upload it again.

After uploading the jar file(s), click the menu File > New > Data source > Custom.

The Tool will display the dialog to create the data source.

Creating a custom data source

Creating a custom data source

The following data are requested in this dialog:

  • Name. Name of the new data source.

  • Select Select jars and in the box below, select the jar file(s) that contains the code of the custom data source you want to create.

    Note

    To select two or more Jars, hold Ctrl and click on the required Jars.

  • In Class name, select the Java class that implements the Custom data source.

    In our ListDir example, select com.denodo.vdp.demo.wrapper.custom.listdir.ListDirWrapper.

  • Classpath. Leave this empty. This field is provided only for backward compatibility.

  • In the Metadata tab, you can:

    • Set the folder where the data source will be stored.

    • Provide a description.

    • Set the infrastructure information where the data source is deployed. You can set the name of the infrastructure provider (e.g. on-premise, Amazon Web Services, Microsoft Azure, Google Cloud Platform, etc.) and the region name. You can type a custom name if the provider name or region is not on the list.

When editing the data source, you can also change its owner by clicking the button image1.

Then, click Save to create the data source.

Click Create base view to create a base view over the new data source.

Custom Sources, also called Custom wrappers, may accept input parameters. If they do, you have to provide their value when creating the base view. The Administration Tool will show a suitable component to edit the parameter value depending on its type: a text box, a drop-down list, a check box, etc. The section Developing Custom Wrappers of the Virtual DataPort Developer Guide lists the available types.

Important

If the value of a parameter contains one of the following characters, you have to escape them with the character \ because they are reserved characters in Virtual DataPort:

  • \

  • ^

  • {

  • }

  • @

If you load the value of a parameter with Load file of the “Value editor”, the Tool automatically escapes these characters. To open this editor, click on image1 beside the parameter box. This editor is only available for parameters with the type LongString.

We recommend loading the value of a parameter from a file if the value is very long or may contain reserved characters.

If the custom wrapper allows pass-through session credentials it is required to define one input parameter of type “login” and another one of type “password”. If these parameters are present in the custom wrapper, the Administration Tool will show a “Pass-through session credentials” check box to enable this behavior. When executing a view created with this option enabled, the server will use the login and password used by the user to authenticate in the server to fill the login and password parameters of the custom wrapper.

Warning

Users should be careful when enabling the cache for views that involve data sources with pass-through credentials enabled. The section Considerations When Configuring Data Sources with Pass-Through Credentials explains the issues that may arise.

Editing input parameter values of a custom wrapper

Editing input parameter values of a custom wrapper

The ListDir wrapper has the following input parameters:

  • DIR_ROUTE = <your route of the directory you want to list>

  • RECURSIVE = <optionally select this check box>

The values of the input parameters of the wrapper may affect the schema of the base views or the behavior of the wrapper.

This feature is very useful to develop Custom sources that can retrieve data from a source and the schema of the data may have different schemas.

Creating a base view from a Custom data source

Creating a base view from a Custom data source

Now, click Ok to display the schema that the new base view will have (see New base view from a Custom data source).

New base view from a Custom data source

New base view from a Custom data source

At this point, you can change the name of the view and the type of its fields. In the Metadata tab, set the folder where the base view will be stored and provide a description.

Click Save to create the base view.

After creating the base view, you can click on Edit to modify the view. In this dialog, you can rename the fields of the view, change their type and store the view in another folder.

When editing the base view, you can also change its owner by clicking the button image1.

Using Interpolation Variables as Custom Sources’ Input Parameters

When creating a base view over a Custom source that has input parameters, you can provide:

  • The values of these parameters.

  • Or, interpolation variables.

If you provide interpolation variables, the Tool will ask sample values of these variables in order to query the source and know the schema of the base view. That is because a custom wrapper may return a different schema depending on the value of these parameters.

For example, in the ListDir wrapper, instead of putting the values in the fields DIR_ROUTE and RECURSIVE, you can put interpolation variables. E.g., fill in the field DIR_ROUTE with @DIR_ROUTE and RECURSIVE, with @RECURSIVE.

After creating the base view, you will see that it has two extra fields: DIR_ROUTE and RECURSIVE. These fields will be required to query the view, so they have to be provided in the WHERE clause of the query.

When filling in the values of input parameters, you can combine several interpolation variables in the same parameter. E.g. DIR_ROUTE = @DIR_ROUTE@DOMAIN.

You can also mix a variable with static text. E.g. DIR_ROUTE = @DIR_ROUTE\@dir1. In this case, the character @ is escaped, so the text @dir1 is interpreted as a literal.

Add feedback