Extensions (Plugins)¶
Denodo Scheduler allows users to create their own Aracne filters, exporters, handlers or crawlers.
The following sections describe how to implement a new Aracne filter, exporter, handler or crawler.
Denodo4E, an Eclipse plug-in which provides tools for creating,
debugging and deploying Denodo extensions, including filters, exporters,
handlers and crawlers, is included in the Denodo Platform. Please read
the README in <DENODO_HOME>/tools/denodo4e
for more information.
Once the extension is implemented, it needs to be packed in a JAR file
along with an XML file that specifies its configuration metadata (the
extension type, its name, and its input parameters), and, optionally,
another XML file that specifies the default values of the input
parameters specified in the metadata. The metadata is used by the
administration tool to configure the extension through an auto-generated
configuration wizard (metadata specifies the value of the parameters the
extension receives in its init
method), and the default values can
be used to initially set the values of the parameters in the
configuration wizard.
The name and location of the metadata file must be the same as that of
the class that implements the extension (i.e. in the same package as the
implementation class). It must follow the DTD custom-metaconfig.dtd
which can be found in the JAR file
<DENODO_HOME>/lib/contrib/denodo-configuration.jar
. In this XML file
the element
tag allows the type (type
) and subtype (subType
)
to be specified for the extension. Each configuration parameter is
specified using the param
element, whose attributes are the
parameter name (name
), its obligatoriness (mandatory
), if it can
have multiple values (multivalued
), its Java type (javaType
), if
it contains sensitive data (hidden
), the label to be displayed in
the configuration wizard (displayName
), and a help text to be shown
as a tooltip in the configuration wizard (helpText
). Hidden
parameters’ values will be shown as strings of dots in the configuration
wizard, and if no displayName
is specified, then the parameter
name
will be used as the parameter label. If a parameter is
compound, the components of the parameter are listed within the
components
element.
The following metadata XML file for the POP3/IMAP mail server crawler included in Scheduler is shown as an example. As can be seen, it is a crawler element called “Mail”, which has a machine name and a list of user accounts with logins and passwords as input parameters, from which to retrieve e-mails.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE metaconfig SYSTEM "custom-metaconfig.dtd">
<metaconfig>
<element type="crawler" subType="Mail" />
<param javaType="java.lang.String" mandatory="true" multivalued="false"
name="host"/>
<param mandatory="true" multivalued="true" name="accounts">
<components>
<param javaType="java.lang.String" mandatory="true"
multivalued="false" name="user" />
<param javaType="java.lang.String" mandatory="true"
multivalued="false" name="password" hidden="true" />
</components>
</param>
</metaconfig>
Note
The export/import process (see section Connection Details for details) attempts to convert from id to name and then from name to id the values of all parameters called “datasource” (because the id of a data source will be different for each installation). So, if you need to use a data source’s id as the value for an extension’s parameter, you should call it “datasource”. If you need to use a data source’s name as the value for an extension’s parameter, then you should use a parameter name different to “datasource” (otherwise, after exporting and then importing a job using this extension, the value of the “datasource” parameter would be replaced by an id).
The optional default values file must be placed in the same directory
than the metadata file and must have the same name prefixed by
“default_”. It must follow the DTD default-custom-config.dtd
which
can also be found in the JAR file
<DENODO_HOME>/lib/contrib/denodo-configuration.jar
. In this XML file
the simple-param
and compound-param
tags are used to set the
default value of simple and compound parameters, respectively. Both
accept an attribute (name
) to specify the name of the parameter. The
value is specified using a nested value
tag. In the case of the
simple parameters the value
tag contains the parameter value itself,
and in the case of the compound parameters it contains nested
simple-param
or compound-param
tags to specify the values of the
parameters composing it. Only one default value can be specified for
both simple and compound parameters, even though they are multivalued.
If the parameter is multivalued, the attribute instances
can be used
in the parameter tag to specify the number of parameters of that type to
be initially created (its default value is 1, and it can be set to 0 if
no parameters of that type should be created). All the instances of the
multivalued parameters, including those added later by the user, will be
initially created with the default values specified for the parameters
composing it. If the parameter is not multivalued but it is optional,
the attribute instances
can be used to specify if the parameter
should be initially created (instances
should be set to 1) or not
(instances
should be set to 0).
The following default values XML file could be used with the Mail
crawler shown previously. As can be seen, it specifies “mailServer” as
the default value for the host
parameter, and “loginName” as the
default value for the user
account parameter. It also specifies that
one account parameter should be initially created (note that, in this
case, the instances
attribute could be omitted because it is set to
1, which is its default value).
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE metaconfig SYSTEM "default-custom-config.dtd">
<default-parameters>
<simple-param name="host">
<value>mailServer</value>
</simple-param>
<compound-param name="accounts" instances="1">
<value>
<simple-param name="user">
<value>loginName</value>
</simple-param>
</value>
</compound-param>
</default-parameters>
Finally, in the META-INF/MANIFEST.MF
JAR file the following metadata
needs to be specified:
Name: CustomElement
PluginType: filter | exporter | handler | crawler
PluginName: Name of the extension (it will be used as name for the extension in the administration tool).
PluginClass: Name of the extension implementation class.
The following MANIFEST.MF fragment for the Salesforce crawler included in the Scheduler samples is shown as an example.
Manifest-Version: 1.0
...
Name: CustomElement
PluginType: crawler
PluginName: salesforce
PluginClass: com.denodo.crawler.salesforce.SalesforceCrawler
Section Plugins describes how to install a new extension in Scheduler.