Scheduler Client API¶
The Denodo Scheduler Client API allows jobs, filter sequences and data sources to be programmatically configured and managed. It also allows to see the list of jobs, start/stop jobs, manage permissions, etc.
To do this, the platform has a facade
com.denodo.scheduler.client.SchedulerManager which allows the
Getting the job list with information on its state (
See a job’s execution reports (
Create, obtain, update, and delete the configuration of a job, a filter sequence, and a Data Source (
Enable or disable a job:
Execute a job only once, without taking into account the start and end time for its configuration:
Execute a job with its dependencies (see section Jobs, Start With Dependencies):
Execute a job with state. If the execution of a job gives any errors, executing it again with state will only execute those queries that had errors (see section Jobs, Start with state):
Stop a job from running:
Obtain the types of jobs, filters, and data sources allowed by the system: ITP, VDP, etc.:
Pause the job scheduler. When the scheduler is paused, the jobs will wait until it is resumed before being executed again. This order will only run successfully if no job is being executed:
Resume executing the scheduler when it is paused:
All configuration elements (jobs, data sources, and filter sequences)
are based on the object
which represents a collection of parameters (simple or compound) that
constitute the configuration of the element. The list of parameters and
their specification (data type, mandatoriness, etc.) are given by the
The filter sequences comprise an identifier of the sequence and a list
of filters that make it up, each one also being represented by an object
The jobs have an identifier, a description, a type (ARN, ARN-Index, ITP, JDBC, or VDP), some associated handlers, and a time-based scheduler. VDP- and JDBC-type jobs have an extraction section which specifies where the data is going to be extracted from (both differ as to the types of data sources that can be used). ITP-type jobs also have an extraction section, where the wrapper to be executed is indicated (with its parameters, if applicable) for obtaining data. In the ARN jobs extraction section the type of crawler (WebBot, IECrawler, MailCrawler, SalesforceCrawler, GlobalSearch or CustomCrawler) you want to use to collect data from a source is indicated. All these jobs have an export section, where a filter sequence (or none [optional]) is associated with a list of exporters (ARN-Index, JDBC, SQL, CSV); it can define several “filter sequence/exporter” associations. The jobs also have a handlers section, where a list of them is specified (mail, move-to-repository, retry) that will be executed after the job export stage.
For their part, ARN-Index-type jobs, in their extraction section, define a list of maintenance actions to be executed on the Aracne indexes; these jobs do not have an export section.
An example is shown below for creating a VDP-type job, assuming that the
data source has been previously created and has the identifier stored in
vdpDatasourceIdentifier variable. This example shows how the
Configuration object for a job is created, using
ParameterStructure objects to specify the job parameters.
Configuration jobConfig = new Configuration(); jobConfig.setDraft(false); jobConfig.setType("job"); jobConfig.setSubType("vdp"); ParameterStructure parameters = new ParameterStructure(); parameters.put(Helper.buildSimpleMonoValuedParam("name", "jobName")); parameters.put(Helper.buildSimpleMonoValuedParam("description", "jobDescription")); parameters.put(Helper.buildSimpleMonoValuedParam("dataSource", vdpDatasourceIdentifier)); parameters.put(Helper.buildSimpleMonoValuedParam("parameterizedQuery", "select * from emp")); Parameter trigger = new CompoundParameter(); trigger.setName("trigger"); Collection<ParameterStructure> params = new ArrayList<ParameterStructure>(); ParameterStructure param = new ParameterStructure(); param.put(Helper.buildSimpleMonoValuedParam("type", "cron")); param.put(Helper.buildSimpleMonoValuedParam("cronExpression", "0 10,44 14 ? 3 WED")); params.add(param); trigger.setValues(params); parameters.put(trigger); jobConfig.setParameters(parameters); schedulerManager.addElement(projectId, jobConfig);
The name of the parameters to use for configuring each type of element
can be obtained from the XML files in
DENODO_HOME/metadata/scheduler/elements. They are organized in
subfolders by element type (data source, job, exporter …).
For more information please refer to the Denodo Scheduler Javadoc
documentation and the examples in