SKIP TO MAIN CONTENT
Sign In
Register
Denodo Platform 9.2
Expert Trails
Cache
Datasource Connectivity
Deployment Topology
Development Lifecycle
Modeling and Metadata Organization
Monitoring
Operations
Query Performance Optimization
Security
Considerations for Designing a Self-Service Analytics Initiative
Enterprise Data Services with APIs
Resources
User Manuals
Knowledge Base
Videos
Tutorials
Browse Content Library
Drivers
Data Management Blog
Technical Newsletter
Test Drives
Agile BI and Analytics on AWS
Data Catalog and Data Science on AWS
Agile BI and Analytics on Azure
Data as a Service (DaaS) on GCP
Agile BI and Analytics on Alibaba
Denodo Express
Learn
Training Courses
Learning Paths
Course Catalog
Your Progress
Workshops
Certification Exams
Academic Program
Overview
University Challenge
Collaborate
Q&A Forum
All Questions
Ask a Question
Search by tag
Denodo Sharing
User Groups
Support
Support Site
Contact Us
USER MANUALS
Denodo Platform
6.0
Denodo Platform Installation Guide
Legal Note
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
Preinstallation Tasks
Hardware Requirements
Recommendations when Running Denodo on Amazon AWS
Disk Space Requirements of the Denodo Platform
Requirements for the Virtual DataPort Administration Tool
Software Requirements
Supported Platforms
Supported Java Runtime Environments (JRE)
Supported Browsers
Software Requirements for ITPilot
Software Requirements for Aracne
Other Preinstallation Tasks
Check that the Required Ports Are Free
Check the PATH Environment Variable on Windows
Select a User Account to Install the Denodo Platform
Close All the Browsers
Download the Installer
Using the Graphical Installation Wizard
Commons Settings
Components and Configuration of Virtual DataPort
Components and Configuration of ITPilot
Initial Configuration of the Browser Pool
Wrapper Server
Verification Server
Installation Settings of Aracne
Aracne Server
Indexing/Search Engine Server
Installation Settings of Scheduler
Embedded Web Container
Using the Command Line Installer
Unattended Installation of the Denodo Platform and Its Updates
Modifying the Denodo Platform Installer to Include the Last Update
Unattended Installation of the Denodo Platform
Unattended Installation of Updates and Hotfixes
Postinstallation Tasks
Install the License
Install the Latest Update
Enable SSL Connections in the Denodo Platform Servers
Obtaining and Installing an SSL Certificate
Enabling SSL in Denodo Platform Servers
Enabling SSL in Denodo Platform Tools
Enabling HTTPS in the Embedded Apache Tomcat
Enabling SSL for External Clients
Configure Windows Services
Installing the Denodo Web Applications as Windows Services
Postinstallation Tasks in Virtual DataPort
Change the Host Name in the Virtual DataPort Server
Change the Default Password
Setting-up Kerberos Authentication
Setting-up Kerberos Authentication in the Information Self-Service Tool
Enabling the Support for ODBC Sources When the Virtual DataPort Server Runs on Linux
Installing the Connector for Oracle Essbase
Installing the SAP JCo Connector
Installing the XMLA Connector for SAP BW and SAP BI (Multidimensional Sources)
Installing the Microsoft Access Driver
Installing the JMS Connectors to Create JMS Listeners and Web Services with SOAP Over JMS
Configuring the VCS Clients
Importing the Certificates of Data Sources (SSL Connections)
Increasing the Maximum Simultaneous Requests
Resource Manager: Modify the Priority of a Query Dynamically
Launching the Administration Tool in High DPI Displays
Preparing the Connection to Databases
Grant Privileges in SAP BW to the Virtual DataPort User Account
Grant Privileges on SAP for BAPI Sources
Enable XMLA on Microsoft SQL Server Analysis Services (Multidimensional Sources)
Postinstallation Tasks in ITPilot
Checking that the Generation Environment Has Been Installed
Configuration of Microsoft Internet Explorer
Enabling Internet Explorer Sequence Generation Toolbar in Microsoft Windows Server 2008
Disabling Internet Explorer Enhanced Security Configuration in Microsoft Windows 2008
Disabling Internet Explorer Enhanced Security Configuration in Microsoft Windows Server 2012
Manual Configuration of Adobe Acrobat Professional
Automatic Verification Database
Launching the Wrapper Generator Tool in High DPI Displays
Postinstallation Tasks in Aracne and Scheduler
Microsoft Internet Explorer Configuration
Setting-up Kerberos Authentication in Scheduler
Postinstallation Tasks in the Denodo Web Container
Secure the “Export” Endpoint of the Web Container
Installing Updates and Hotfixes
Installing an Update or Hotfix Graphically
Installing an Update or Hotfix from the Command Line
Denodo Platform Control Center
Launching the Denodo Platform Control Center
Control Center Help
Online Help Configuration
Starting Platform Servers and Tools
Configuration of Custom Environments
Virtual Machine and Web Container Configuration
Uninstalling the Denodo Platform
Configuration of the JVM Parameters from the Command Line
Appendix
Default Ports Used by the Denodo Platform Modules
Limitations of the Denodo Express License
Using Kerberos Authentication in Virtual DataPort Without Joining a Kerberos Realm
Using Kerberos Authentication in the Information Self-Service Tool Without Joining a Kerberos Realm
Using Kerberos Authentication in Scheduler Without Joining a Kerberos Realm
How to Debug Kerberos in Web Applications
Virtual DataPort Server and the Web Administration Tools are on the Same Installation
Virtual DataPort Server and the Web Administration Tools are not on the Same Installation
Providing a Krb5 File for Kerberos Authentication
Launching the Denodo Standalone Applications in High DPI Displays
Deploying the Web Administration Tool in an External Web Container
Troubleshooting the Denodo Platform Installer
Denodo Platform Migration Guide
Preface
Scope
Who Should Use This Document
Outline of the Migration Process
Before Installing a New Version of the Denodo Platform
Export the Metadata of the Current Installation
Modify the Virtual DataPort Metadata
Import the Metadata to the New Installation
After Importing the Metadata Into the New Installation
Test the New Denodo Platform Installation
Backward Compatibility
Changes in the Denodo Platform 6.0
Changes Common to All the Modules
Virtual DataPort
Scheduler
Embedded Web Container
Changes in the Denodo Platform 5.5
Cache
CAST Function
Reserved Words
Prohibit Incompatible Source Type Properties
Three-valued Logic
Features Removed in Denodo 5.5
Changes in the Denodo Platform 5.0
Case-Sensitivity of Elements
Comparison of Strings
Dependencies Between Elements
Exporting to a Repository of Files
JMS Listeners
Web Services Published By Denodo
Stored Procedures
Features Deprecated in Denodo 5.0
Denodo Platform New Features Guide
Preface
Scope
Audience
What Is New in Denodo 6.0 Update 20201123
Installer
Command Line Installer
Unattended Installations
Client Installer
Common to All Modules
Information Self-Service Tool
Control Center
Privileges
Logging System
Clearer Date Format
Stability Improvements: SSL
Windows Services
Virtual DataPort
Using Elements from Other Database
Administration Tool
Execution Trace
Data Lineage: Show the Implementation Expressions of Interface Views
“Please Wait Dialog” Indicates User is waiting for Transaction
Date and Time Values
Support for Windows 10
JDBC Data Sources
Support for Amazon Redshift
Support for Teradata Version 14 and 15
Upgrade of the jTDS Driver
Oracle Adapter Indicates Escape Character
IBM DB2: Support for Nickname Elements
Microsoft SQL Server: Support for Pass-Through Authentication Using the Domain of the User
Data Sources with OAuth Authentication: Tokens as Interpolation Variables
Excel Data Sources: Allow Selecting Multiple WorkSheets
Execution Engine
Automatic Data Movement Based on Statistics
Improvements in the Cost-Based Optimizations
Support for Analytical Functions Delegable to the Source
Support for ORDER BY with Expressions
Canceled Queries Are Terminated after Waiting Five Minutes for all the Threads to Finish
Navigational Queries
SELECT_NAVIGATIONAL: Support for the EXPAND clause
SELECT_NAVIGATIONAL: Support for Projecting COUNT(*) and Other Aggregation Functions with an Alias.
Performance Improvements
LDAP Authentication
Improvements on the Metadata Cache
Custom Functions API: Provide Access to Contextual Information of the Query
Out-of-the-box Stored Procedures: CATALOG_PERMISSIONS
Monitoring
Denodo Monitor
MBean VDBServerManagementInfo: New Operation getRequestById.
MBeans TransactionsManagementInfo Events: New Attribute SessionId
Version Control System (VCS)
Support for GIT
Cache
New Database Adapters: Vertica 7 and Teradata 14 and 15
Changing a Parameter of the Connection
Resource Manager
Upgrade of the Java Secure Channel Library (JSCH)
JDBC Driver
Support for the JDBC 4.1 API
Support for Ignoring the “setAutoCommit” Parameter
ODBC Driver
RESTful Web Service: Simpler Format for Conditions with Compound Values
Backward Compatibility Between Updates of the Same Major Version
New Version of the Logging Library: Apache Log4j 2
Embedded Web Container
New Version of the Web Container: Apache Tomcat 8
Tomcat Uses One More Port
Virtual DataPort Administration Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
General Architecture
Physical Layer
Logical Layer
Data Module: Cache
User Layer
Installation and Execution
Launching the Virtual DataPort Server
Launching the Virtual DataPort Administration Tool
Documentation and Online Help
Main Areas of the Administration Tool
Tool Preferences
Using the VQL Shell
Query Monitor
Catalog Search
Invalidate Cache
Trace Viewer
Creating Views
Access to the Graphic Administration Tool
Importing Data Sources and Creating Base Views
JDBC Sources
ODBC Sources
SOAP Web Service Sources
Multidimensional Database Sources
Path Types in Virtual DataPort
XML Sources
JSON Sources
Delimited File Sources
Excel Sources
Web Sources (WWW)
Google Search Sources
Aracne Sources
LDAP Sources
BAPI Sources
Custom Sources
Viewing the Schema of a Base View
Data Source Configuration Properties
Source Refresh
Working with Blob Fields of Base Views
Deleting a Data Source
Creating Derived Views
Creating Union Views
Creating Join Views
Creating Selection Views
Creating Flatten Views
Creating Intersection Views
Creating Minus Views
Creating Interface Views
Viewing the Schema of a Derived View
Tree View
Data Lineage
Materialized Tables
Temporary Tables
Querying Views
Execution Trace of a Statement
Editing / Replacing a View
Views Affected by Modifications
Advanced Configuration of Views
Configuring the Cache of a View
Memory Usage and Swapping Policy of Views
Internationalization Configuration
Query Capabilities
View Configuration Properties
Indexes of Views
Statistics of the View
Editing the Execution Plan
Defining the Data Movements of the View
Stored Procedures
Importing Stored Procedures
Executing Stored Procedures
Use of Stored Procedures in Creating Views
Publication of Web Services
Publishing SOAP Web Services
Operations Tab
Settings Tab (SOAP)
Advanced Tab (SOAP)
Publishing REST Web Services
Resources Tab
Settings Tab (REST)
Advanced Tab (REST)
XSLT Transformations
Web Services Authentication
HTTP Basic
HTTP Basic with LDAP
HTTP Basic with VDP
HTTP Digest
HTTP SPNEGO (Kerberos)
SAML 2.0
How Web Services Query the Virtual DataPort Server
Connection from the Web Services to the Server
Web Service Container Status Table
Invoking the Exported Web Services
Invoking Web Services with SAML Authentication
Types Conversion Table for REST / SOAP Published Web Services
Publication of Views as Widgets
Publish a View as a Widget
Auxiliary Web Services
Export to JSR-168 or JSR-286 Portlet
Export as Microsoft Web Part
Deployment of a Microsoft Web Part
RESTful Architecture
Primary Keys of Views
Associations
Why Associations Are Useful?
Creating an Association
Multiplicity of Associations
Referential Integrity in Associations
Role Preconditions
RESTful Web Service
Representations
IDU Requests
Requests with Input Compound Values
Status Codes
Obtaining the Number of Rows of a Result Set
Security and Access Privileges
Configuring the RESTful Web Service
JMS Listeners
Creating a New JMS Listener
Who Can Create JMS Listeners
Server Administration - Configuring the Server
Server Connectivity
Threads Pool
Configuring the Cache
Limiting the Number of Concurrent Requests
Default Configuration of HTTP Proxy
Configuring the Memory Usage and Swapping Policy
Memory Usage Settings
Swapping Settings
Maximum Size of a Query
Queries Optimization
Configuring Runtime Parameters for Stored Procedures
Configuring the Default Internationalization
Identifiers Charset
Restricted Mode
Unicode Mode
Kerberos Authentication
Creating the Roles of the Virtual DataPort Users
Creating an LDAP Data Source
Setting-Up the Kerberos Authentication in the Virtual DataPort Server
Configuring the Administration Tool to Use Kerberos Authentication
SAML Authentication
Privileges: Global Settings
The FILE Privilege
Privileges of Users with the Role “serveradmin”
Memory Management
Streaming Vs Non-Streaming Operators
Swapping Parameters
Limit the Maximum Amount of Memory of a Query
Edge Cases in Streaming Operation
Cache Load Processes
Resource Manager
Defining a Plan
Defining a Rule
Exporting and Importing the Server Metadata
Exporting the Server Metadata
Exporting and Importing Elements Across Different Environments
Exporting Environment-Dependent and Independent Elements to Different Files
Export to a File with Properties
Importing Metadata into a Server
Importing Extensions
Cache Module
Generic Support for Other Databases
Cache Modes
Partial Mode
Full Mode
Incremental Mode
Recommended Parameters for Queries that Load the Cache
Caching Very Large Data Sets
Caching the Result of Queries that Fail
Cache Maintenance Task
Maintaining the Underlying Database
Version Control Systems Integration
VCS Configuration
Virtual DataPort Server Configuration
Environment Management
Database Configuration
Importing an Existing Database from a VCS Server
VCS Integration Features
Supported Element Types
Integration with the Server Explorer
VCS Operations for Microsoft TFS and Subversion
VCS Operations for GIT
Delete Operations
Other Version Control Operations
Dependencies Management
VCS Settings of the Administration Tool
Scenarios And Recommended Uses
Picking a VCS Workflow
Centralized Workflow with Shared Databases
Distributed Workflow
Centralized Workflow with Private Databases
Best Practices to Manage Global Elements
Promoting a Database from Development to Testing and Production
Using VCS Environments
Best Practices When Using the Integration with a VCS
Tagging and Branching Releases
Recommendations for the Testing Environment
Uniqueness Detection
Enabling Uniqueness Detection
Examples
LS Optimization for Subversion
Activating the LS Optimization
Tested Environments
Databases, Users and Access Rights in Virtual DataPort
Databases in Virtual DataPort
User and Access Right in Virtual DataPort
Types of Access Rights
Column Privileges
Row Restrictions
Enforcing Column Privileges, Row Restrictions and Custom Policies
Roles
Administration of Databases, Users, Roles and Their Access Rights
Creating Databases
Configuring and Deleting Databases
Creating Users
Modifying and Deleting Users
Creating Roles
Monitoring the Virtual DataPort Server
Monitoring with a Java Management Extensions (JMX) Agent
Using JavaTM VisualVM
General Information on the Server
General Information on Data Sources
Information on the Cache
Information and Events on Catalog Access (DDL Statements)
Information and Events on the Running of Statements
Information and Events on Transactions
Setting the User Agent of an Application
Denodo Monitor
Local Monitors
Server Monitors
Virtual DataPort Monitors
Configuring the Denodo Monitor
Launching the Denodo Monitor
Cluster Architectures / Server Backup
Denodo Tools
How to Check If a Virtual DataPort Server Is Alive
Connecting from a JDBC Client Through a Load Balancer
Using the Ping Script
Alternatives to the Ping Script
Using the Import/Export Scripts for Backup And/or Replication
Export Script
Import Script
Configuring Several Instances of a Virtual DataPort Server
How to Configure the Slave Instances
Bulk Data Load
Amazon Redshift
DB2
Impala
Oracle
Snowflake
Microsoft SQL Server
Teradata
Settings of the Generation of the Temporary Files
Optimizing Queries
Optimizing DF Data Sources
Optimizing Join Operations
Merge Join
Nested Join
Nested Parallel Join
Hash Join
Automatic Simplification of Queries
Removing Redundant Branches of Queries (Partitioned Unions)
Pushing Down GROUP BY Views Below JOIN Views
Pushing Down GROUP BY Views Below UNION Views
Selecting the Most Optimal Source When the Data Is Replicated in Several Sources
Cost-Based Optimization
Enabling the Cost-Based Optimization
Gathering the Statistics of Views
Tuning the Cost-Based Optimization Process
Current Limitations of the Cost-Based Optimization Process
Data Movement
Data Movements From/To Netezza Databases
Examples of Data Movement
Known Limitations
Options of the CONTEXT Clause that Control a Data Movement
Appendix
Supported JDBC Data Sources
Mapping Multidimensional Data to a Relational Model
Creating a Multidimensional Base Views Over a Multidimensional Data Source
Mapping the Result of MDX Queries
Query Delegation to SAP BW
Considerations When Configuring Data Sources with Pass-Through Credentials
Configuring the Logging Engine
Installing the Denodo Solution for Microsoft SharePoint
Customizing Events and Public Render Parameters of JSR-286 Portlets
Transforming Incoming/Outgoing Soap/Rest Messages with XSLT Stylesheets
JMS Connection Details: JNDI Properties
Apache ActiveMQ 5.6.0
IBM WebSphere MQ 7.0
Progress SonicMQ 8.0
Execution Trace Information
Web Services Created with Previous Versions of the Denodo Platform
Publishing Web Services
XSLT Transformations of Web Services Created with Previous Versions
SOAP Over JMS on Web Services Created with Previous Versions of the Denodo Platform
Authentication in Web Services Created with Previous Versions
Types Conversion Table for REST / SOAP Web Services Created with Previous Versions
Invoking Web Services Created with Previous Versions
Allow URIs with Slash and Backslash in Apache Tomcat
Useful Tools to Debug Issues with Active Directory or Other LDAP Servers
Single User Mode
Explicit Single User Mode
Automatic Single User Mode
Resource Manager: Available Fields to Evaluate a Rule
Examples of Resource Manager Rules
Values of the Attribute “Access Interface”
Avoiding SQL Injections
Virtual DataPort Developer Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
Examples
Access Through JDBC
Parameters of the JDBC Connection URL
Connecting to Virtual DataPort Through a Load Balancer
When to Use the “Basic” Version of the JDBC Driver
Connecting to Virtual DataPort with SSL
Connecting to Virtual DataPort Using Kerberos Authentication
When the Client Application Does Not Belong to the Domain
Details of the JDBC Interface
Description of Views and Their Fields
Retrieving the Content Type of Blob Values
Obtaining the Names of Elements Inside a Struct (Register)
Access Through ODBC
Configuration of the ODBC Driver on Windows
Install the ODBC Driver on Windows
Set Up a DSN on Windows
Configuration of the ODBC Driver in Linux and Other UNIX
Install the ODBC Driver on Linux
Set Up a DSN on Linux and Other UNIX
Creating a DSN-Less Connection
Integration with Third-Party Applications
Increasing the Performance of the Denodo ODBC Driver
Supporting Queries with Brackets
Maximum Length of Text Values
Maximum Length of Error Messages
Disabling Transactions
Access Through an ADO.NET Data Provider
Using Kerberos Authentication
Access Through OLE DB
Developing Extensions
Developing Custom Functions
Creating Custom Functions with Annotations
Creating Custom Functions Using Name Conventions
Compound Types
Custom Function Return Type
Getting Information About the Context of the Query
Developing Stored Procedures
Required Libraries to Develop Stored Procedures
Developing Custom Wrappers
Required Libraries to Develop Custom Wrappers
Extending AbstractCustomWrapper
Overriding AbstractCustomWrapper
Dealing with Conditions
Dealing with the ORDER BY Clause
Configuring a Custom Wrapper
Updating the Custom Wrapper Plan
Developing Custom Input Filters
Required Libraries to Develop Custom Filters
Developing Custom Filters
Custom Policies
Developing a Custom Policy
Appendix
Output Schema of the LIST Command
Output Schema of the DESC Commands
Error Codes Returned by Virtual DataPort
Virtual DataPort VQL Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
General Overview of Virtual DataPort
Creating or Defining Data
Defining Base Views
Defining Data Sources and Wrappers
Defining the Views of the Global Schema
Executing Statements
Language for Defining and Processing Data: VQL
Data Types
Internationalization
Statements
Unicode Identifiers
Logical Operators
Comparison Operators
Comparing Literals
Three-valued Logic
Functions for Conditions and Derived Attributes
Syntax Conventions
Syntax of Functions and Condition Values
Comments
Creating a Base View
Query Capabilities: Search Methods and Wrappers
Query Constraints
Assigning Wrappers to Search Methods
Example of How a Search Method Is Created
Modifying a Base View
Queries: SELECT Statement
FROM Clause
JOIN Operation
INTERSECT Operation
MINUS Operation
FLATTEN View (Flattening Data Structures)
Subqueries in the WHERE Clause of the Query
WITH Clause
SELECT Clause
Derived Attributes
WHERE Clause
Conditions with Compound Values
Group BY Clause
Use of Aggregation Functions
HAVING Clause
UNION Clause
Specifying Projections in UNION Queries
ORDER BY Clause
OFFSET, FETCH and LIMIT
CONTEXT Clause
TRACE Clause
CASE Clause
Defining a Derived View
Modifying a Derived View
Defining an Interface View
Defining the Statistics of a View
Materialized Tables
Creating Materialized Tables
Inserting Data Into Materialized Tables
Temporary Tables
Creating Temporary Tables
Inserts, Updates and Deletes Over Views
INSERT Statement
UPDATE Statement
DELETE Statement
Return Modified Rows
Limitations
Use of WITH CHECK OPTION
Transactions in Virtual DataPort
Stored Procedures
Importing a Stored Procedure
Use of Stored Procedures
Predefined Stored Procedures
CACHE_CONTENT
CATALOG_ELEMENTS
CATALOG_FKS
CATALOG_METADATA_VIEWS
CATALOG_PERMISSIONS
CATALOG_PKS
CATALOG_VDP_METADATA_VIEWS
CATALOG_VIEWS
CLEAN_CACHE_DATABASE
COLUMN_DEPENDENCIES
COMPACT_METADATA_TABLES
DUAL
GENERATE_SMART_STATS_FOR_FIELDS
GENERATE_SMART_STATS_FOR_FIELDS_BY_TABLENAME
GENERATE_STATS
GENERATE_STATS_FOR_FIELDS
GENERATE_VQL_TO_CREATE_JDBC_BASE_VIEW
GET_CACHE_COLUMNS
GET_CACHE_CONFIGURATION
GET_CACHE_TABLE
GET_CATALOG_EFFECTIVE_PERMISSIONS
GET_ELEMENTS
GET_EXPORTED_KEYS
GET_FOREIGN_KEYS
GET_JDBC_DATASOURCE_TABLES
GET_PRIMARY_KEYS
GET_PROCEDURE_COLUMNS
GET_SESSIONS
GET_SOURCE_COLUMNS
GET_SOURCE_TABLE
GET_TYPE_ATTRIBUTES
GET_USER_DEFINED_TYPES
GET_VIEW_COLUMNS
GET_VIEW_INDEXES
GET_VIEWS
LIST_JDBC_DATASOURCE_TABLES
LOGCONTROLLER
SOURCE_CHANGES
VIEW_DEPENDENCIES
WAIT
WEBCONTAINER_ELEMENTS
WRITELOGERROR
WRITELOGINFO
Defining Other Elements of the Catalog
Creating Folders
Defining a Data Type
Defining a Map
Defining JAR Extensions
Definining JMS Listeners
Creating Databases, Users, Roles and Access Privileges
Creating and Modifying Virtual DataPort Databases
Creating Users
Modifying and Deleting Users
Connecting to Another Database
Modifying the Privileges of a User
Managing User Roles
Describing Catalog Elements
Exporting Metadata
Importing Metadata
Listing Elements in the Catalog
Removing Elements from the Catalog
Publication of Web Services
Creating New SOAP Web Services
Creating New REST Web Services
Connection Parameters
Web Services Authentication
Basic and Digest
Basic LDAP
SAML
VDP
WSS
Embedded Web Container Management
Deploying and Exporting SOAP and REST Web Services
Publication of Widgets
Create New Widgets
Export a Widget
Deployment and Export of Auxiliary Web Services
RESTful Architecture
Associations
Navigational Queries
Help Command
Generating Wrappers and Data Sources
Valid Conversions Between Types in Wrappers and VDP Types
Native-type Conversions of a Wrapper to Java Types
Filters
Creating Data Sources
JDBC Data Sources
ODBC Data Sources
Multidimensional Data Sources
Data Sources for Web Services
XML Data Sources
JSON Data Sources
DF Data Sources
Denodo Aracne Data Sources
Google Search Data Sources
LDAP Data Sources
BAPI Data Sources
Custom Data Sources
Data Source Configuration Properties
Specifying Paths in Virtual DataPort
Creating Wrappers
Execution Context and Interpolation Strings
Wrapper Metadata
JDBC Wrappers
Multidimensional Databases Wrappers
ODBC Wrappers
WWW Wrappers
Web Services Wrappers
XML Wrappers
JSON Wrappers
DF Wrappers
Denodo Aracne Wrappers
Google Search Wrappers
LDAP Wrappers
BAPI Wrappers
CUSTOM Wrappers
Wrapper Configuration Properties
Query WRAPPER Statements
Version Control Systems Integration Commands
Statements to Work with Centralized Version Control Systems
VCSCOMMIT
VCSUPDATE
VCSCONTENTS
VCSSHOW
REVISIONS
Statements to Work with GIT
DVCSCOMMIT
DVCSPUSH
DVCSPULL
DVCSREVERT
Commands for Environment Management
Other Version Control Commands
VCSDISCARD
Resource Manager
Managing the Plans of the Resource Manager
Managing the Rules of the Resource Manager
Advanced Characteristics
Management of Compound Values
Processing of Compound Types: Example
Dynamic Choice of Join Strategy
Using the Cache
Cache Invalidation
Displaying the Contents of the Cache
Configuring Swapping Policies
Managing Internationalization Configurations
Replacing an I18n Map
Deleting an I18n Map
Execution Context of a Query and Interpolation Strings
Adding Variables to Selection Conditions (GETVAR and SETVAR)
Appendix
Syntax of Condition Functions
Arithmetic Functions
Text Processing Functions
Date Processing Functions
XML Processing Functions
Type Conversion Functions
Aggregation Functions
Analytic Functions (Window Functions)
Other Functions
Syntax of Search Expressions for the Contains Operator
Exact Terms and Phrases
Term Modifiers
Boolean Operators
Groups
Escaping Special Characters
Support for the Contains Operator of Each Source Type
Aracne
Google Search
CASE Clause Examples
Date and Time Pattern Strings
Web Services Created with Previous Versions of the Denodo Platform
Information Self-Service Tool Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
Installation and Execution
Launching the Information Self-Service Tool
Authentication And Authorization
Configuration
Servers Configuration
Search Configuration
Personalization
User Interface
Export Format Configuration
Global Parameters
Database Connection URIs
Kerberos Configuration
User Configuration
Databases
Summary
Associations
Search
Metadata Search
Views
Summary of Views
Query
Associations of Views
Lineage of Views
Search of Views
Connect
Saved Queries
Deploying Saved Queries
Copying Saved Queries
Appendix
Installing the Information Self-Service Tool on a Cluster: Sharing the Same Settings Across All the Nodes
Setting Up the Common Database
Configure the Information Self-Service Tool to Use the Common Database
Diagnostic & Monitoring Tool Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
Installation and Execution
Launching the Monitoring and Diagnostic Tool
Architecture
Authentication and Authorization
Kerberos Authentication
Creating Servers and Environments
Creating Servers
Creating Environments
Monitoring
Monitoring Servers
Monitoring - Resources
Monitoring - State
Monitoring - Sessions
Monitoring - Requests
Monitoring - Cache
Monitoring - Data Sources
Monitoring - Threads
Monitoring Environments
Overview
Creating Diagnostics and Diagnostic Intervals
Creating Diagnostics
Creating Diagnostic Intervals
Diagnosing
Diagnosing - Resources
Diagnosing - State
Diagnosing - Sessions
Diagnosing - Requests
Diagnosing - Cache
Diagnosing - Data Sources
Diagnosing - Threads
Diagnosing - Errors
Export and Import
Export
Import
Configuration
Configuration - General
Configuration - Overview
Configuration - Resources
Configuration - State
Configuration - Sessions
Configuration - Requests
Configuration - Cache
Configuration - Data Sources
Configuration - Threads
Configuration - Errors
Virtual DataPort API Javadoc
Scheduler Administration Guide
Preface
Scope
Who Should Use This Manual
Summary of Contents
Introduction
General Architecture
Installation and Execution
Administration
Authentication
Web Configuration
User Configuration
Kerberos Configuration
Scheduler Server Configuration
Server Configuration Sections
Permissions
Plugins and JDBC Adapters
Log Configuration
Creating and Scheduling Jobs
Jobs
Data Sources
ARN Data Sources
ARN-Index Data Sources
CSV Data Sources
Elasticsearch Sources
ITP Data Sources
JDBC Data Sources
VDP Data Sources
Filter Sequences
Boolean Content Filter
Content Extraction Filter (HTML, PDF, Word, Excel, PowerPoint, XML, EML, and Text)
Field Aggregation Filter
Summary Generation Filter
Title Generation Filter
URL Unicity and Standardization Filters
Useful Web Content Extraction Filter
Configuring New Jobs
General Structure of a Job
Aracne-type Job Extraction Section
VDP Extraction Section
ITP Extraction Section
JDBC Extraction Section
VDPCache Extraction Section
VDPIndexer Extraction Section
Data Schema Generated by the Different Types of Extraction Jobs
Jobs for Maintaining Aracne Indexes
Postprocessing Section (Exporters)
Retry Section
Handler Section
Time-based Job Scheduling Section
Dependencies Among Jobs
Developer API
Scheduler Client API
Extensions (Plugins)
Filters
Exporters
Handlers
Aracne Custom Crawlers
Appendix
Dateformat Function Syntax
Regular Expressions for Filters
JDBC Drivers
Use of the Import/Export Scripts for Backup
Export
Import
Use of the Ping Script
Scheduler API Javadoc
Aracne Administration Guide
Preface
Scope
Who Should Use This Manual
Summary of Contents
Introduction
General Architecture
Installation and Execution
Administration
Authentication
Aracne Server Administration (ARN-CRAWLER)
Proxy Configuration
Execution Threads
Aracne Search/Index Engine Server Administration (ARN-INDEXER)
Aracne Server Configuration
Administration of Indexes
Administration of Index Schemas
Search Engine
Import and Export Server Metadata
Log Configuration
Denodo Aracne API - Search/Indexing
Most Relevant Terms of a Document
Extensions
Creating New Functions for Regular Expressions
Appendix
Apache Lucene Search Syntax
Terms
Fields
Term Modifiers
Boolean Operators
Grouping
Field Grouping
Escaping Special Characters
Use of the Import/Export Scripts for Backup
Aracne API Javadoc
ITPilot User Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
Denodo ITPilot Environments
Administration Tool
Generation Environment
Execution Environment
Verification Environment
Distribution of Environments
Distribution of the Generation Environment
Distribution of the Execution Environment
Distribution of the Verification Environment
Installation and Initial Configuration
Execution
Starting Up the Administration Tool
Starting Up the Browser Pool
Starting Up the Wrapper Server
Starting Up the Verification Server
Starting Up the Verification Server with a Graphical Tool
Starting Up the PDF Conversion Server
Web Administration Tool
Adding, Editing and Deleting Servers
Configuring the Browser Pool Server
Global Configuration Parameters
MSIE and Denodo Browser Configuration
Configuration of the Wrapper Server
List of Wrappers
Server Configuration (Wrapper Server)
Embedded Denodo Browser Configuration
Extensions
Configuring the Verification Server
Server Configuration (Verification Server)
Database Configuration
Mail Configuration
Rules Configuration
Verification Server Graphical Configuration and Monitoring Tool
An Overview of the Graphical Tool
Configuration of the Verification Environment
Monitoring the Wrappers Under Verification
Wrapper Execution Trace Logs
Enabling Wrapper Trace Logging
Log Categories and Effective Log Level
Appenders
Configuring Custom Appenders
Custom Appenders and Logger Additivity
ITPilot Generation Environment Guide
Preface
Scope
Who Sould Use This Document
Summary of Contents
Introduction
Presentation
Development of Component-Based Wrappers
Installation and Execution
Introduction to the Tools
Generation Environment Tools - Part I
Presentation of the Example
Creating Projects and Wrappers
Components In ITPilot
Input and Output Parameters
Process Initialization
Use of the Catalog Explorer
Web Browsing Automation
Component Configuration
Comparison Between MSIE and Denodo Browser
Advanced Configuration of the Sequence and Next Interval Iterator Components
Error Processing On the Web Browsing Automation
Configuration of the Extractor Component
Defining the Structure of the Data and Assigning Examples
Generating a DEXTL Specification from the Examples
Testing the Generated Specification
Locale Configuration of the Extractor Component
Error Processing On the Extractor Component
Processing the Retrieved Results
Use of the Iterator Component
Processing the Individual Records: Use of Record Constructor
Returning Results: Output Component
Wrapper Generation, Tests and Exporting
Wrapper Generation
Wrapper Execution
Deploying Wrappers to the Wrapper Server
Generation Envirtonment Tools - Part II
Extracting Multipaginated Data
Pagination Using Single Block or Multiple Block Schemes
Access to Details Pages
Access to the Details Page: Using Extractor Sequence
Individual Test of Extractor Sequence, Next Interval Iterator and Form Iterator Components
Specifying the Structure of the Detail Page Information
Generating the Access Specification to the Details Page
Iteration on the Details Page Structures and Creation of the Output Record
Generation Tool Global Preferences
Tagsets and Scanners
Understanding Tagsets
Graphical Creation of New Tagsets
Tagsets and Scanners Included in the Distribution
Lexer Types
Generating the Data Extraction Specifications Manually
Creating a Structure Using the Structure Pane
Obtaining the Tokens of the Data to Extract
Creating a DEXTL Specification from the Tags
Using the Sequence Debugger
Breakpoint Management
Saving a Flow as a Custom Component
Deploying a Custom Component to the Wrapper Server
Wrapper Advanced Options: Specific Browser Pool and Locale
Specific Browser Pool
Locale
Migrating Wrappers Between Generation Environments: Import and Export
Exporting Wrappers
Importing Wrappers
Managing Extensions
Adding Extensions
Deleting Extensions
Exporting a Flow as an Extension
Generating Navigation Sequences
Description of the Navigation Sequences Generator Interface
Toolbar State Info Dialog
Steps for Generating a Navigation Sequence
Checking Navigation Sequences in Systems with Cookie-Based Session Authentication and Maintenance
The SelectFrame Action
The Transpose Table Button and Action
The Cancel Navigation Button
Recording Sequences with Passwords
Properties of the Navigation Bar
Generating Sequences Using an Authenticated Proxy
Criteria for Selecting NSEQL Commands
Advanced Options
XPath Generator Utility
Appendix A: ITPilot Operators
Logical Operators
Comparison Operators
Appendix B: ITPilot Functions
Arithmetic Functions
Text Processing Functions
Functions for Handling Null Values
Functions for Handling Lists
Date Processing Functions
Type Conversion Functions
Functions for URL Processing
Functions for Page Handling
Functions for Handling Compressed Files
Extraction Functions
Case Clause
Appendix C: Catalog of Components
Add Record To List
Condition
Using the Conditions Editor
Create List
Create Persistent Browser
Diff
Execute JavaScript
Expression
Using the Expressions Editor
Extractor
Extractor Sequence
Filter
Form Iterator
Get Page
INIT
Iterator
JDBC Extractor
Loop
Next Interval Iterator
Output
Record Constructor
Release Persistent Browser
Repeat
SaveFile
Script
Sequence
Appendix D: Constraints of the Simplified DOM
Appendix E: List of Reserved Words
ITPilot Developer Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
ITPilot Development API
Connecting to the Server
Obtaining Wrappers
Using Wrappers
Processing Query Results
Canceling Queries
Example of Use
Creating Custom ITPilot Functions
Naming Conventions and Annotations
Compound Types
Page Type
Custom Function Return Type
Example
ITPilot DEXTL Guide
Preface
Scope
Who Should Use This Manual
Summary of Contents
Introduction
An Overview of DEXTL
Basic Syntax
Fundamentals of Pattern Specification
Ambiguity of Patterns and Demarcation of the Search Space
Relations and Subrelations
Advanced Syntax
Extracting Tag Attributes
Tags with Evaluated Attributes
Voracious Separators
Dynamic Separators
Attributes with FIXED Value and Default Value
Format Tags
Tagsets
Hierarchy of Elements
Alternative Specifications
Scanners
Lexer Types
Structure of a DEXTL Program
Declarations Section
Specification Section
Escape Characters in DEXTL
DEXTL Quick Reference Sheets
Putting All Together
Browser View
HTML Code View
FORMAT TAGS VIEW
DEXTL VIEW
RESULTS VIEW
ITPilot NSEQL Guide
Preface
Scope
Who Should Use This Manual
Summary of Contents
Introduction
Basic Concepts of NSEQL
Navigation Sequences: Basic Syntax
NSEQL Commands: General Overview
Configuration of Page Waiting Times
Example
List of NSEQL Commands
Usual Parameters
Error Parameters
Navigation Commands
CANCELNEXTNAVIGATION
CANCELNAVIGATION
EXTENDEDWAITPAGES
GOBACK
GOFORWARD
NAVIGATE
IGNORENAVIGATIONERRORS
SETMAXDOWNLOADTIME
NAVIGATEEXTENDED
POSTDATA
REFRESH
STOP
WAIT
WAITPAGES
Commands for Processing Links
CLICKONANCHORBYHREF
CLICKONANCHORBYHREFREC
CLICKONANCHORBYPOSITION
CLICKONANCHORBYTEXT
CLICKONANCHORBYTEXTREC
CLICKONAREABYHREF
CLICKONAREABYPOSITION
SELECTANCHORBYHREF
SELECTANCHORBYHREFREC
SELECTANCHORBYPOSITION
SELECTANCHORBYTEXT
SELECTANCHORBYTEXTREC
SETTARGETURL
Commands for Selecting Forms
FINDFORMBYACTION
FINDFORMBYACTIONREC
FINDFORMBYNAME
FINDFORMBYNAMEREC
FINDFORMBYPOSITION
RESETFORM
Commands for Executing Actions on Elements of a Form
CLICKONELEMENT
CLICKONINPUTELEMENT
FIREEVENT
SELECTINDEXBYPOSITION
SELECTINDEXBYTEXT
SELECTINDEXBYVALUE
SELECTOPTIONBYPOSITION
SELECTOPTIONBYTEXT
SELECTOPTIONBYVALUE
SETINPUTVALUE
SETTEXTAREAVALUE
SETVALUE
SETCHECKED
SUBMITFORM
UNSELECTALL
Frame Selection Commands
FINDFRAMEBYNAME
FINDFRAMEBYNAMEREC
FINDFRAMEBYPOSITION
FINDFRAMEBYSOURCE
FINDFRAMEBYSOURCEREC
RESETFRAME
Commands for Selecting Elements
FINDACTIVEELEMENTINDOCUMENT
FINDCHILDELEMENTBYATTRIBUTE
FINDCHILDELEMENTBYPOSITION
FINDCHILDELEMENTBYTEXT
FINDELEMENTBYATTRIBUTE
FINDELEMENTBYATTRIBUTEREC
FINDELEMENTBYCHILD
FINDELEMENTBYCHILDREC
FINDELEMENTBYPOSITION
FINDELEMENTBYTEXT
FINDELEMENTBYTEXTREC
FINDELEMENTBYXPATH
Commands for Executing on Actions on Elements
CLICKONSELECTEDELEMENT
FIREEVENTONSELECTEDELEMENT
TRANSPOSESELECTEDTABLE
Commands for Handling Windows
ALLOWOPENNEWWINDOWS
CLOSEWINDOW
GETWINDOWNAME
OPENALERTDIALOGSINNEWWINDOW
OPENNEWWINDOW
OPENNEWWINDOWSUNNAMED
OPENWEBPAGEDIALOGSINNEWWINDOW
SELECTWINDOW
SETWINDOWNAME
CLOSEINACTIVEWINDOWS
Commands for Handling Pop-Up Dialogs
ONDIALOG
ONPRINTDIALOG
ONWEBPAGEDIALOG
ONCERTIFICATEDIALOGDENODOBROWSER
SETDIALOGPROPERTIES
SETSILENT
Commands for Editing the Browser Properties
SETAMBIENTPROPERTIES
SETPROXYAUTHINFO
SETUSERAGENT
Informative Commands
FINDCOOKIES
GETEXTENDEDSOURCECODE
GETLASTURL
GETLASTURLPOSTPARAMETERS
GETLOCATION
GETMIMETYPE
GETSELECTOPTIONS
GETSOURCECODE
GETTITLE
ISDOWNLOADING
QUERYDOWNLOADEDPAGES
Conversion Commands
CONVERTEXCELTOHTML
CONVERTWORDTOHTML
CONVERTPDFTOHTML
Commands for Saving Files
SAVEFILE
Commands for Executing Code
EXECUTEJS
EXECUTEJSBEFORELOAD
Commands for Clearing Browser Data
CLEARPERSISTENTCOOKIES
CLEARCACHE
Util Functions
EncodeSEQ
ITPilot API Javadoc
Denodo4E User Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Installing Denodo4E
VDP/ITP Server Tutorial
Creating an Extension Project with a Custom Stored Procedure
Creating Debug Configurations for the VDP/ITP Server and the VDP Administration Tool
Starting the VDP/ITP Server in debug mode
Debugging the Stored Procedure
Deploying the Stored Procedure
Wrapper Generation Tool Tutorial
Creating an Extension Project with an ITPilot Custom Function
Creating a Debug Configuration for the Wrapper Generation Tool
Starting the Wrapper Generation Tool
Debugging the Custom Function
Deploying the Custom Function
Reference of the Dialogs of the Denodo4E Plugin
Create
The Denodo Extension Project Wizard
The Denodo Extension Wizard
Debug
Denodo Application Debug Configuration
Supported Denodo Applications
Support for Dependencies
Debug Remote Applications
Deploy
The Deploy Wizard
Supported Extension Types
DenodoConnect Components
Denodo Cloud Cache Load Bypass Stored Procedure - User Manual
Denodo Dashboard (Deprecated) - User Manual
Denodo Distributed File System Custom Wrapper - User Manual
Denodo Dynamics 365 Business Central Templates - Quick Use Guide
Denodo Dynamics 365 Customer Engagement Templates - Quick Use Guide
Denodo Email Exported Files Custom Handler - User Manual
Denodo FileSystem CustomWrapper - User Manual
Denodo GeoJSON Service - User Manual
Denodo Google Sheets Custom Wrapper - User Manual
Denodo Google Sheets Tool - User Manual
Denodo HBase CustomWrapper - User Manual
Denodo Hibernate Dialect - User Manual
Denodo Incremental Cache Load Stored Procedure - User Manual
Denodo Kafka Custom Wrapper - User Manual
Denodo Log Custom Wrapper - User Manual
Denodo MapReduce Custom Wrapper - User Manual
Denodo Model Bridge - User Manual
Denodo MongoDB Custom Wrapper - User Manual
Denodo OData 2.0 Service - User Manual
Denodo OData 4.0 Service - User Manual
Denodo OData2 Custom Wrapper - User Manual
Denodo OData4 Custom Wrapper - User Manual
Denodo Power BI Custom Connector - User Manual
Denodo RFCReadTable Custom Wrapper - User Manual
Denodo Replicator - User Manual
Denodo SFTP Exported Files Custom Handler - User Manual
Denodo SSH Custom Wrapper - User Manual
Denodo SalesForce REST Custom Wrapper - User Manual
Denodo ServiceNow Templates - Quick guide
Denodo Tableau Exporter - User Manual
Denodo Templates for Twitter - Quick Use Guide
Denodo Testing Tool - User Manual
Denodo XSLT Mapper - User Manual
Denodo XtraFuncs - User Manual
Generic XML Exporter - User Manual
Quick Start Guides
Denodo Platform
Denodo Platform for AWS 6.0 Quick Start Guide
Denodo Platform for Azure 6.0 Quick Start Guide
Denodo Express 6.0 Quick Start Guide
Important:
To see the latest version of the document click here
User Manuals
/
Virtual DataPort VQL Guide
/
Defining Other Elements of the Catalog
Defining Other Elements of the Catalog
¶
The following subsection explain how to create other elements of the catalog:
Creating Folders
Defining a Data Type
Defining a Map
Defining JAR Extensions
Definining JMS Listeners
Feedback:
Share:
Add feedback
