Denodo Logo
community
  • Q&A
    • All Questions
    • Ask a Question
    • Search by tag
  • KB
    • Overview
    • Best Practices
    • Combining Data
    • Common Errors
    • Custom Elements
    • Data Sources
    • ITPilot
    • Installation & Updates
    • Northbound Connections
    • Operation
    • Performance & Optimization
    • Publishing
    • Scheduler
    • Security
    • VQL
  • USER MANUALS
    • Denodo Platform 8.0
    • Denodo Platform 7.0
    • Denodo Platform 6.0
    • Older Versions
  • VIDEOS
    • Featured
    • Data Combination
    • Data Services
    • Operation and Source Connectivity
    • Updates
    • Cloud
    • Solution Manager
  • TUTORIALS
    • Overview
    • Data Virtualization Basics
    • Data Services
    • Big Data
    • Agile BI
    • Web Automation
    • Workload Management
    • Data Discovery
    • Custom Components
  • EXPRESS
  • DRIVERS
    • Overview
    • JDBC
    • ODBC
  • FREE TRIAL
REGISTER
SIGN IN
    Denodo Logo
    community
    Denodo Platform 7.0
  • Denodo Platform Installation Guide
    • Legal Note
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Introduction
    • Preinstallation Tasks
      • Hardware Requirements
        • Recommendations when Running Denodo on Amazon AWS
        • Recommendations when Running Denodo on Microsoft Azure
        • Running the Denodo Platform on Docker
        • Disk Space Requirements of the Denodo Platform
        • Requirements for the Virtual DataPort Administration Tool
      • Software Requirements
        • Supported Platforms
        • Supported Java Runtime Environments (JRE)
        • Supported Browsers
        • Software Requirements for ITPilot
      • Other Preinstallation Tasks
        • Check that the Required Ports Are Free
        • Check the PATH Environment Variable on Windows
        • Select a User Account to Install the Denodo Platform
        • Close All the Browsers
      • Download the Installer
    • Using the Graphical Installation Wizard
      • Commons Settings
      • Components and Configuration of Virtual DataPort
      • Components and Configuration of ITPilot
        • Initial Configuration of the Browser Pool
        • Wrapper Server
        • Verification Server
      • Installation Settings of Scheduler
        • Scheduler Server
        • Scheduler Index Server
      • Embedded Web Container
    • Using the Command Line Installer
    • Unattended Installation of the Denodo Platform
      • Modifying the Denodo Platform Installer to Include the Latest Update
    • Post-Installation Tasks
      • Install the Latest Update
      • Change the Host Name in the Virtual DataPort Server
        • Changing the Host Name of the Denodo Servers Graphically
        • Changing the Host Name of the Denodo Servers from the Command Line
      • Register the Denodo Servers in the License Manager
      • Enable SSL/TLS in the Denodo Platform
        • Obtaining and Installing an SSL/TLS Certificate
        • Enabling SSL/TLS in Denodo Platform Servers
        • Enabling HTTPS in the Embedded Apache Tomcat
        • Enabling SSL/TLS in the Administration Tool and Others
        • Enabling SSL/TLS for External Clients
        • SSL/TLS Versions Supported by the Denodo Platform Servers
      • Configure Windows Services
        • Installing the Denodo Web Applications as Windows Services
      • Post-Installation Tasks: Virtual DataPort
        • Change the Default Passwords
        • Setting-up Kerberos Authentication
        • Setting-up Kerberos Authentication in the Data Catalog
        • Enabling the Support for ODBC Sources When the Virtual DataPort Server Runs on Linux
        • Installing the Connector for Oracle Essbase
        • Installing the SAP JCo Connector
        • Installing the Microsoft Access Driver
        • Installing the JMS Connectors to Create JMS Listeners and Web Services with SOAP Over JMS
        • Configuring the VCS Clients
        • Importing the Certificates of Data Sources (SSL/TLS Connections)
        • Increasing the Maximum Simultaneous Requests
        • Resource Manager: Modify the Priority of a Query Dynamically
        • Preparing the Connection to Databases
        • Grant Privileges in SAP BW to the Virtual DataPort User Account
        • Grant Privileges on SAP for BAPI Sources
        • Enable XMLA on Microsoft SQL Server Analysis Services (Multidimensional Sources)
      • Post-Installation Tasks: ITPilot
        • Checking that the Generation Environment Has Been Installed
        • Configuration of Microsoft Internet Explorer
        • Enabling Internet Explorer Sequence Generation Toolbar in Microsoft Windows Server 2008
        • Disabling Internet Explorer Enhanced Security Configuration in Microsoft Windows 2008
        • Disabling Internet Explorer Enhanced Security Configuration in Microsoft Windows Server 2012
        • Manual Configuration of Adobe Acrobat Professional
        • Automatic Verification Database
        • Launching the Wrapper Generator Tool in High DPI Displays
      • Post-Installation Tasks: Scheduler
        • Change the Default Passwords
        • Setting-up Kerberos Authentication in Scheduler
      • Post-Installation Tasks: Web Container
        • Hide Name and Version in Error Pages
        • Secure the “Export” Endpoint of the Web Container
        • Enable Authentication on the Monitoring Interface
      • Upgrading from a Previous Version
    • Installing Updates and Hotfixes
      • Installing an Update or Hotfix Graphically
      • Installing an Update or Hotfix from the Command Line
    • Denodo Platform Control Center
      • Launching the Denodo Platform Control Center
      • Starting Platform Servers and Tools
      • Denodo Platform Configuration
        • Configuring the Connection to the License Manager
      • Configuration of the JVM Parameters from the Command Line
      • Control Center Help
      • Uninstalling the Denodo Platform
    • Appendix
      • Default Ports Used by the Denodo Platform Modules
      • Limitations of Denodo Express
      • Using Kerberos Authentication in Virtual DataPort Without Joining a Kerberos Realm
      • Using Kerberos Authentication in the Data Catalog Without Joining a Kerberos Realm
      • Using Kerberos Authentication in Scheduler Without Joining a Kerberos Realm
      • How to Debug Kerberos in Web Applications
        • Virtual DataPort Server and the Web Administration Tools are on the Same Installation
        • Virtual DataPort Server and the Web Administration Tools are not on the Same Installation
      • Providing a Krb5 File for Kerberos Authentication
      • Launching the Denodo Standalone Applications in High DPI Displays
      • Deploying the Web Administration Tool in an External Web Container
      • Troubleshooting the Denodo Platform Installer
      • Transparent Metadata Encryption
  • Denodo Platform Migration Guide
    • Preface
    • Outline of the Migration Process
      • Migrating from Denodo 5.5 or Earlier
    • Before Beginning the Migration
    • Export the Metadata of the Current Installation
    • Modify the Virtual DataPort Metadata
    • Import the Metadata to the New Installation
    • After Importing the Metadata into the New Installation
      • Migrate Salesforce Base Views to Use the New Data Source
    • Test the New Denodo Platform Installation
    • Backward Compatibility
      • Changes in the Denodo Platform 7.0
        • Changes Common to All the Modules of Denodo 7.0
        • Changes in Virtual DataPort 7.0
        • Changes in the Privileges System in Virtual DataPort 7.0
        • Features Removed in Virtual DataPort 7.0
        • Changes Introduced in Aracne 7.0
      • Changes in the Denodo Platform 6.0
        • Changes Common to All the Modules of Denodo 6.0
        • Changes in Virtual DataPort 6.0
        • Changes in Scheduler 6.0
        • Changes in the Embedded Web Container of Denodo 6.0
      • Changes in the Denodo Platform 5.5
        • Cache
        • CAST Function
        • Reserved Words
        • Prohibit Setting Incompatible Source Type Properties
        • Three-valued Logic
        • Features Removed in Denodo 5.5
      • Changes in the Denodo Platform 5.0
        • Case-Sensitivity of Elements
        • Dependencies Between Elements
        • Exporting to a Repository of Files
        • JMS Listeners
        • Web Services Published by Denodo
        • Stored Procedures
    • Features Deprecated in Virtual DataPort 7.0
      • Data Type “Date”
      • VQL Syntax
        • Contains, Containsand, Containsor and Iscontained Operators
        • SQLFRAGMENT Clause
        • Syntax ALTER TABLE <base view name> ADD SEARCHMETHOD
        • Parameters of the CONTEXT Clause of SELECT Statements
        • Deprecated Functions
      • Data Sources: Decrypt Filter
      • Aracne Data Sources
      • Google Search
      • Denodo Stored Procedures
      • Denodo Stored Procedures API: getNumOfAffectedRows Method
      • Denodo Custom Wrappers API: Deprecated Methods
      • Denodo JDBC Driver
      • Denodo Web Services
      • Widgets
      • Script Export
      • Data Catalog: Roles “selfserviceadmin”, “selfserviceexporter”
      • Launching Several Instances of a Virtual DataPort Server
      • Denodo4Eclipse Plugin
    • Features Deprecated in Scheduler 7.0
      • Aracne Data Sources
      • Filter Sequences
      • Jobs
      • Exporters
        • SQL Exporter
        • File-Repository Exporter
      • Move-File-Repository Handler
    • Features Deprecated in Solution Manager 7.0
      • REST API: Authentication Using Session
  • Denodo Platform New Features Guide
    • Preface
      • Scope
      • Audience
    • What Is New in Denodo 7.0 Update 20210224
    • What Is New in Denodo 7.0 Update 20201116
    • What Is New in Denodo 7.0 Update 20200803
    • What Is New in Denodo 7.0 Update 20200310
    • What Is New in Denodo 7.0 Update 20190903
    • 7.0 GA: New Features Common to All Modules
      • New Tool: Solution Manager
      • New Tool: Web Panel
      • Control Center
      • New Supported Platforms
      • Data Catalog
      • Diagnostic & Monitoring Tool
      • OData Service
      • Others
        • Kerberos Authentication
    • 7.0 GA: New Features of the Installer
      • Other Changes
    • 7.0 GA: New Features of Virtual DataPort
      • Data Sources
        • Salesforce.com Data Source
        • JDBC Data Sources
        • DF Data Sources
        • Custom Data Sources
        • HTTP Requests of Data Sources
      • Execution Engine
        • New Data Types
        • Massive Parallel Processing
        • Query Optimizer
        • Cost-Based Optimizer
        • Other Improvements
      • Cache Engine
      • Denodo Stored Procedures
      • Web Services
      • Administration Tool
      • JDBC Driver
      • Administration
      • Privileges
      • Monitoring
      • Denodo4E Eclipse Plugin
      • Others
    • 7.0 GA: New Features of Scheduler
      • VDPIndexer and VDPCache Jobs
      • Import/Export
      • Authentication
      • Deprecated Elements
    • 7.0 GA: New Features of the Embedded Web Container
      • New Version of the Web Container: Apache Tomcat 8.5
      • Allow URIs Slash and Backslash by Default
      • Others
  • Virtual DataPort Administration Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Introduction
    • General Architecture
      • Physical Layer
      • Logical Layer
        • Data Module: Cache
      • User Layer
    • Installation and Execution
      • Launching the Virtual DataPort Server
      • Launching the Virtual DataPort Administration Tool
        • Main Areas of the Administration Tool
        • Tool Preferences
        • Using the VQL Shell
        • Query Monitor
        • Catalog Search
        • Invalidate Cache
        • Trace Viewer
    • Creating Data Sources and Base Views
      • JDBC Sources
        • Importing JDBC Sources
        • Database Specific Information
        • Connecting to a JDBC Source with Kerberos Authentication
        • Creating Base Views from a JDBC Data Source
        • Importing Graphically Stored Procedures from a Database
        • Creating Base Views from SQL Queries
      • ODBC Sources
        • Creating Base Views from an ODBC Data Source
      • SOAP Web Service Sources
        • Importing SOAP Web Service Sources
        • Creating Base Views from a SOAP Web Service
      • Multidimensional Database Sources
        • Creating a Base View Over a Multidimensional Data Source, Graphically
        • Creating a Base View Over a Multidimensional Data Source with an MDX Query
        • Multidimensional Data Sources to Oracle Essbase
      • Path Types in Virtual DataPort
        • Local Path
        • Path From Variable
        • HTTP Path
        • Denodo Browser Path
        • FTP / SFTP / FTPS Path
        • Compressed or Encrypted Data Sources
        • Custom Input Filters
        • Paths and Other Values with Interpolation Variables
      • XML Sources
      • JSON Sources
      • Delimited File Sources
        • Examples of How to Define a Tuple Pattern
        • Paths Using Date Ranges
      • Excel Sources
      • Web Sources (WWW)
      • Google Search Sources
      • Aracne Sources
      • LDAP Sources
      • BAPI Sources
      • Salesforce Sources
        • Registering Denodo as a Connected Application
        • Creating a Salesforce Data Source
        • Creating Base Views from a Salesforce Data Source
        • Creating Base Views from SOQL Queries
        • Binary Fields
        • Limitations of Salesforce
        • Additional Settings
      • Custom Sources
        • Using Interpolation Variables as Custom Sources’ Input Parameters
      • Viewing the Schema of a Base View
        • View the VQL of an Element
        • Used By…
        • Source Type Properties
      • Data Source Configuration Properties
        • ORDER BY Properties of the Source Configuration
      • Source Refresh
      • Working with Blob Fields of Base Views
    • Creating Derived Views
      • Creating Union Views
        • Creating Standard Union Views
        • Creating Partitioned Union Views
      • Creating Join Views
        • Join Conditions with Similarity Operators
      • Creating Selection Views
        • Creating Conditions with the Compound Values Editor
        • Parameters of Derived Views
      • Creating Flatten Views
      • Creating Intersection Views
      • Creating Minus Views
      • Creating Interface Views
      • Viewing the Schema of a Derived View
      • Tree View
      • Data Lineage
      • Advanced Configuration of Views
        • Configuring the Cache of a View
        • Memory Usage and Swapping Policy of Views
        • Internationalization Configuration
        • Query Capabilities
        • View Configuration Properties
        • Indexes of Views
        • Statistics of the View
        • Editing the Execution Plan
        • Defining the Data Movements of the View
      • Materialized Tables
      • Temporary Tables
      • Querying Views
        • Execution Trace of a Statement
      • Editing / Replacing a View
        • Modifying Base Views
        • Modifying Derived Views
        • Views Affected by Modifications
    • Stored Procedures
      • Importing Stored Procedures
      • Executing Stored Procedures
      • Use of Stored Procedures in Creating Views
    • Publication of Web Services
      • Publishing SOAP Web Services
        • Operations Tab
        • Settings Tab (SOAP)
        • Advanced Tab (SOAP)
      • Publishing REST Web Services
        • Resources Tab
        • Settings Tab (REST)
        • Advanced Tab (REST)
      • XSLT Transformations
      • Web Services Authentication
        • HTTP Basic
        • HTTP Basic with VDP
        • HTTP Digest
        • HTTP SPNEGO (Kerberos)
        • OAuth 2.0
        • SAML 2.0
        • Impersonating a User
      • How Web Services Query the Virtual DataPort Server
        • Connection from the Web Services to the Server
      • Web Service Container Status Table
      • Invoking Denodo Web Services
        • Invoking Web Services with SAML Authentication
        • Types Conversion Table for REST / SOAP Published Web Services
      • Privileges Required
    • Publication of Views as Widgets
      • Publish a View as a Widget
      • Auxiliary Web Services
      • Export to JSR-168 or JSR-286 Portlet
      • Export as Microsoft Web Part
      • Deployment of a Microsoft Web Part
    • RESTful Architecture
      • Primary Keys of Views
      • Associations
        • Creating an Association
        • Multiplicity of Associations
        • Referential Integrity in Associations
        • Role Preconditions
        • Why You Should Define Associations Between Views
      • RESTful Web Service
        • Representations
        • Input Parameters of the RESTful Web Service
        • IDU Requests
        • Requests with Input Compound Values
        • Status Codes
        • Obtaining the Number of Rows of a Result Set
        • Security and Access Privileges
        • Configuring the RESTful Web Service
      • Denodo OData 4.0 Service
        • Serving Metadata
        • Querying Data: The Basics
        • Navigating Associations
        • Advanced Querying
        • Pagination
        • Configuration
        • Entity Collection Streaming
        • Debug Option
        • Privilege Requirements
        • Limitations
        • Features
    • JMS Listeners
      • Creating a New JMS Listener
      • Privileges Required
      • Advanced JMS Configuration
        • Acknowledge Mode
        • Acknowledge On Query Finish and Serialized Processing
      • Enabling Advanced Logging for JMS Listeners
    • Remote Tables
      • Managing Remote Tables
        • Creating Remote Tables
        • Editing Remote Tables
        • Refresh Data
        • Dropping Remote Tables
    • Server Administration - Configuring the Server
      • Server Connectivity
      • Threads Pool
      • Configuring the Cache
      • Limiting the Number of Concurrent Requests
      • Default Configuration of HTTP Proxy
      • Configuring the Memory Usage and Swapping Policy
        • Memory Usage Settings
        • Swapping Settings
        • Maximum Size of a Query
      • Queries Optimization
      • Configuring Runtime Parameters for Stored Procedures
      • Configuring the Default Internationalization
      • Identifiers Charset
        • Restricted Mode
        • Unicode Mode
      • Kerberos Authentication
        • Creating the Roles of the Virtual DataPort Users
        • Creating an LDAP Data Source
        • Setting-Up the Kerberos Authentication in the Virtual DataPort Server
        • Configuring the Administration Tool to Use Kerberos Authentication
      • OAuth Authentication
        • Setting-Up the OAuth Authentication in the Virtual DataPort Server
        • Creating the Roles of the Virtual DataPort Users
      • SAML Authentication
        • Validation of the Response Signature
      • Privileges
        • The FILE Privilege
    • Memory Management
      • Streaming Vs Non-Streaming Operators
      • Swapping Parameters
      • Limit the Maximum Amount of Memory of a Query
      • Edge Cases in Streaming Operation
      • Cache Load Processes
    • Resource Manager
      • Defining a Plan
      • Defining a Rule
    • Exporting and Importing the Server Metadata
      • Exporting the Server Metadata
      • Exporting and Importing Elements Across Different Environments
        • Exporting Environment-Dependent and Independent Elements to Different Files
        • Export to a File with Properties
      • Importing Metadata into a Server
    • Importing Extensions
    • Cache Module
      • Cache Modes
        • Partial Mode
        • Full Mode
        • Incremental Mode
        • Recommended Parameters for Queries that Load the Cache
        • Caching Very Large Data Sets
        • Caching the Result of Queries that Fail
      • Cache Maintenance Task
        • Maintaining the Underlying Database
      • Specific Information about Cache Databases
        • Amazon Athena
        • Azure SQL Data Warehouse
        • Presto
        • Spark
        • Yellowbrick
        • Databases with HDFS Storage
      • Generic Support for Other Databases
    • Version Control Systems Integration
      • VCS Configuration
        • Virtual DataPort Server Configuration
        • Environment Management
        • Database Configuration
        • Importing an Existing Database from a VCS Server
      • VCS Integration Features
        • Supported Element Types
        • Integration with the Server Explorer
        • VCS Operations for Microsoft TFS and Subversion
        • VCS Operations for GIT
        • Delete Operations
        • Other Version Control Operations
        • Dependencies Management
        • VCS Settings of the Administration Tool
      • Scenarios and Recommended Uses
        • Picking a VCS Workflow
        • Centralized Workflow with Shared Databases
        • Distributed Workflow
        • Centralized Workflow with Private Databases
        • Best Practices to Manage Global Elements
        • Promoting Changes from Development to Testing and Production
        • Using VCS Environments
      • Best Practices When Using the Integration with a VCS
        • Tagging and Branching Releases
        • Recommendations for the Testing Environment
      • Uniqueness Detection
        • Enabling Uniqueness Detection
        • Examples
      • LS Optimization for Subversion
        • Activating the LS Optimization
        • Tested Environments
    • Databases, Users and Access Rights in Virtual DataPort
      • Databases in Virtual DataPort
      • User and Access Right in Virtual DataPort
        • Types of Access Rights
        • Column Privileges
        • Row Restrictions
        • Enforcing Column Privileges, Row Restrictions and Custom Policies
        • Roles
      • Administration of Databases, Users, Roles and Their Access Rights
        • Creating Databases
        • Configuring and Deleting Databases
        • Creating Users
        • Modifying and Deleting Users
        • Creating Roles
    • Monitoring the Virtual DataPort Server
      • Monitoring with a Java Management Extensions (JMX) Agent
        • Using JavaTM VisualVM
        • General Information on the Server
        • Information on Data Sources
        • Information on the Cache
        • Information and Events on Catalog Access (DDL Statements)
        • Information and Events on the Running of Statements
        • Information and Events on Transactions
        • Setting the User Agent of an Application
      • Denodo Monitor
        • Local Monitors
        • Server Monitors
        • Virtual DataPort Monitors
        • Configuring the Denodo Monitor
        • Launching the Denodo Monitor
    • Cluster Architectures / Server Backup
      • Denodo Tools
      • How to Check If a Virtual DataPort Server Is Alive
        • Connecting from a JDBC Client Through a Load Balancer
        • Using the Ping Script
        • Alternatives to the Ping Script
      • Using the Import/Export Scripts for Backup And/or Replication
        • Export Script
        • Import Script
      • Launching Several Virtual DataPort Instances of the Same Installation
        • How to Configure the Secondary Instances
        • Deploying Web Services on This Configuration
    • Bulk Data Load
      • Amazon Athena
      • Amazon Redshift
      • DB2
      • Databricks
        • Mounting External File Systems on the DBFS
      • Hive
      • Impala
      • MySQL
      • Oracle
      • PostgreSQL
      • Presto
      • Snowflake
      • Spark
      • Microsoft SQL Server
      • Teradata
      • Yellowbrick
      • Settings of the Generation of the Temporary Files
        • Databases with HDFS Storage
    • Optimizing Queries
      • Optimizing DF Data Sources
      • Optimizing Join Operations
        • Merge Join
        • Nested Join
        • Nested Parallel Join
        • Hash Join
      • Automatic Simplification of Queries
        • Removing Redundant Branches of Queries (Partitioned Unions)
        • Pushing Down GROUP BY Views Below JOIN Views
        • Pushing Down GROUP BY Views Below UNION Views
        • Selecting the Most Optimal Source When the Data Is Replicated in Several Sources
      • Cost-Based Optimization
        • Enabling the Cost-Based Optimization
        • Gathering the Statistics of Views
        • Tuning the Cost-Based Optimization Process
        • Current Limitations of the Cost-Based Optimization Process
      • Data Movement
        • Data Movements From/To Netezza Databases
        • Examples of Data Movement
        • Known Limitations
        • Options of the CONTEXT Clause that Control a Data Movement
      • Parallel Processing
        • Use of an MPP Engine as Cache
        • Force Movement of Views to the MPP Engine Using the CONTEXT Clause
    • Appendix
      • Supported JDBC Data Sources
      • Backward Compatibility Between the Virtual DataPort Server and Its Clients
      • Mapping Multidimensional Data to a Relational Model
        • Creating a Multidimensional Base Views Over a Multidimensional Data Source
        • Mapping the Result of MDX Queries
      • Considerations When Configuring Data Sources with Pass-Through Credentials
        • Web Services
      • Configuring the Logging System
      • Configuring the Network Interface Restriction in Virtual DataPort
      • Installing the Denodo Solution for Microsoft SharePoint
      • Customizing Events and Public Render Parameters of JSR-286 Portlets
      • Transforming Incoming/Outgoing Soap/Rest Messages with XSLT Stylesheets
      • JMS Connection Details: JNDI Properties
        • Apache ActiveMQ 5.6.0
        • IBM WebSphere MQ 7.0
        • Progress SonicMQ 8.0
      • Execution Trace Information
      • Web Services Created with Previous Versions of the Denodo Platform
        • Publishing Web Services
        • XSLT Transformations of Web Services Created with Previous Versions
        • SOAP Over JMS on Web Services Created with Previous Versions of the Denodo Platform
        • Authentication in Web Services Created with Previous Versions
        • Types Conversion Table for REST / SOAP Web Services Created with Previous Versions
        • Invoking Web Services Created with Previous Versions
      • Useful Tools to Debug Issues with Active Directory or Other LDAP Servers
      • Single User Mode
        • Explicit Single User Mode
        • Automatic Single User Mode
      • Resource Manager: Available Fields to Evaluate a Rule
        • Examples of Resource Manager Rules
      • Values of the Attribute “Access Interface”
      • Avoiding SQL Injections
      • Enabling the Support for ODBC Sources When an External JRE is Used
  • Virtual DataPort Developer Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Introduction
      • Examples
    • Access Through JDBC
      • JDBC Driver Parameters
      • Connecting to Virtual DataPort Through a Load Balancer
      • Connecting to Virtual DataPort Using Kerberos Authentication
        • When the Client Application Does Not Belong to the Domain
      • Details of the JDBC Interface
        • Description of Views and Their Fields
        • Retrieving the Content Type of Blob Values
        • Working with Datetime Values with the Denodo JDBC Driver
        • Obtaining the Names of Elements Inside a Struct (Register)
        • Connecting from Applications that Filter Incoming Serialization Data
    • Access Through ODBC
      • Configuration of the ODBC Driver on Windows
        • Install the ODBC Driver on Windows
        • Set Up a DSN on Windows
      • Configuration of the ODBC Driver in Linux and Other UNIX
        • Obtain the Appropriate ODBC Driver
        • Install UnixODBC
        • Register the Denodo ODBC Driver in UnixODBC
        • Register a Data Source (DSN) on UnixODBC
        • Compiling UnixODBC
        • Troubleshooting Issues
      • Creating a DSN-Less Connection
      • Details of the ODBC Interface
        • How the ODBC Interface Reports the Datetime and Interval Data Types
      • Integration with Third-Party Applications
        • Supporting Queries with Brackets
        • Maximum Length of Text Values
        • Maximum Length of Error Messages
        • Disabling Transactions
      • Backward Compatibility of the ODBC Driver
    • Access Through an ADO.NET Data Provider
      • Using Kerberos Authentication
    • Access Through OLE DB
    • Developing Extensions
      • Developing Custom Functions
        • Creating Custom Functions with Annotations
        • Creating Custom Functions Using Name Conventions
        • Compound Types
        • Custom Function Return Type
        • Getting Information About the Context of the Query
        • Dealing with Datetime and Interval Types
      • Developing Stored Procedures
        • Using Datetime Values in Denodo Stored Procedures
        • Required Libraries to Develop Stored Procedures
      • Developing Custom Wrappers
        • Required Libraries to Develop Custom Wrappers
        • Extending AbstractCustomWrapper
        • Overriding AbstractCustomWrapper
        • Dealing with Conditions
        • Dealing with the ORDER BY Clause
        • Configuring a Custom Wrapper
        • Updating the Custom Wrapper Plan
        • Dealing with Datetime and Interval Types
      • Developing Custom Input Filters
        • Required Libraries to Develop Custom Filters
        • Developing Custom Filters
    • Custom Policies
      • Developing a Custom Policy
    • Appendix
      • Output Schema of the LIST Command
      • Output Schema of the DESC Commands
      • Error Codes Returned by Virtual DataPort
      • Compile the Denodo ODBC Driver
        • Pre-requisites to Compile the ODBC Driver on Linux
        • Compile the ODBC Driver (Standard Method)
        • Compile the ODBC Driver to Obtain Forwardable Tickets
        • Troubleshooting the Compilation of the Denodo ODBC Driver
  • Virtual DataPort VQL Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Introduction
    • General Overview of Virtual DataPort
      • Creating or Defining Data
        • Defining Base Views
        • Defining Data Sources and Wrappers
        • Defining the Views of the Global Schema
      • Executing Statements
    • Language for Defining and Processing Data: VQL
      • Data Types
        • Data Types for Dates, Timestamps and Intervals
        • Internationalization
      • Statements
      • Unicode Identifiers
      • Logical Operators
      • Comparison Operators
        • Comparing Literals
        • Three-valued Logic
      • Functions for Conditions and Derived Attributes
      • Syntax Conventions
        • Syntax of Functions and Condition Values
      • Comments
    • Generating Wrappers and Data Sources
      • Creating Data Sources
        • JDBC Data Sources
        • ODBC Data Sources
        • Multidimensional Data Sources
        • Data Sources for SOAP Web Services
        • XML Data Sources
        • JSON Sources
        • DF Data Sources
        • Denodo Aracne Data Sources
        • Google Search Data Sources
        • LDAP Data Sources
        • BAPI Data Sources
        • Salesforce Data Sources
        • Custom Data Sources
        • Data Source Configuration Properties
        • Specifying Paths in Virtual DataPort
      • Creating Wrappers
        • Execution Context and Interpolation Strings
        • Wrapper Metadata
        • JDBC Wrappers
        • Multidimensional Databases Wrappers
        • ODBC Wrappers
        • WWW Wrappers
        • Web Services Wrappers
        • XML Wrappers
        • JSON Wrappers
        • DF Wrappers
        • Denodo Aracne Wrappers
        • Google Search Wrappers
        • LDAP Wrappers
        • BAPI Wrappers
        • Salesforce Wrappers
        • CUSTOM Wrappers
        • Wrapper Configuration Properties
      • Valid Conversions Between Types in Wrappers and VDP Types
        • Native-type Conversions of a Wrapper to Java Types
        • Filters
      • QUERY WRAPPER Statements
    • Creating a Base View
      • Query Capabilities: Search Methods and Wrappers
        • Query Constraints
        • Assigning Wrappers to Search Methods
        • Example of How a Search Method Is Created
      • Modifying a Base View
    • Queries: SELECT Statement
      • FROM Clause
        • JOIN Operation
        • INTERSECT Operation
        • MINUS Operation
        • FLATTEN View (Flattening Data Structures)
        • Subqueries in the WHERE Clause of the Query
      • WITH Clause
      • SELECT Clause
        • Derived Attributes
      • WHERE Clause
        • Conditions with Compound Values
      • Group BY Clause
        • Use of Aggregation Functions
      • HAVING Clause
      • UNION Clause
        • Enabling the Standard SQL Union
      • ORDER BY Clause
      • OFFSET, FETCH and LIMIT
      • CONTEXT Clause
      • TRACE Clause
      • CASE Clause
    • Defining a Derived View
      • Modifying a Derived View
      • Defining an Interface View
    • Defining the Statistics of a View
    • Materialized Tables
      • Creating Materialized Tables
        • Inserting Data into Materialized Tables
    • Temporary Tables
      • Creating Temporary Tables
        • Privileges Required
    • Remote Tables
      • CREATE REMOTE TABLE Command
    • Inserts, Updates and Deletes Over Views
      • INSERT Statement
      • UPDATE Statement
      • DELETE Statement
      • Return Modified Rows
        • Limitations
      • Use of WITH CHECK OPTION
    • Refreshing Views
      • REFRESH Statement
    • Transactions in Virtual DataPort
    • Stored Procedures
      • Importing a Stored Procedure
      • Use of Stored Procedures
      • Predefined Stored Procedures
        • CACHE_CONTENT
        • CATALOG_ELEMENTS
        • CATALOG_FKS
        • CATALOG_METADATA_VIEWS
        • CATALOG_PERMISSIONS
        • CATALOG_PKS
        • CATALOG_VDP_METADATA_VIEWS
        • CATALOG_VIEWS
        • CLEAN_CACHE_DATABASE
        • COLUMN_DEPENDENCIES
        • COMPACT_HADOOP_CACHE
        • COMPACT_METADATA_TABLES
        • CREATE_REMOTE_TABLE
        • DROP_REMOTE_TABLE
        • DUAL
        • GENERATE_SMART_STATS_FOR_FIELDS
        • GENERATE_SMART_STATS_FOR_FIELDS_BY_TABLENAME
        • GENERATE_STATS
        • GENERATE_STATS_FOR_FIELDS
        • GENERATE_VQL_TO_CREATE_JDBC_BASE_VIEW
        • GET_ACTIVE_LOGGERS
        • GET_ASSOCIATIONS
        • GET_CACHE_COLUMNS
        • GET_CACHE_CONFIGURATION
        • GET_CACHE_TABLE
        • GET_CATALOG_EFFECTIVE_PERMISSIONS
        • GET_CATALOG_METADATA_WS
        • GET_DELEGATED_SQLSENTENCE
        • GET_ELEMENTS
        • GET_EXPORTED_KEYS
        • GET_FOREIGN_KEYS
        • GET_JDBC_DATASOURCE_TABLES
        • GET_PRIMARY_KEYS
        • GET_PROCEDURE_COLUMNS
        • GET_SERVER_CONNECTIVITY
        • GET_SESSIONS
        • GET_SOURCE_CHANGES
        • GET_SOURCE_COLUMNS
        • GET_SOURCE_TABLE
        • GET_TYPE_ATTRIBUTES
        • GET_USER_DEFINED_TYPES
        • GET_VIEW_COLUMNS
        • GET_VIEW_INDEXES
        • GET_VIEW_STATISTICS
        • GET_VIEWS
        • LIST_JDBC_DATASOURCE_TABLES
        • LOGCONTROLLER
        • MIGRATE_DATE_TYPES
        • PING_DATA_SOURCE
        • SOURCE_CHANGES
        • VIEW_DEPENDENCIES
        • WAIT
        • WEBCONTAINER_ELEMENTS
        • WEBCONTAINER_ELEMENT_STATUS
        • WRITELOGERROR
        • WRITELOGINFO
    • Defining Other Elements of the Catalog
      • Creating Folders
      • Defining a Data Type
      • Defining a Map
      • Defining JAR Extensions
      • Defining JMS Listeners
    • Creating Databases, Users, Roles and Access Privileges
      • Creating and Modifying Virtual DataPort Databases
      • Managing Users
        • Modifying and Deleting Users
        • Connecting to Another Database
        • Modifying the Privileges of a User
        • Managing User Roles
      • Ownership of Elements
    • Describing Catalog Elements
      • Exporting Metadata
      • Importing Metadata
    • Listing Elements in the Catalog
    • Removing Elements from the Catalog
    • Publication of Web Services
      • Creating SOAP Web Services
      • Modifying SOAP Web Services
      • Creating REST Web Services
      • Modifying REST Web Services
      • Connection Parameters
      • Web Services Authentication
        • Basic and Digest
        • OAuth 2.0 and OpenID
        • SAML
        • VDP
        • WSS
      • Embedded Web Container Management
      • Deploying and Exporting SOAP and REST Web Services
    • Publication of Widgets
      • Create New Widgets
      • Export a Widget
      • Deployment and Export of Auxiliary Web Services
    • RESTful Architecture
      • Associations
      • Navigational Queries
    • Help Command
    • Version Control Systems Integration Commands
      • Statements to Work with Centralized Version Control Systems
        • VCSCOMMIT
        • VCSUPDATE
        • VCSCONTENTS
        • VCSSHOW
        • REVISIONS
      • Statements to Work with GIT
        • DVCSCOMMIT
        • DVCSPUSH
        • DVCSPULL
        • DVCSREVERT
      • Commands for Environment Management
      • Other Version Control Commands
        • VCSDISCARD
    • Resource Manager
      • Managing the Plans of the Resource Manager
      • Managing the Rules of the Resource Manager
    • Advanced Characteristics
      • Management of Compound Values
        • Processing of Compound Types: Example
      • Changing Settings of Virtual DataPort and the Web Container
      • Dynamic Choice of Join Strategy
      • Using the Cache
        • Cache Invalidation
        • Displaying the Contents of the Cache
      • Configuring Swapping Policies
      • Managing Internationalization Configurations
        • Replacing an I18n Map
        • Deleting an I18n Map
      • Execution Context of a Query and Interpolation Strings
      • Adding Variables to Selection Conditions (GETVAR and SETVAR)
    • Appendix
      • Syntax of Condition Functions
        • Arithmetic Functions
        • Text Processing Functions
        • Datetime Processing Functions
        • XML Processing Functions
        • Type Conversion Functions
        • Aggregation Functions
        • Analytic Functions (Window Functions)
        • Other Functions
      • Syntax of Search Expressions for the Contains Operator
        • Exact Terms and Phrases
        • Term Modifiers
        • Boolean Operators
        • Groups
        • Escaping Special Characters
      • Support for the Contains Operator of Each Source Type
      • CASE Clause Examples
      • Date and Time Pattern Strings
      • Web Services Created with Previous Versions of the Denodo Platform
  • Data Catalog Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Introduction
    • Installation and Execution
      • Launching the Data Catalog
    • Authentication and Authorization
    • Administration
      • Servers Configuration
        • Configure the Virtual DataPort Servers
        • Configure the Connection Settings to the Virtual DataPort Servers
        • Configure the Internal Database Used by the Data Catalog
      • Content Search Configuration
      • Kerberos Configuration
      • Personalization
        • Elements Shown to the Users
        • Browse Configuration
        • User Interface
        • Export Format Configuration
        • Database, View and Web Service Connection URIs
        • Usage Statistics
      • Catalog Management
        • Categories Configuration
        • Tags Configuration
        • Databases Configuration
        • Elements Configuration
        • Property Groups Configuration
      • Import and Export Data Catalog Metadata
        • Import and Export Data Catalog Metadata from the UI
        • Import and Export Data Catalog Metadata Using a Script
    • Configuration
    • Browse
      • Databases
      • Categories
      • Tags
      • Relationships
      • Views
        • Summary of Views
        • Query of Views
        • Associations of Views
        • Lineage of Views
        • Search of Views
        • Usage of Views
      • Web Services
        • Summary of Web Services
        • Query a Web Service
    • Saved Queries
      • Deploying Saved Queries
      • Copying Saved Queries
    • Search
      • Catalog Search
      • Content Search
      • Queries
    • Appendix
      • Installing the Data Catalog on a Cluster: Sharing the Same Settings Across All the Nodes
        • Setting Up the Common Database
        • Configure the Data Catalog to Use the Common Database
      • REST API
        • Synchronize Elements with the Server
        • Compute the Usage Statistics
        • Export Metadata
        • Import Metadata
      • Apache Lucene Search Syntax
        • Terms
        • Fields
        • Term Modifiers
        • Boolean Operators
        • Grouping
        • Field Grouping
        • Escaping Special Characters
  • Diagnostic & Monitoring Tool Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Introduction
    • Installation and Execution
      • Launching the Monitoring and Diagnostic Tool
    • Architecture
    • Authentication and Authorization
      • Kerberos Authentication
    • Creating Servers and Environments
      • Creating Servers
      • Creating Environments
    • Monitoring
      • Monitoring Servers
        • Monitoring - Resources
        • Monitoring - State
        • Monitoring - Sessions
        • Monitoring - Requests
        • Monitoring - Cache
        • Monitoring - Data Sources
        • Monitoring - Threads
      • Monitoring Environments
      • Overview
    • Creating Diagnostics and Diagnostic Intervals
      • Creating Diagnostics
      • Creating Diagnostic Intervals
    • Diagnosing
      • Diagnosing - Resources
      • Diagnosing - State
      • Diagnosing - Sessions
      • Diagnosing - Requests
      • Diagnosing - Cache
      • Diagnosing - Data Sources
      • Diagnosing - Threads
      • Diagnosing - Errors
    • Export and Import
      • Export
      • Import
    • Configuration
      • User Configuration
        • Configuration - General
        • Configuration - Overview
        • Configuration - Resources
        • Configuration - State
        • Configuration - Sessions
        • Configuration - Requests
        • Configuration - Cache
        • Configuration - Data Sources
        • Configuration - Threads
        • Configuration - Errors
      • Server Configuration
        • Informative Message
  • Virtual DataPort API Javadoc
  • Scheduler Administration Guide
    • Preface
      • Scope
      • Who Should Use This Manual
      • Summary of Contents
    • Introduction
    • General Architecture
    • Installation and Execution
    • Administration
      • Authentication
      • Web Configuration
        • User Configuration
        • Kerberos Configuration
      • Scheduler Server Configuration
        • Server Configuration Sections
        • Permissions
        • Plugins and JDBC Adapters
      • Log Configuration
    • Creating and Scheduling Jobs
      • Jobs
      • Data Sources
        • ARN Data Sources
        • Scheduler Index Data Sources
        • CSV Data Sources
        • Elasticsearch Sources
        • ITP Data Sources
        • JDBC Data Sources
        • VDP Data Sources
      • Filter Sequences
      • Configuring New Jobs
        • General Structure of a Job
        • Aracne-type Job Extraction Section
        • VDP Extraction Section
        • ITP Extraction Section
        • JDBC Extraction Section
        • VDPCache Extraction Section
        • VDPIndexer Extraction Section
        • Data Schema Generated by the Different Types of Extraction Jobs
        • Jobs for Maintaining Aracne Indexes
        • Postprocessing Section (Exporters)
        • Retry Section
        • Handler Section
        • Time-based Job Scheduling Section
        • Dependencies Among Jobs
    • Developer API
      • Scheduler Client API
      • Extensions (Plugins)
        • Filters
        • Exporters
        • Handlers
        • Aracne Custom Crawlers
    • Appendix
      • Dateformat Function Syntax
      • Regular Expressions for Filters
      • JDBC Drivers
      • Use of the Import/Export Scripts for Backup (Scheduler)
        • Export
        • Import
      • Use of the Import/Export Scripts for Backup (Scheduler Index)
        • Export
        • Import
      • Use of the Ping Script
      • ChangePassword Script (Scheduler)
      • ChangePassword Script (Scheduler Index)
  • Scheduler API Javadoc
  • ITPilot User Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Introduction
      • Denodo ITPilot Environments
        • Administration Tool
        • Generation Environment
        • Execution Environment
        • Verification Environment
    • Distribution of Environments
      • Distribution of the Generation Environment
      • Distribution of the Execution Environment
      • Distribution of the Verification Environment
    • Installation and Initial Configuration
    • Execution
      • Starting Up the Administration Tool
      • Starting Up the Browser Pool
      • Starting Up the Wrapper Server
      • Starting Up the Verification Server
      • Starting Up the Verification Server with a Graphical Tool
      • Starting Up the PDF Conversion Server
    • Web Administration Tool
      • Adding, Editing and Deleting Servers
      • Configuring the Browser Pool Server
        • Global Configuration Parameters
        • MSIE and Denodo Browser Configuration
      • Configuration of the Wrapper Server
        • List of Wrappers
        • Server Configuration (Wrapper Server)
        • Embedded Denodo Browser Configuration
        • Extensions
      • Configuring the Verification Server
        • Server Configuration (Verification Server)
        • Database Configuration
        • Mail Configuration
        • Rules Configuration
    • Verification Server Graphical Configuration and Monitoring Tool
      • An Overview of the Graphical Tool
      • Configuration of the Verification Environment
      • Monitoring the Wrappers Under Verification
    • Wrapper Execution Trace Logs
      • Enabling Wrapper Trace Logging
      • Log Categories and Effective Log Level
      • Appenders
        • Configuring Custom Appenders
        • Custom Appenders and Logger Additivity
  • ITPilot Generation Environment Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Introduction
      • Presentation
      • Development of Component-Based Wrappers
    • Installation and Execution
      • Introduction to the Tools
    • Generation Environment Tools - Part I
      • Presentation of the Example
      • Creating Projects and Wrappers
      • Components in ITPilot
        • Input and Output Parameters
      • Process Initialization
        • Use of the Catalog Explorer
      • Web Browsing Automation
        • Component Configuration
        • Comparison Between MSIE and Denodo Browser
        • Advanced Configuration of the Sequence and Next Interval Iterator Components
        • Error Processing on the Web Browsing Automation
      • Configuration of the Extractor Component
        • Defining the Structure of the Data and Assigning Examples
        • Generating a DEXTL Specification from the Examples
        • Testing the Generated Specification
        • Locale Configuration of the Extractor Component
        • Error Processing on the Extractor Component
      • Processing the Retrieved Results
        • Use of the Iterator Component
        • Processing the Individual Records: Use of Record Constructor
        • Returning Results: Output Component
      • Wrapper Generation, Tests and Exporting
        • Wrapper Generation
        • Wrapper Execution
        • Deploying Wrappers to the Wrapper Server
    • Generation Environment Tools - Part II
      • Extracting Multipaginated Data
        • Pagination Using Single Block or Multiple Block Schemes
      • Access to Details Pages
        • Access to the Details Page: Using Extractor Sequence
        • Individual Test of Extractor Sequence, Next Interval Iterator and Form Iterator Components
        • Specifying the Structure of the Detail Page Information
        • Generating the Access Specification to the Details Page
        • Iteration on the Details Page Structures and Creation of the Output Record
      • Generation Tool Global Preferences
      • Tagsets and Scanners
        • Understanding Tagsets
        • Graphical Creation of New Tagsets
        • Tagsets and Scanners Included in the Distribution
        • Lexer Types
      • Generating the Data Extraction Specifications Manually
        • Creating a Structure Using the Structure Pane
        • Obtaining the Tokens of the Data to Extract
        • Creating a DEXTL Specification from the Tags
      • Using the Sequence Debugger
        • Breakpoint Management
      • Saving a Flow as a Custom Component
        • Deploying a Custom Component to the Wrapper Server
      • Wrapper Advanced Options: Specific Browser Pool and Locale
        • Specific Browser Pool
        • Locale
      • Migrating Wrappers Between Generation Environments: Import and Export
        • Exporting Wrappers
        • Importing Wrappers
      • Managing Extensions
        • Adding Extensions
        • Deleting Extensions
        • Exporting a Flow as an Extension
    • Generating Navigation Sequences
      • Description of the Navigation Sequences Generator Interface
        • Toolbar State Info Dialog
      • Steps for Generating a Navigation Sequence
        • Checking Navigation Sequences in Systems with Cookie-Based Session Authentication and Maintenance
      • The SelectFrame Action
      • The Transpose Table Button and Action
      • The Cancel Navigation Button
      • Recording Sequences with Passwords
      • Properties of the Navigation Bar
        • Generating Sequences Using an Authenticated Proxy
        • Criteria for Selecting NSEQL Commands
        • Advanced Options
      • XPath Generator Utility
    • Appendix A: ITPilot Operators
      • Logical Operators
      • Comparison Operators
    • Appendix B: ITPilot Functions
      • Arithmetic Functions
      • Text Processing Functions
      • Functions for Handling Null Values
      • Functions for Handling Lists
      • Date Processing Functions
      • Type Conversion Functions
      • Functions for URL Processing
      • Functions for Page Handling
      • Functions for Handling Compressed Files
      • Extraction Functions
      • Case Clause
    • Appendix C: Catalog of Components
      • Add Record To List
      • Condition
      • Using the Conditions Editor
      • Create List
      • Create Persistent Browser
      • Diff
      • Execute JavaScript
      • Expression
        • Using the Expressions Editor
      • Extractor
      • Extractor Sequence
      • Filter
      • Form Iterator
      • Get Page
      • INIT
      • Iterator
      • JDBC Extractor
      • Loop
      • Next Interval Iterator
      • Output
      • Record Constructor
      • Release Persistent Browser
      • Repeat
      • SaveFile
      • Script
      • Sequence
    • Appendix D: Constraints of the Simplified DOM
    • Appendix E: List of Reserved Words
  • ITPilot Developer Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Introduction
    • ITPilot Development API
    • Connecting to the Server
      • Obtaining Wrappers
      • Using Wrappers
      • Processing Query Results
        • Canceling Queries
      • Example of Use
    • Creating Custom ITPilot Functions
      • Naming Conventions and Annotations
      • Compound Types
      • Page Type
      • Custom Function Return Type
      • Example
  • ITPilot DEXTL Guide
    • Preface
      • Scope
      • Who Should Use This Manual
      • Summary of Contents
    • Introduction
    • An Overview of DEXTL
    • Basic Syntax
      • Fundamentals of Pattern Specification
      • Ambiguity of Patterns and Demarcation of the Search Space
      • Relations and Subrelations
    • Advanced Syntax
      • Extracting Tag Attributes
      • Tags with Evaluated Attributes
      • Voracious Separators
      • Dynamic Separators
      • Attributes with FIXED Value and Default Value
      • Format Tags
        • Tagsets
      • Hierarchy of Elements
      • Alternative Specifications
      • Scanners
        • Lexer Types
      • Structure of a DEXTL Program
        • Declarations Section
        • Specification Section
      • Escape Characters in DEXTL
    • DEXTL Quick Reference Sheets
    • Putting All Together
      • Browser View
      • HTML Code View
      • FORMAT TAGS VIEW
      • DEXTL VIEW
      • RESULTS VIEW
  • ITPilot NSEQL Guide
    • Preface
      • Scope
      • Who Should Use This Manual
      • Summary of Contents
    • Introduction
    • Basic Concepts of NSEQL
      • Navigation Sequences: Basic Syntax
      • NSEQL Commands: General Overview
      • Configuration of Page Waiting Times
      • Example
    • List of NSEQL Commands
      • Usual Parameters
      • Error Parameters
      • Navigation Commands
        • CANCELNEXTNAVIGATION
        • CANCELNAVIGATION
        • EXTENDEDWAITPAGES
        • GOBACK
        • GOFORWARD
        • NAVIGATE
        • IGNORENAVIGATIONERRORS
        • SETMAXDOWNLOADTIME
        • NAVIGATEEXTENDED
        • POSTDATA
        • REFRESH
        • STOP
        • WAIT
        • WAITPAGES
      • Commands for Processing Links
        • CLICKONANCHORBYHREF
        • CLICKONANCHORBYHREFREC
        • CLICKONANCHORBYPOSITION
        • CLICKONANCHORBYTEXT
        • CLICKONANCHORBYTEXTREC
        • CLICKONAREABYHREF
        • CLICKONAREABYPOSITION
        • SELECTANCHORBYHREF
        • SELECTANCHORBYHREFREC
        • SELECTANCHORBYPOSITION
        • SELECTANCHORBYTEXT
        • SELECTANCHORBYTEXTREC
        • SETTARGETURL
      • Commands for Selecting Forms
        • FINDFORMBYACTION
        • FINDFORMBYACTIONREC
        • FINDFORMBYNAME
        • FINDFORMBYNAMEREC
        • FINDFORMBYPOSITION
        • RESETFORM
      • Commands for Executing Actions on Elements of a Form
        • CLICKONELEMENT
        • CLICKONINPUTELEMENT
        • FIREEVENT
        • SELECTINDEXBYPOSITION
        • SELECTINDEXBYTEXT
        • SELECTINDEXBYVALUE
        • SELECTOPTIONBYPOSITION
        • SELECTOPTIONBYTEXT
        • SELECTOPTIONBYVALUE
        • SETINPUTVALUE
        • SETTEXTAREAVALUE
        • SETVALUE
        • SETCHECKED
        • SUBMITFORM
        • UNSELECTALL
      • Frame Selection Commands
        • FINDFRAMEBYNAME
        • FINDFRAMEBYNAMEREC
        • FINDFRAMEBYPOSITION
        • FINDFRAMEBYSOURCE
        • FINDFRAMEBYSOURCEREC
        • RESETFRAME
      • Commands for Selecting Elements
        • FINDACTIVEELEMENTINDOCUMENT
        • FINDCHILDELEMENTBYATTRIBUTE
        • FINDCHILDELEMENTBYPOSITION
        • FINDCHILDELEMENTBYTEXT
        • FINDELEMENTBYATTRIBUTE
        • FINDELEMENTBYATTRIBUTEREC
        • FINDELEMENTBYCHILD
        • FINDELEMENTBYCHILDREC
        • FINDELEMENTBYPOSITION
        • FINDELEMENTBYTEXT
        • FINDELEMENTBYTEXTREC
        • FINDELEMENTBYXPATH
      • Commands for Executing on Actions on Elements
        • CLICKONSELECTEDELEMENT
        • FIREEVENTONSELECTEDELEMENT
        • TRANSPOSESELECTEDTABLE
      • Commands for Handling Windows
        • ALLOWOPENNEWWINDOWS
        • CLOSEWINDOW
        • GETWINDOWNAME
        • OPENALERTDIALOGSINNEWWINDOW
        • OPENNEWWINDOW
        • OPENNEWWINDOWSUNNAMED
        • OPENWEBPAGEDIALOGSINNEWWINDOW
        • SELECTWINDOW
        • SETWINDOWNAME
        • CLOSEINACTIVEWINDOWS
      • Commands for Handling Pop-Up Dialogs
        • ONDIALOG
        • ONPRINTDIALOG
        • ONWEBPAGEDIALOG
        • ONCERTIFICATEDIALOGDENODOBROWSER
        • SETDIALOGPROPERTIES
        • SETSILENT
      • Commands for Editing the Browser Properties
        • SETAMBIENTPROPERTIES
        • SETPROXYAUTHINFO
        • SETUSERAGENT
      • Informative Commands
        • FINDCOOKIES
        • GETEXTENDEDSOURCECODE
        • GETLASTURL
        • GETLASTURLPOSTPARAMETERS
        • GETLOCATION
        • GETMIMETYPE
        • GETSELECTOPTIONS
        • GETSOURCECODE
        • GETTITLE
        • ISDOWNLOADING
        • QUERYDOWNLOADEDPAGES
      • Conversion Commands
        • CONVERTEXCELTOHTML
        • CONVERTWORDTOHTML
        • CONVERTPDFTOHTML
      • Commands for Saving Files
        • SAVEFILE
      • Commands for Executing Code
        • EXECUTEJS
        • EXECUTEJSBEFORELOAD
      • Commands for Clearing Browser Data
        • CLEARPERSISTENTCOOKIES
        • CLEARCACHE
    • Util Functions
      • EncodeSEQ
  • ITPilot API Javadoc
  • Solution Manager Installation Guide
    • Preface
      • Who Should Use This Document
    • Preinstallation Tasks
      • Hardware Requirements
      • Software Requirements
        • Supported Platforms
        • Supported Java Runtime Environments (JRE)
        • Supported Browsers
      • Other Preinstallation Tasks
        • Check that the Required Ports are Free
        • Check the PATH Environment Variable on Windows
        • Select a User Account to Install the Solution Manager
      • Download the Installer
    • Using the Graphical Installation Wizard
      • Commons Settings
      • Components and Configuration of Virtual DataPort
      • Components and Configuration of Solution Manager
      • Components and Configuration of Web Panel
      • Embedded Web Container
    • Using the Command Line Installer
    • Unattended Installation of the Solution Manager
      • Modifying the Solution Manager Installer to Include the Latest Update
      • Unattended Installation of the Solution Manager
    • Post-Installation Tasks
      • Install the Latest Update
      • Change the Default Passwords
      • Install the License
      • Enable SSL/TLS in the Solution Manager
        • Obtaining and Installing an SSL Certificate
        • Enabling SSL/TLS in Solution Manager Servers
      • Configure Windows Services
      • Installing the Solution Manager Web Applications as Windows Services
      • Post-Installation Tasks: Virtual DataPort
      • High Availability
      • Disable “Export” Endpoint of the Web Container
    • Denodo Platform Control Center
      • Starting Platform Servers and Tools
      • Installing Updates and Hotfixes
      • Virtual Machine and Web Container Configuration
      • Configuration of the JVM Parameters from the Command Line
      • Control Center Help
      • Uninstalling the Solution Manager
    • Appendix
      • Default Ports Used by the Solution Manager Modules
      • Using Kerberos Authentication in Virtual DataPort Without Joining a Kerberos Realm
      • Using Kerberos Authentication in Solution Manager Without Joining a Kerberos Realm
      • Providing a Krb5 File for Kerberos Authentication
      • Launching the Denodo Standalone Applications in High DPI Displays
      • Troubleshooting the Solution Manager Installer
      • Transparent Metadata Encryption in the Solution Manager
      • Configuring the Network Interface Restriction in the Solution Manager
  • Solution Manager Administration Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Introduction
      • General Architecture
    • Launching the Solution Manager Components
      • Launching the License Manager Server
      • Launching the Solution Manager Server
      • Launching the Solution Manager Administration Tool
    • Introducing the Solution Manager Administration Tool
    • Authentication and Authorization
      • Authenticating with Virtual DataPort
      • Authenticating with Kerberos
      • Authorization
        • Solution Manager Administrator
        • Promotion Administrator
        • Promotion Administrator for Certain Environments
        • Promotion
        • Promotion for Specific Environments
        • Overview of the Promotion Roles
        • JMX Administrators
        • Global Administrator
        • Developer
    • Environments
      • Creating Environments
      • Configuring Environments
        • Configuring Virtual DataPort Properties
        • Configuring Deployments
        • Configuring Deployment Scripts
        • Configuring Environment Logging Level
    • Clusters
      • Creating Clusters
      • Configuring Clusters
        • Configuring Cluster Load Balancing Variables
        • Configuring Scheduler Properties
        • Configuring Cluster Logging Level
    • Servers
      • Creating Servers
      • Configuring Servers
        • Configuring Server Load Balancing Variables
        • Configuring Server Logging Level
    • Load Balancing Variables
      • Creating Load Balancing Variables
      • Assigning Values to Loading Balancing Variables
    • Promotions
      • Creating Revisions
        • General Options
        • Additional Options
        • Revision Candidate Elements
      • Load Revisions from VQL
        • New Revision from VQL Dialog
      • Revisions Table
        • Create a Revision
        • Edit a Revision
        • See the Information of a Revision
        • Remove Revisions
        • Filter Revisions in the Table
        • Download the Metadata of a Revision
        • Validate a Revision
        • Check the Validation Summary of a Revision
        • Copy Revision
        • Deploy Revisions
        • Check the Environments where a Revision Was Deployed
      • Deploying Revisions
        • Deployment Preconditions
        • Deployment Paradigms
        • Cache Limitations
      • Deployments Table
        • Cancel Deployments
        • Filter Deployments in the Table
        • Check the Progress Summary of a Deployment
        • See the Errors of a Deployment
        • See the Revisions of a Deployment
        • Download Backup Data for a Deployment
      • Deployment Progress
    • License Management
      • Install a License
        • Check Global License Information
      • Assign a License to an Environment
        • Check the License Information for an Environment
      • Assign a License to a Server
      • How Licenses Work
        • How a Server Resolves its License
        • How Denodo Servers Behave on License Responses
      • License Usages Table
    • Monitoring
      • Monitoring Configuration
      • JDBC Logging Configuration
    • Configuration
      • Setting-Up an External Database
        • Changing the Password of the External Database
      • VCS Configuration
      • Setting-Up the Informative Message
    • Import and Export Catalog
    • Appendix
      • REST API
        • Get the List of Environments
        • Create an Environment
        • Get the List of Licenses Available
        • Update an Environment
        • Delete an Environment
        • Get the List of Virtual DataPort Properties Associated to an Environment
        • Create and Delete Environment Properties
        • Get the List of Clusters
        • Create a Cluster
        • Get a Cluster
        • Update a Cluster
        • Delete a Cluster
        • Get the List of Scheduler Properties Associated to a Cluster
        • Create and Delete Cluster Properties
        • Get the List of Servers
        • Create a server
        • Get a Server
        • Update a Server
        • Delete a Server
        • Get the List of Revisions
        • Create a Revision from a VQL file
        • Get the List of Deployments
        • Start a New Deployment from a List of Revisions
        • Get the Progress of a Deployment
        • Free License Usage
        • Ping License Manager Server
        • Ping to Solution Manager Server
      • Best Practices: Deployment of Updates Across Your Organization
        • Updating Solution Manager with High Availability
      • Solution Manager Compatibility with the Denodo Platform Servers
      • How To Initialize a Git Repository
  • Web Panel Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Installation and Execution
      • Launching the Web Panel Administration Tool
      • Installing Web Panel as Windows Service
    • Accessing Denodo Web Tools
    • Administration
      • Authentication
      • Configuration
        • Virtual DataPort Server Set-Up
        • Kerberos Set-Up
        • Informative Message Set-Up
      • Web Applications Management
  • Denodo4E User Guide
    • Preface
      • Scope
      • Who Should Use This Document
      • Summary of Contents
    • Installing Denodo4E
    • VDP/ITP Server Tutorial
      • Creating an Extension Project with a Custom Stored Procedure
      • Creating Debug Configurations for the VDP/ITP Server and the VDP Administration Tool
      • Starting the VDP/ITP Server in Debug Mode
      • Debugging the Stored Procedure
      • Deploying the Stored Procedure
    • Wrapper Generation Tool Tutorial
      • Creating an Extension Project with an ITPilot Custom Function
      • Creating a Debug Configuration for the Wrapper Generation Tool
      • Starting the Wrapper Generation Tool
      • Debugging the Custom Function
      • Deploying the Custom Function
    • Reference of the Dialogs of the Denodo4E Plugin
      • Create
        • The Denodo Extension Project Wizard
        • The Denodo Extension Wizard
      • Debug
        • Denodo Application Debug Configuration
        • Supported Denodo Applications
        • Support for Dependencies
        • Debug Remote Applications
      • Deploy
        • The Deploy Wizard
      • Supported Extension Types
  • DenodoConnect Components
    • Apache Zeppelin for Denodo - User Manual
    • Denodo Cloud Cache Load Bypass Stored Procedure - User Manual
    • Denodo Custom Wrapper for IBM Cognos TM1 - User Manual
    • Denodo Custom Wrapper for SAS - User Manual
    • Denodo Dialect for SQLAlchemy (feat. support for Apache Superset and Jupyter Notebook) - User Manual
    • Denodo Dialect for Spark SQL - User Manual
    • Denodo Distributed File System Custom Wrapper - User Manual
    • Denodo Dynamics 365 Business Central Templates - Quick Use Guide
    • Denodo Dynamics 365 Customer Engagement Templates - Quick Use Guide
    • Denodo Email Exported Files Custom Handler - User Manual
    • Denodo FileSystem CustomWrapper - User Manual
    • Denodo GeoJSON Service - User Manual
    • Denodo Google Sheets Custom Wrapper - User Manual
    • Denodo Google Sheets Tool - User Manual
    • Denodo Governance Bridge - User Manual
    • Denodo GraphQL Service - User Manual
    • Denodo HBase CustomWrapper - User Manual
    • Denodo Hibernate Dialect - User Manual
    • Denodo Incremental Cache Load Stored Procedure - User Manual
    • Denodo Kafka Custom Wrapper - User Manual
    • Denodo Log Custom Wrapper - User Manual
    • Denodo Model Bridge - User Manual
    • Denodo MongoDB Custom Wrapper - User Manual
    • Denodo Monitor Reports - User Manual
    • Denodo OData 2.0 Service - User Manual
    • Denodo OData2 Custom Wrapper - User Manual
    • Denodo OData4 Custom Wrapper - User Manual
    • Denodo Power BI Custom Connector - User Manual
    • Denodo Presto Cluster on Kubernetes - User Manual
    • Denodo RFCReadTable Custom Wrapper - User Manual
    • Denodo Replicator - User Manual
    • Denodo SFTP Exported Files Custom Handler - User Manual
    • Denodo SSH Custom Wrapper - User Manual
    • Denodo SalesForce REST Custom Wrapper - User Manual
    • Denodo ServiceNow Templates - Quick guide
    • Denodo Tableau Exporter - User Manual
    • Denodo Templates for Adobe Analytics - Quick Use Guide
    • Denodo Templates for Google Analytics - Quick Use Guide
    • Denodo Templates for Twitter - Quick Use Guide
    • Denodo Testing Tool - User Manual
    • Denodo XSLT Mapper - User Manual
    • Denodo XtraFuncs - User Manual
    • Generic XML Exporter - User Manual
  • Denodo Express Quick Start Guide
  • Denodo Platform Container QuickStart Guide
  • Denodo Platform for AWS BYOL Quick Start Guide
  • Denodo Platform for AWS Quick Start Guide
  • Denodo Platform for Azure BYOL Quick Start Guide
  • Denodo Platform for Azure Quick Start Guide
  • Denodo Platform for GCP BYOL Quick Start Guide
  • Denodo Platform for GCP Quick Start Guide
Important: To see the latest version of the document click here
  • User Manuals /
  • Virtual DataPort Administration Guide /
  • Version Control Systems Integration /
  • Scenarios and Recommended Uses

Scenarios and Recommended Uses¶

Denodo proposes three workflows when working with a version control system (VCS) on the development environment:

  1. Centralized workflow with shared databases

    • There is a single Virtual DataPort server and each developer uses its Administration Tool to connect to this Server.

    • For each project, there is one single database. We recommend having one database per project.

    • The developers of each project work with the same database.

  2. Distributed workflow

    • Each developer has a local installation of the Virtual DataPort server and its administration tool. Each local installation is under control of the developer. I.e. the developer is an administrator of that server.

    • Each developer has a database for each project and they check in and out the changes from the central repository of the organization.

    • Although all the developers will be administrators on their own Denodo servers, this workflow still requires the figure of the administrator that coordinates changes in jar extensions and internationalization maps. The purpose is to avoid that changes in jar extensions or i18n maps affect other projects.

      The section Best Practices to Manage Global Elements (below) explains this in more detail.

  3. Centralized workflow with private databases

    • There is a single Virtual DataPort server and each developer uses its Administration Tool to connect to this Server.

    • For each project, there is one central database that is managed by the project manager or the administrator.

    • For each project, the administrator creates a database for each developer of the project and grants each developer the ADMIN privilege (administrator of the database) over that database.

    • When working on a project, each developer connects to her own database.

      After completing a task, the developer must check in the changes to make it available to the other developers.

Picking a VCS Workflow¶

All the workflows we described have pros and cons, so it is important to determine which one fits better the organization’s development environment.

  1. Centralized workflow with shared databases

    With this workflow you achieve the main purposes of a version control system:

    1. Store elements on a central repository, keeping a history of all the changes.

    2. Being able to identify the author of a specific change.

    3. Restore a specific revision

    The drawback of this workflow compared to the others is that it does not provide the capability of detecting and resolving conflicting changes performed by different developers over the same element. That is because with this workflow, the developers of a project work on the same database so they must coordinate among themselves to prevent modifying the same elements simultaneously.

  2. Distributed workflow

    Use this workflow if you need the capability of several developers detecting the type of conflicts mentioned above. If two developers make different changes on the same element, the second developer to check the change into the VCS will find that the element has been modified previously and will have to resolve the conflicts. With this workflow each developer needs to install and manage a Virtual DataPort server.

  3. Centralized workflow with private databases

    This workflow also provides the conflict detection/resolution of the distributed workflow. The difference with the distributed workflow is that, in the distributed workflow, each developer has her own Virtual DataPort server; in this one, they all work with the same Virtual DataPort server, although they are connected to their own database.

The following sections provide more details about each suggested workflow:

  • Centralized workflow with shared databases

  • Distributed workflow

  • Centralized workflow with private databases

Centralized Workflow with Shared Databases¶

This workflow adapts to the majority of projects. In this workflow, all the developers of a project share the same database.

The following diagram shows a Virtual DataPort server that follows this workflow:

  • There are two projects: A and B. Each project has its own database.

Centralized workflow with shared databases

The workflow is the following:

  1. At the beginning of the project, the administrator creates the database and checks it into the VCS. It also grants the developers the privileges to access this database.

    If the administrator grants the EXECUTE and WRITE privileges to a developer over a set of views instead of the entire database, the administrator also has to grant the EXECUTE privilege over the views directly referenced by these derived views. Otherwise, the check ins of the views the user can modify will fail. The reason is that when Denodo checks a view into the VCS, it needs to obtain the VQL of that view and to do that, the user needs the EXECUTE privilege over the views referenced by this view.

    If the users have the privilege WRITE over a view V1 but do not have the privilege EXECUTE over the views directly referenced by this view:

    • They cannot modify V1.

    • They can change the settings of the Options dialog of V1.

    • They cannot check in the changes to they do in the Options dialog of V1 because they cannot obtain the VQL of V1.

  2. To check in changes to a base view, the developer needs to have the WRITE privilege over the view. In the past, the user needed to be the owner of the base view or the administrator of the database.

  3. Developers begin to work and when they complete a task, they check the changes into the VCS. The developers must coordinate among themselves to prevent modifying the same elements simultaneously.

  4. When a developer changes a view and this change invalidates other derived views (e.g. remove a field of a view and this field is used by another view), the developer must fix the affected derived views/web services. Then, do a single check in that contains all these changes.

    If the developer makes a change that invalidates other views but does not have the privileges to modify the affected views, a developer that has the privileges to modify all the affected views should fix them. Then, this same developer should do a single check in that includes the initial changes to the view and the fix to the affected views.

    If including all the changes in a single commit is not possible, the VQL of some revisions could be invalid.

  5. As a conclusion of the item above, when developers have access to different sets of views of the database, not the entire database, we recommend they check in the changes in layers. That is, the group that can modify lower-level elements (data sources and base views) should check in the changes to these first. Then, the group that can modify intermediate views, check in changes to intermediate views and then, the same with final views and web services. This will prevent the situation described above.

  6. When developers require a new extension, they request it to an administrator. The administrator imports it and checks it in using the new wizard for checking in and out global elements. As soon as the administrator adds it to the Server, it becomes immediately available to all developers. However, the administrator should check that extension into the VCS to keep a history of the changes.

Take the following into account:

  • With this workflow, developers do not need to check in their changes because as they all use the same database, they all have access to the changes immediately. However, we recommend they check in their changes after completing a task in order to keep a history of all the changes.

  • From now on, only administrators and administrators of a database will be able to revert to a previous version. This behavior prevents the problems that could arise if a standard user reverted to a version that would involve executing VQL statements that the user was not allowed to execute. This rule applies to elements of a database and global elements.

  • If a database needs to be reverted to a version that requires an older version of a global element, an administrator has to revert the global elements first and then, the elements of the database. The section Best Practices to Manage Global Elements below explains these steps in detail.

  • In this workflow, when a user checks in a database or a folder, the Check in dialog may list elements that the Server Explorer (the tab on the left) does not list or are marked as synchronized (with the icon icon_vcs_synchronized). This occurs when another user created/modified elements on the database or folder the user is checking in, since the user refreshed the database (i.e. clicked the menu File > Refresh).

  • In this workflow, after doing a commit, a user may see new elements that were not there before. The reason is that after a commit, the administration tool refreshes the database (equivalent to clicking the menu File > Refresh). If another user created an element on this database since the last refresh, they will now appear in the elements tree.

Distributed Workflow¶

This is the recommended workflow in projects for which you need the capability of detecting and resolving conflicting changes performed by different developers over the same element.

In this workflow, all developers have their own Virtual DataPort server and they are administrators of their own servers. Therefore, they can manage the global elements without any restriction.

The following diagram shows a Virtual DataPort server that follows this workflow:

  • There are two projects: project A and project B. Each project has its own database.

  • The developers have their own Virtual DataPort server so they only need to check out the database of the projects they work with.

    • Developer 1 only works on project B so only checks out the database B.

    • Developer 2 works on project A and project B. In addition, there is a non-versioned database just for testing.

  • Both developers manage the global elements used by the elements of its databases.

Distributed workflow

Distributed workflow¶

The workflow is the following:

  1. At the beginning of a project, the coordinator of the project creates the database and notifies all the developers.

  2. Each developer checks that database out and begins working.

  3. When a developer wants to share work with other developers, she has to check the changes into the VCS.

  4. When developers require a new extension, they request it to an administrator. The administrator imports it and checks it in using the new wizard for checking in and out global elements. The administrator will later notify the developers of the project and they will obtain the new element using the new wizard that only checks in and out global elements.

  5. When a developer changes a view and this change invalidates other derived views (e.g. remove a field of a view and this field is used by another view), the developer should fix the affected derived views/web services and do a single check in that contains all these changes. If the developer only checks in the change in the view but not the fix for the affected views, when other developers check out changes from the VCS, they could get an error if the VQL stored in the VCS is not valid.

    In addition, this is a way of making sure that you can revert back to any revision of the project. If including all the changes in a single commit is not possible, indicate so in the commit message to prevent that in the future, someone tries to revert back to this commit.

Centralized Workflow with Private Databases¶

This is the recommended workflow for when you only have a Virtual DataPort server available and you also need the capability of detecting and resolving conflicting changes performed by different developers over the same element.

In this workflow, there is a single Virtual DataPort server for development. For each project, there is one central database that is managed by the project manager and one database for each developer of the project. All these databases point to the same database in the VCS.

The following diagram shows a Virtual DataPort server that follows this workflow:

  • There are two projects: project A and project B. Each project has its own database.

  • Developer 1 and Developer 2 work on project A, so for this project we have these databases:
    project_a, project_a_developer1 and project_a_developer2.
  • Only Developer 1 works on project B, so for this project we have these databases:
    project_b and project_b_developer1.
Centralized workflow with private databases

Centralized workflow with private databases¶

The workflow is the following:

  1. At the beginning of a project, an administrator creates these databases:

    • The main database of the project is called canonical database. The coordinator will be the administrator of the database and will check out all the changes to this database periodically. When a database needs to be promoted to another environment, the coordinator will check out the database and export it to a VQL file.

      The administrator will create this database, enable version control for this database and check in the change. This will create the database in the repository.

    • One database for each developer of the project. Each database that will be assigned to a developer. The administrator will grant the privilege “administrator of the database” over each database to the developer assigned to that database. This is the privilege ADMIN you can assign to users.

      To replicate each database, we recommend using the wizard Import database of the menu VCS Management because that will locate the available databases in the repository and the administrator only has to select it and provide a local name for it. This local name should follow the convention:

      <name of the project>_<username of the developer>.
      
  2. The developers of the project connect to their own database and began working.

  3. When a developer wants to share work with other developers, she has to check the changes into the VCS.

  4. The other developers need to do a check out of their database to see the changes performed by others.

  5. When the developers need to create a derived view over a view of another database, they have to point to the views of canonical database, not the developer databases. That is, if developer 2 needs to create a selection view over the view employee of project B, developer 2 has to create the derived view over the view employee of the database project_b, not project_b_developer1. The reason is that in other environments (testing, QA, etc.), only the canonical databases will exist, so the references between databases have to be created with them.

    The administrator has to grant the developers the privileges CONNECT and EXECUTE over the canonical databases over whose views, they will have to create views.

  6. When a developer changes a view and this change invalidates other derived views or web services (e.g. remove a field of a view and this field is used by another view), the developer should fix the affected elements. Then, do a single check in that contains all these changes. If the developer only checks in the change to the view but not the fix for the affected elements, when other developers check out these changes from the VCS, they will get an error because the VQL stored in the VCS is not valid.

    In addition, this is a way of making sure that you can revert back to any revision of the project. If including all the changes in a single commit is not possible, indicate so in the commit message to prevent that in the future, someone tries to revert back to this commit.

  7. When the developers require a new extension, they request it to an administrator. The administrator imports into the canonical database and checks it into the VCS using the new wizard for checking in and out global elements. As soon as the administrator add it to the Server, it becomes immediately available to all developers. However, the administrator should check that extension into the VCS to keep a history of the changes.

  8. If a database needs to be reverted to a version that requires an older version of a global element, an administrator has to revert the global elements first and then, the elements of the database. The section Best Practices to Manage Global Elements (below) explains these steps in detail.

Best Practices to Manage Global Elements¶

Global elements are elements that do not belong to a specific database. For example, users, roles, server settings, etc.

In this section, when we discuss global elements, we are only speaking of the two types of global elements that Virtual DataPort stores on a VCS: jar extensions and internationalization maps (i18n maps).

Follow these best practices to manage global elements when using the VCS support:

  • Administrators of the Denodo server must be in charge of managing the global elements. This is:

    • Uploading jar extensions

    • Managing internationalization maps (i18n maps)

    The aim is to avoid affecting other projects that also use these elements. Only administrators can manage jar extensions and i18n maps. They do that by right-clicking on the database > VCS > Global elements. From this menu, administrators can check in changes to the jar extensions and the i18n maps.

  • When the administrators create, modify or delete global elements, they must check the changes into the VCS. That way, they keep a history of these changes. In addition, when using the distributed workflow, the check in is necessary so the developers have access to the new elements.

  • When developers need to create or modify or delete global elements, they have to ask one of the administrators to do it.

    When working with the distributed workflow, after the administrator that coordinates the project changes in global elements checks in a global element, the developers must do a check out to obtain it. The reason is that each developer has her own Server. In the distributed workflow, the developers are the administrators of their Denodo servers so they can successfully check out global elements.

    This is not necessary in any of the centralized workflows because the elements are already in the Server once the administrator loads them.

  • If a database needs to be reverted to a version that requires an older version of a global element, the process has to be done in two steps:

    1. Revert the global element using a new wizard whose function will be just this (this wizard will be available in the next update).

      Only administrators will be able to use this wizard.

    2. Revert the elements that depend on the database. To do this operation, all the elements have to be checked in or discard the modifications (there cannot be local changes). The dialog to revert changes only shows commits over elements of that database, not global elements or elements from other databases.

  • Developers can check in elements that depend on global elements. E.g. derived views that use a custom function included in an extension.

  • Folders only can be created by an administrator or an administrator of the database where it is created, so it is recommended to check in the folders just after be created.

As all the databases share the global elements, there may be problems when you need different versions of the same extension. For example, let us say there are two projects project A and project B and each require a different version of a jar extension. Another example: you need to revert project A to a previous version that requires an older version of a jar extension and project B requires a newer one. If you run into these scenarios, consider using these naming conventions:

  • When a project needs a new version of a jar extension and another project still needs the old version, load the new version of the extension with a new name. This new name should be the name of the extension followed by the name of the project that needs this new version. The developers of this project will need to know that from now on, they need to use that new jar extension and not the existing one.

    We only recommend doing this when it is necessary to avoid having many versions of the same library loaded.

  • If a project needs to modify a map that is being used by another project, the administrator should create a new map, adding a suffix to its name so developers can distinguish it.

More Information about Global Elements¶

Virtual DataPort manages jars extensions and internationalization (i18n) maps as global elements, common to all databases. The reasons why they do not belong to an individual database are:

  • Jar extensions are global because they extend the capabilities of the Virtual DataPort server, not a specific database. If they were linked to a single database, on many occasions, they would have to be loaded several times. Having the same extension replicated would increase the resources needed to store and use these extensions.

  • Internationalization (i18n) maps are global because usually they are used across all the databases. Therefore, if you create one, it can be used in all databases instead of having to replicate it across all of them.

Administrators are in charge of managing global elements and checking them into the VCS.

In Denodo 6.0 prior to update 20180618, the management of global elements used by databases synchronized with VCS was transparent to the user. When you did a check out and there were changes on a global element, these were checked-out as well. When you checked in an element that depended on global elements that had been modified, the check in also included the change to the global element.

  • Depending on the privileges of the user that performed the check out, the check out may not finished correctly. For example, if the user only had privileges to modify certain views of the database and the check out involved modifying a data source, the database remained in an out-of-sync status. Only administrators and administrators of that database can take a database out of this status.

  • When a global element is modified, all the databases with elements that use this global element are marked as modified as well. Then, all these databases have to be checked into the VCS even though is not really necessary because the jar has already been stored in the VCS after the check in of the first database.

To avoid these problems, Denodo 6.0 update 20180618 includes the following changes regarding the management of i18n maps and jar extensions:

  1. There is a wizard just for checking in and out global elements. This wizard is only available to administrators. To open it, right-click on a database > VCS > Global elements.

  2. Developers are not be able to check in or out global elements from the VCS. The implications are:

    • When a developer checks changes into the VCS, the commit will never include modifications to global elements, even if they have been modified by an administrator.

    • When a developer checks out changes from the VCS, the Virtual DataPort server ignores the changes to these elements.

    • The wizard to revert elements of a database (not global elements) will only list commits that contain changes to the elements of that database.

      Because of the wizard described in step 1, there will not be commits that modify global elements and elements of a database. A commit will only modify one type of element or the other.

Promoting Changes from Development to Testing and Production¶

After the development team has finished its work, it is time for testing.

The best way to promote a database or a set of changes from the development server to the testing server is to use the Solution Manager to create a revision. Then deploy the revision to the testing environment. After testing that the new views work as expected, deploy the same revision to the production environment. With previous versions of Denodo, the recommended way to promote elements was creating a script that invoked the scripts “export” and “import” of Denodo. However, doing it with the Solution Manager is better because:

  • The GUI of the Solution Manager makes it easier for developers to select the elements to promote.

  • It provides capabilities to promote elements to a cluster of Virtual DataPort servers, not just one server.

  • The revisions are persisted so you can review them later and keep a history of the changes.

  • …

Promoting a Database Using Scripts Instead of the Solution Manager¶

If you want to promote changes using the scripts “export” and “import” instead of the Solution Manager, the administrator must follow these steps:

  1. Connect to the central database of the project (in our example, project1 or project2) and check out the changes from the VCS.

  2. Export to a file the entire database with the option “Export environment specific properties separately”.

  3. Edit the generated properties file to adapt its values to the testing environment. E.g. Modify the URL of the JDBC data sources to point to the testing databases along with new credentials, etc.

    On subsequent promotions of a database, the administrator can use the properties file created on previous occasions. If the administrator imports a VQL file and the properties file does not have all the properties, the import process will fail without changing anything on the Server. If this happens, the import process will return a list of the properties that are missing so the administrator can add them to the properties file.

  4. Import the VQL file with the properties file into the testing server.

After this, the users can begin testing the views they have created.

Regarding this process, we recommend the following:

  • To promote a database from one environment to the other, develop a script that invokes the scripts “import” and “export” of the Denodo Platform.

  • The process (e.g. the script) to promote a database from development to testing, and from testing to production should be the same. That way, when promoting to testing, you make sure that the process works.

  • To promote changes from one environment to another (e.g. from development to testing), we recommend promoting the entire database and not just a set of views. This avoids that some changes are not promoted accidentally.

  • To promote a database from testing to production use a VQL file and not VCS because the process of importing a database from a VCS repository takes longer than from a VQL file.

  • Because of the recommendation above, we also recommend promoting databases from development to testing using a VQL file. That way, when you migrate a database from development to testing, you test that the script that migrates databases between environments works.

Using VCS Environments¶

Defining several VCS environments is useful when:

  • The development team is geographically distributed. For example, there is a team of Denodo developers in London and another in Denver.

  • And on each location, there is a replica of the data sources they use. For example, there is one Oracle database in London and another Oracle database in Denver and their views, tables… are the same (although it’s data may be different).

  • And the teams of each location want to point to the data sources of their location.

If your organization does not meet all these conditions, we recommend creating only one environment in the server called “development”.

Best Practices When Using the Integration with a VCS VCS Settings of the Administration Tool

  • Denodo Site
  • Support Site
  • Partner Portal
  • Contact Us
  • Terms of Use
  • Privacy & Cookies Policy
  • ©  Denodo Technologies