SKIP TO MAIN CONTENT
Sign In
Register
Denodo Platform 9 is released!
Expert Trails
Cache
Datasource Connectivity
Deployment Topology
Development Lifecycle
Modeling and Metadata Organization
Monitoring
Operations
Query Performance Optimization
Security
Considerations for Designing a Self-Service Analytics Initiative
Enterprise Data Services with APIs
Q&A
All Questions
Ask a Question
Search by tag
Resources
User Manuals
Knowledge Base
Videos
Tutorials
Browse Content Library
Drivers
Data Management Blog
Technical Newsletter
Test Drives
Agile BI and Analytics on AWS
Data Catalog and Data Science on AWS
Agile BI and Analytics on Azure
Data as a Service (DaaS) on GCP
Agile BI and Analytics on Alibaba
Free Trials
Denodo Express
Learn
Training Courses
Learning Paths
Course Catalog
Your Progress
Workshops
Certification Exams
Academic Program
Overview
University Challenge
User Groups
Support
Support Site
Contact Us
USER MANUALS
Denodo Platform
9.0
- Select one -
Denodo 9.1-beta
Denodo 9.0
English
日本語
Denodo Platform Installation Guide
Legal Notes
Third-Party Java Libraries Distributed with the Denodo Platform 9.0
Third-Party Non-Java Libraries Distributed with the Denodo Platform 9.0
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
Pre-Installation Tasks
Hardware Requirements
Minimum Requirements to Run the Denodo Platform On Premise
Minimum Requirements to Run the Denodo Platform on Amazon AWS
Minimum Requirements to Run the Denodo Platform on Microsoft Azure
Minimum Hardware Requirements when Installing each Module Separately
Considerations Regarding the Disk Space Requirements of the Denodo Platform
Running the Denodo Platform on Docker
Requirements for the Virtual DataPort Administration Tool
Software Requirements
Supported Operating Systems
Supported Browsers
Configuring the Server for the Denodo Platform
Select a User Account to Install the Denodo Platform
Grant Privileges to This User Account
Check the PATH Environment Variable (Windows)
Check that the Required Ports Are Free
Download the Installer
Using the Graphical Installation Wizard
Settings you can Modify when you Select the Custom Installation
Common to All Modules
Virtual DataPort
Scheduler Server
Scheduler Index Server
Embedded Web Container
Using the Command Line Installer
Unattended Installation of the Denodo Platform
Modifying the Denodo Platform Installer to Include the Latest Update
Post-Installation Tasks
Replicate the Encryption Key Across All the Installations
If You Already Started One of the Components of the Denodo Platform / Solution Manager
Install the Latest Update
Register the Denodo Servers in the License Manager
Enable SSL/TLS in the Denodo Platform
Obtaining and Installing an SSL/TLS Certificate
Denodo SSL/TLS Configurator Script
Enabling SSL/TLS in Denodo Platform Servers
Enabling HTTPS in the Embedded Apache Tomcat
Enabling SSL/TLS in the Administration Tool and Others
Enabling SSL/TLS for External Clients
SSL/TLS Versions Supported by the Denodo Platform Servers
Change the Default Passwords
Virtual DataPort
Design Studio
Data Catalog
Scheduler
Set the Host Name of Denodo Servers
Virtual DataPort
Scheduler Server
Scheduler Index Server
Configure Windows Services
Upgrading from a Previous Version and Other Tasks
Installing Updates and Hotfixes
Installing an Update or Hotfix Graphically
Installing an Update or Hotfix from the Command Line
Troubleshooting the Updates Installer
Denodo Platform Control Center
Launching the Denodo Platform Control Center
Starting Platform Servers and Tools
Denodo Platform Configuration
Configuring the Connection to the License Manager
Configuration of the JVM Parameters from the Command Line
Control Center Help
Uninstalling the Denodo Platform
Appendix
Supported Java Development Kits (JDK)
Troubleshooting the Denodo Platform Installer
Denodo Platform Upgrade Guide
Preface
Introduction to Upgrading the Denodo Platform
Major Steps in the Upgrade Process
Running Both Versions in the Same Computer
Upgrading from Denodo Platform 7.0 or Earlier
Upgrading from Denodo Platform 7.0, 6.0 and 5.5
Upgrading from Denodo Platform 5.0 or Earlier
Preparing to Upgrade
Common to All Modules
Kerberos
Virtual DataPort
Virtual DataPort: Select a Catalog/Schema for the Cache Engine
Virtual DataPort: Create a Repository for Version Control
Data Catalog: Create a Catalog/Schema on the External Database
Scheduler: Create a Catalog/Schema on the External Database
Solution Manager
Solution Manager: Create a Git Repository or Branch for Version Control
Solution Manager: Create a Catalog/Schema on the External Database
Pre-Upgrade Tasks for the New Installation
Common to All Modules
Copy the Encryption Key
Install the Latest Update
Enable SSL
Configure Windows Services
Kerberos: Copy Krb5 File
Virtual DataPort
Virtual DataPort: Enable the External Database
Virtual DataPort: JDBC Data Sources that Use the Bulk Data Load API
Virtual DataPort: Enable the Cache Engine
Virtual DataPort: Install Third-Party Connectors
Virtual DataPort: Other Third-Party Jar Files
Virtual DataPort: Version Control System
Virtual DataPort: Delete Unused Objects
Virtual DataPort: Delete Aracne and Google Search Data Sources
Virtual DataPort - Published Web Services: Customize Logo
Virtual DataPort - Denodo Monitor: Configure to Store the Information in a Database
Data Catalog: Enable the External Database
Scheduler: Enable the External Database
Solution Manager
Solution Manager: Enable the External Database
Solution Manager: High Availability
Solution Manager: Enable Git Support
Solution Manager: Configure Monitoring JDBC Appenders
Upgrading from Denodo 6.0: Export the Settings and Metadata of the Current Installation
Transforming Views Automatically to Use the New Datetime Data Types
Configuration Properties of Virtual DataPort for Upgrades: “exportMigrationCompatibility” and “exportOnlyResources”
Alternatives for When You Cannot Install the Latest Update of Denodo 7.0
Upgrading from Denodo 7.0: Export the Settings and Metadata of the Current Installation
Configuration Properties of Virtual DataPort for Upgrades: “exportMigrationCompatibility” and “exportOnlyResources”
Alternatives for When You Cannot Install the Latest Update of Denodo 7.0
Upgrading from Denodo 8.0: Export the Settings and Metadata of the Current Installation
Configuration Properties of Virtual DataPort for Upgrades: “exportMigrationCompatibility” and “exportOnlyResources”
Alternatives for When You Cannot Install the Latest Update of Denodo 8.0
Review the Virtual DataPort Metadata
Import Settings and Metadata in to the New Installation
Common Errors when Importing the Metadata of Virtual DataPort to the New Version
Post-Upgrade Tasks for the New Installation
Common to All Components
Configure the Connection to the Denodo Security Token Server
Review Memory Allocation In Production Servers
Virtual DataPort
Increase the Memory Allocated to the Tomcat of Denodo Platform
Design Studio
Virtual DataPort Administration Tool
Embedded MPP
Data Catalog
Scheduler
Scheduler Index
Solution Manager
External Applications: Upgrade the JDBC and ODBC Drivers
Test the New Denodo Platform Installation
Backward Compatibility
Changes in Denodo Platform 9
Features Removed from Denodo Platform 9
Changes that Apply to All the Modules of the Denodo Platform and Solution Manager
Virtual DataPort
Design Studio
Scheduler
Solution Manager
Diagnostic & Monitoring Tool
User Manuals
Changes in Denodo Platform 8.0 GA
Changes Common to All the Modules of Denodo 8.0 GA
Changes in Virtual DataPort 8.0 GA
Changes in Solution Manager 8.0 GA
Features Removed from Aracne 8.0 GA
Features Removed from Virtual DataPort 8.0 GA
Features Removed from Scheduler 8.0 GA
Features Removed from Solution Manager 8.0 GA
Changes in Denodo Platform 7.0
Changes Common to All the Modules of Denodo 7.0
Changes in Virtual DataPort 7.0
Changes in the Privileges System in Virtual DataPort 7.0
Features Removed from Virtual DataPort 7.0
Changes Introduced in Aracne 7.0
Changes in Denodo Platform 6.0
Changes Common to All the Modules of Denodo 6.0
Changes in Virtual DataPort 6.0
Changes in Scheduler 6.0
Changes in the Embedded Web Container of Denodo 6.0
Changes in Denodo Platform 5.5
Changes in Virtual DataPort 5.5
Features Removed in Denodo 5.5
Changes in Denodo Platform 5.0
Changes in Virtual DataPort 5.0
Features Deprecated in Denodo Platform
Virtual DataPort
Administration Tool
Data Type “Date”
VQL Syntax
Data Sources
Denodo Stored Procedures
Role JMXAdmin
Denodo Stored Procedures API: getNumOfAffectedRows Method
Denodo Custom Wrappers API: Deprecated Methods
Denodo JDBC Driver
Denodo Web Services
Version Control Systems (VCS)
Script Export: Deprecated Parameters
Launching Several Virtual DataPort Instances of the Same Installation
Data Catalog
Roles “selfserviceadmin”, “selfserviceexporter”
Scheduler
Private Scheduler REST API
Solution Manager
REST API: Authentication Using Session
Denodo Platform New Features Guide
Preface
Audience
Denodo 9 Highlights
New Features of the Denodo Platform 9
Denodo Platform Administration Guide
Preface
Audience
Audit Trail In Data Catalog and Scheduler
Audit Information
How to Configure Audit in Scheduler
How to Configure Audit in Data Catalog
Masking Sensitive Information
Default Ports Used by the Denodo Platform Modules
Denodo Platform - Subscription Bundles
Denodo Enterprise Plus
AI & Recommendations FeaturePack
Semantics FeaturePack
Embedded Massive Parallel Processing (MPP) FeaturePack
Denodo Security Token Configuration File
Common Settings
Token Signing Credentials
Authentication Modules
SAML 2.0 Delegation
OAuth 2.0 Delegation
OpenID Connect Delegation
Enable Authentication in the Monitoring Interface of the Web Container
Enabling Kerberos Authentication Without Joining a Kerberos Realm
Health Monitoring
How to Debug Kerberos in Web Applications
Virtual DataPort Server and the Web Administration Tools Are on the Same Installation
Virtual DataPort Server and the Web Administration Tools Are not on the Same Installation
Increasing the Maximum Simultaneous Requests
Installation Encryption Key
Configure Keystore Password Loader
Direct Keystore Password Configuration
File Keystore Password Configuration
Environment Variable Keystore Password Configuration
AWS Keystore Password Configuration
Azure Keystore Password Configuration
Google Cloud Keystore Password Configuration
Keystore Management
Generating a New Keystore with an Encryption Key and its Configuration
Update the Keystore Password
Reset the Encryption Key
Language of the Web Tools
Install a Language File for Data Catalog
Install a Language File for Design Studio
Install a Language File for the Scheduler Administration Tool
Data Catalog, Design Studio and Scheduler in Other Languages
Considerations When Translating the Language Files
How to Refresh Custom Messages After Installing an Update
Password Policies in the Denodo Platform and Solution Manager
Policies that Are Disabled by Default in Virtual DataPort and Solution Manager
Password Expiration
Lock Account After Many Attempts to Log in with the Wrong Password
When All the Administrators are Locked Out of Virtual DataPort or Solution Manager
Important Notes
Prevent Load Balancers From Closing Connections of Data Catalog and Design Studio
Enabling this Feature
Providing a Krb5 File for Kerberos Authentication
Restrictions of Denodo Express
Reverse Proxy Settings (Design Studio, Data Catalog, Scheduler)
Reverse Proxy Configuration
Server Process Identifiers
Session Configuration (Design Studio, Data Catalog, Scheduler, Diagnostic & Monitoring Tool)
Maximum Session Timeout
Idle Session Timeout
Single-User Concurrent Sessions Control
Setting-up Kerberos Authentication
Configuring the Kerberos Authentication Server (e.g. Active Directory)
Creating a User in the Active Directory
Declaring a Service Principal Name (SPN)
Enable Kerberos Delegation for this User Account
Generating a Keytab File
Distribute the Keytab and Enable Kerberos Authentication on Denodo Servers
Setting-up Kerberos Authentication in Scheduler
Scheduler and Virtual DataPort Are in the Same Machine
Scheduler and Virtual DataPort Are in Different Machines
Setting-up Kerberos Authentication in the Data Catalog
The Data Catalog and the Virtual DataPort Server Are in Different Machines
The Data Catalog and the Virtual DataPort Server Are in the Same Machine
Transparent Metadata Encryption
Virtual DataPort Administration Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
General Architecture
Physical Layer
Logical Layer
Data Module: Cache
User Layer
Starting Virtual DataPort
Starting the Virtual DataPort Server
Launching the Administration Tool
Main Areas of the Administration Tool
Tool Preferences
Using the VQL Shell
Query Monitor
Catalog Search
Invalidate Cache
Trace Viewer
Maximum Session Idle Timeout
Starting the Design Studio
Main Areas of the Design Studio
Design Studio Preferences
Using the Design Studio VQL Shell
Design Studio Query Monitor
Design Studio Catalog Search
Design Studio Invalidate Cache
Design Studio Trace Viewer
Logging in to Design Studio
Creating Data Sources and Base Views
JDBC Sources
Preparing the Connection to the Database
Importing JDBC Sources
Database Specific Information
Connecting to a JDBC Source with Kerberos Authentication
Connecting to a JDBC Source with OAuth Authentication
Creating Base Views from a JDBC Data Source
Importing Graphically Stored Procedures from a Database
Creating Base Views from SQL Queries
ODBC Sources
Enabling the Support for ODBC Sources
Enabling the Support for ODBC Sources When Using an External JRE
Creating an ODBC Data Source
Creating Base Views over an ODBC Data Source
SOAP Web Service Sources
Importing SOAP Web Service Sources
Creating Base Views from a SOAP Web Service
Multidimensional Database Sources
Creating a Base View Over a Multidimensional Data Source, Graphically
Creating a Base View Over a SAP Bex Query (only for SAP BW)
Creating a Base View Over a Multidimensional Data Source with an MDX Query
Multidimensional Data Sources to Oracle Essbase
Multidimensional Data Sources Push Down Capabilities
Path Types in Virtual DataPort
Local Path
Path From Variable
HTTP Path
FTP / SFTP / FTPS Path
HDFS Path
S3 Path
ADLS Path (ABFS)
Hadoop-Compatible Storage
Compressed or Encrypted Data Sources
Custom Input Filters
Paths and Other Values with Interpolation Variables
XML Sources
Direct
From OpenAPI 3 Document
JSON Sources
Direct
From OpenAPI 3 Document
Delimited File Sources
Examples of How to Define a Tuple Pattern
Paths Using Date Ranges
Excel Sources
Size Limit When Processing Excel Files
Fields Detected as Text During Introspection with Another Format
LDAP Sources
BAPI Sources
Salesforce Sources
Registering Denodo as a Connected Application
Creating a Salesforce Data Source
Creating Base Views from a Salesforce Data Source
Creating Base Views from SOQL Queries
Binary Fields
Limitations of Salesforce
Additional Settings
Bulk Mode
MongoDB Sources
Creating a MongoDB Data Source
Creating Base Views from a MongoDB Data Source
MongoDB Aggregation Pipelines
Delegation Capabilities and Limitations
Object Storage data in Parquet and Delta Format
How to Connect to a Kerberized HDFS from Embedded MPP Data Sources
From Object Storage vs From MPP Catalogs
Schema Evolution
Source Refresh on Views from Object Storage
Manage Views Created from Object Storage
OData 2 Sources
Create OData v2 Data Sources
Create OData v2 base views
Examples
Known Limitations
Troubleshooting
References
Appendix
OData 4 Sources
Create OData v4 Data Sources
Create OData v4 Base Views
Examples
Known Limitations
References
Custom Sources
Using Interpolation Variables as Custom Sources’ Input Parameters
Software as a Service Wizards
Adobe Analytics
Google Analytics 4
Google Analytics Migration
Marketo
Marketo Bulk Extract
ServiceNow
SharePoint
SharePoint For OData
SharePoint REST API
Viewing the Schema of a Base View
View the VQL of an Element
Used By…
Source Type Properties
Data Source Configuration Properties
Source Refresh
Working with Blob Fields of Base Views
Adobe Analytics
Getting OAuth 2.0 Credentials
Importing Artifacts
Campaign Performance
Query Example
Content Consumption
Query Example
Generic
Query Example
Metric and Dimensions
Query Example
Marketing Ecommerce
Query Example
People Metrics
Query Example
Product Retail
Query Example
Web Acquisition
Query Example
Web Technology
Query Example
Appendix I. How to Simplify the Data Range Format in Views
Google Analytics 4
Getting OAuth 2.0 Credentials
Importing Artifacts
Real Time Reports
Query With Real Time View
Custom Reports
Query With Custom Report View
Custom Pivot Reports
Query With Custom Pivot Report View
GA4 New Features Predefined Reports
Query With GA4 Dimension Configurable Reporting Views
Appendix I. How To Use Google Analytics Data API v1
Example
Google Analytics Migration
Getting OAuth 2.0 Credentials
Importing Artifacts
GA4 Migration Predefined Reports
Query
Marketo
Importing Artifacts
Query Example
Marketo Bulk Extract
Importing Artifacts
Query Example
ServiceNow
Importing Artifacts
SharePoint
Configuring The Application in Azure
SharePoint For OData
Getting OAuth 2.0 Credentials
Importing Artifacts
Query Example
SharePoint REST API
Getting OAuth 2.0 Credentials
Importing Artifacts
Query Example
Creating Derived Views
Creating Union Views
Creating Extended Union Views
Creating Standard Union Views
Creating Partitioned Union Views
Creating Join Views
Join Conditions with Similarity Operators
Creating Selection Views
Creating Conditions with the Compound Values Editor
Parameters of Derived Views
Creating Flatten Views
Creating Intersection Views
Creating Minus Views
Creating Interface Views
Viewing the Schema of a Derived View
Tree View
Data Lineage
Advanced Configuration of Views
Configuring the Cache of a View
Search Methods, Configuration Properties and Internationalization
Indexes of Views
Statistics of the View
Defining the Data Movements of the View
Materialized Tables
Temporary Tables
Querying Views
Execution Trace of a Statement
Editing / Replacing a View
Modifying Base Views
Modifying Derived Views
Views Affected by Modifications
Stored Procedures
Importing Java Stored Procedures
Executing Stored Procedures
Use of Stored Procedures in Creating Views
Publication of Web Services
Publishing SOAP Web Services
Operations Tab
Settings Tab (SOAP)
Advanced Tab (SOAP)
Publishing REST Web Services
Resources Tab
Settings Tab (REST)
Advanced Tab (REST)
XSLT Transformations
Web Services Authentication
HTTP Basic
HTTP Basic with VDP
HTTP Digest
HTTP SPNEGO (Kerberos)
OAuth 2.0 and OpenID
SAML 2.0
Impersonating a User
How Web Services Query the Virtual DataPort Server
Connection from the Web Services to the Server
Web Service Container Status Table
Invoking Denodo Web Services
Invoking Web Services with SAML Authentication
Types Conversion Table for REST / SOAP Published Web Services
Privileges Required
RESTful Architecture
Primary Keys of Views
Associations
Creating an Association
Multiplicity of Associations
Referential Integrity in Associations
Role Preconditions
Why You Should Define Associations Between Views
RESTful Web Service
Representations
Input Parameters of the RESTful Web Service
IDU Requests
Requests with Input Compound Values
Status Codes
Obtaining the Number of Rows of a Result Set
Security and Access Privileges
Configuring the RESTful Web Service
Denodo OData 4.0 Service
Serving Metadata
Querying Data: The Basics
Navigating Associations
Advanced Querying
Pagination
Configuration
Entity Collection Streaming
Debug Option
Privilege Requirements
Limitations
Features
Denodo GraphQL Service
Configuration
Privilege Requirements
Schema
Introspection
Basic Querying
GraphQL Names
Advanced Querying
Authentication
Resource Limitations
Limitations
Features
JMS Listeners
Creating JMS Listeners
Privileges Required
Advanced JMS Configuration
Acknowledge Mode
Acknowledge On Query Finish and Serialized Processing
Enabling Advanced Logging for JMS Listeners
Kafka Listeners
Creating Kafka Listeners
Privileges Required
Advanced Kafka Configuration
Remote Tables
Managing Remote Tables
Creating Remote Tables
Editing Remote Tables
Refresh Data
Dropping Remote Tables
Customizing the Table Creation Template
Remote Tables vs Other Mechanisms for Persisting Data
Server Configuration
Configuring the Cache
Limiting the Number of Concurrent Requests
Credentials Vault
Enabling Access to the Credentials Vault
Supported Credentials Vaults
How A Data Source Obtains the Credentials from the Vault
Specify a Pattern as Secret Name, Parameterized by the Login
Configuring the Default Internationalization
Default Configuration of HTTP Proxy
Identifiers Charset
Restricted Mode
Unicode Mode
Configuring the Memory Usage and Swapping Policy
Memory Usage Settings
Swapping Settings
Maximum Size of a Query
Privileges
Indirect Access Control
The FILE Privilege
Global Settings of REST Web Services
Enable/Disable the OpenAPI Endpoints
Accessing the REST Web Services Through a Reverse Proxy
Changing the HTTP Header “Content-type”
Changing the HTTP Header “WWW-Authenticate”
Obfuscating that the REST Web Services are Published by the Denodo Platform
Queries Optimization
Server Authentication
LDAP Authentication
Kerberos Authentication
SAML Authentication
OAuth Authentication
Denodo Security Token Authentication
Server Connectivity
Configuring Runtime Parameters for Stored Procedures
Threads Pool
Storing the Metadata on an External Database
Replication Process
Databases Supported
Considerations About This Feature
How to Configure Virtual DataPort to Store the Metadata on an External Database
Memory Management
Streaming Vs Non-Streaming Operators
Swapping Parameters
Limit the Maximum Amount of Memory of a Query
Edge Cases in Streaming Operation
Cache Load Processes
Resource Manager
Defining a Plan
Defining a Rule
Logging Resource Manager Actions
Exporting and Importing the Server Metadata
Exporting the Server Metadata
Exporting and Importing Elements Across Different Environments
Exporting Environment-Dependent and Independent Elements to Different Files
Export to a File with Properties
Importing Metadata into a Server
Importing Extensions
Importing a JDBC Driver
Importing a JDBC Driver for a Database that is not Listed
Importing Other Types of Elements
Using a Script to Upload Extensions
Cache Module
Cache Modes
Partial Mode
Full Mode
Incremental Mode
Recommended Parameters for Queries that Load the Cache
Caching Very Large Data Sets
Caching the Result of Queries that Fail
Cache Maintenance Task
Maintaining the Underlying Database
Specific Information about Cache Databases
Amazon Athena
Azure SQL Data Warehouse
Google BigQuery
Kudu
PrestoDB/Trino
Embedded Denodo MPP
Spark
Yellowbrick
Databases Using a Distributed Object Storage
Generic Support for Other Databases
Table Creation Templates Management
Creating or Editing the Table Creation Template On the Data Sources
Configure the Cache Table Creation Templates
Customize the Cache Management Tables Template
Creating or Editing the Table Creation Template On Views
Template Parameters
Internal Parameters List
Version Control Systems Integration
VCS Configuration
Virtual DataPort Server Configuration
Environment Management
Database Configuration
Importing an Existing Database from a VCS Server
VCS Integration Features
Integration with the Server Explorer
VCS Operations for GIT
Dependencies Management
VCS Settings of the Administration Tool
Scenarios and Recommended Uses
Picking a VCS Workflow
Centralized Workflow with Shared Databases
Distributed Workflow
Centralized Workflow with Private Databases
Best Practices to Manage Global Elements
Promoting Changes from Development to Testing and Production
Using VCS Environments
Best Practices When Using the Integration with a VCS
Tagging and Branching Releases
Recommendations for the Testing Environment
Databases, Users and Access Rights in Virtual DataPort
Databases in Virtual DataPort
User and Access Rights in Virtual DataPort
Users
Roles
Types of Access Rights
Privileges at Database Level
Fine-Grained Privileges at View Level
Column Privileges
Row Restrictions
Modeling Fine-Grained Role Restrictions on Multiple Roles
Privileges are Cumulative, not Exclusionary
Data Available to a User with Roles R1,R2 is Equals to the Data Available to R1 + the Data Available to R2 in General but NOT Always
The execution engine only considers the privileges in roles with execution privilege on the query views
Limit Indirect Access to Views Shared with Other Development Teams
Limit Indirect Access to Shared Views Using Global Security Policies
Limit Indirect Access to Shared Views Using the Indirect Access Privilege
Administration of Databases, Users, Roles and Their Access Rights
Creating Databases
Configuring and Deleting Databases
Creating Users
Modifying and Deleting Users
Creating Roles
Privilege Auditing
Logging Details when Processing Privileges
Impersonation
Global Security Policies
Tags Management
Removing Tags
Privileges Required to Manage and Assign/Unassign Tags
Global Security Policy
Global Security Policies Management
Creating a Global Security Policy
Privileges
Import Tags from External Catalog
Monitoring the Virtual DataPort Server
Monitoring with a Java Management Extensions (JMX) Agent
Using JavaTM VisualVM
General Information on the Server
Information on Data Sources
Information on the Cache
Information and Events on Catalog Access (DDL Statements)
Information and Events on the Running of Statements
Information and Events on Transactions
Setting the User Agent of an Application
Setting Cloud Infrastructure Properties
Denodo Monitor
Local Monitors
Server Monitors
Virtual DataPort Monitors
Configuring the Denodo Monitor
Launching the Denodo Monitor
FinOps
Cluster Architectures / Server Backup
Denodo Tools
How to Check If a Virtual DataPort Server Is Alive
Using the Ping Script
Alternatives to the Ping Script
Using the Import/Export Scripts for Backup And/or Replication
Export Script
Import Script
Data Lake Storage Management
CSV, JSON and XML File Formats
Apache Parquet File Format
Delta Lake Table Format
Iceberg Table Format
Bulk Data Load
AlloyDB for PostgreSQL
Amazon Athena
Amazon Aurora MySQL
Amazon Aurora PostgreSQL
Amazon Redshift
DB2
Databricks
Denodo Embedded MPP
Iceberg Tables
Configure the Denodo Server to Use the Embedded MPP as Cache Using Iceberg Tables
Create Iceberg Tables in the Denodo Embedded MPP
Bulk Data Loading into Iceberg Tables
Create Iceberg Tables in the Denodo MPP or Presto Data Source When the Catalog Name is not Iceberg
Troubleshooting
BigQuery
Hive
Impala
Data Load to Impala Using Built-in Implementation
Data Load to Impala Using External Hadoop Installation
Kudu
Steps Virtual DataPort Performs to Bulk Load Data into Kudu
MySQL
Oracle
PostgreSQL
PrestoDB/Trino
Snowflake
Spark
Microsoft SQL Server and Azure Synapse SQL
Teradata
Yellowbrick
Settings of the Generation of the Temporary Files
Generation of Temporary Files for a Hadoop-compatible Distributed Object Storage
Bulk Data Load on a Distributed Object Storage like HDFS, S3 or ADLS
Built-in Implementation
External Hadoop Installation
Optimizing Queries
Optimizing DF Data Sources
Optimizing Join Operations
Merge Join
Nested Join
Nested Parallel Join
Hash Join
Automatic Simplification of Queries
Removing Redundant Branches of Queries (Partitioned Unions)
Pushing Down GROUP BY Views Below JOIN Views
Pushing Down GROUP BY Views Below UNION Views
Selecting the Most Optimal Source When the Data Is Replicated in Several Sources
Cost-Based Optimization
Enabling the Cost-Based Optimization
Gathering the Statistics of Views
Tuning the Cost-Based Optimization Process
Current Limitations of the Cost-Based Optimization Process
Data Movement
Data Movements From/To Netezza Databases
Examples of Data Movement
Known Limitations
Forcing the Data Movement of a View
Options of the CONTEXT Clause that Control a Data Movement
Customize the Data Movement Table Creation Command
Parallel Processing
Embedded MPP Acceleration
Parallel Processing Using an External MPP
Use of an MPP Engine as Cache
Force Movement of Views to the MPP Engine Using the CONTEXT Clause
Window Functions
Execution of Window Functions
Smart Query Acceleration Using Summaries
Managing Summaries
Summary Promotions
Summary Rewrite Optimization
Automatic Summary Recommendations
Denodo Embedded MPP
Sizing Recommendations for the Denodo Embedded MPP
Promoting the Denodo Embedded MPP
Embedded MPP Extended Execution Trace
MPP Route Plan
MPP Execution Plan
MPP Stage Plan
MPP Stage Plan Operation Nodes
Errors in the Extended Execution Trace
Import Model
Overview
Data Modeling
RDFS and OWL Ontologies
Usage
Limitations
Tags Supported by the Import Model Tool
Appendix
Avoiding SQL Injections
Backward Compatibility Between the Virtual DataPort Server and Its Clients
Configuring the Logging System
Obfuscation of Sensitive Information of Requests
Configuring the Network Interface Restriction in the Denodo Platform
Considerations When Configuring Data Sources with Pass-Through Credentials
Web Services
Data Sources That Support Merge Data
Using the Databricks API and DBFS for Bulk Loads
Mounting External File Systems on the DBFS
Change Password Script (Design Studio)
Enable the “Export” Endpoint of the Web Container
Execution Trace Information
Configure Export Key Script
Importing the Certificates of Data Sources (SSL/TLS Connections)
Installing the Connector for Oracle Essbase
Requirements of the Essbase 11 Adapter
Installing the JMS Connectors to Create JMS Listeners and Web Services with SOAP Over JMS
JMS Client Jars
Installing the SAP Java Connector
Installing the Connector
Testing the Connector on Windows
Testing the Connector on Linux
Grant Privileges in SAP for BAPI Data Sources
Grant Privileges in SAP BW for Multidimensional Data Sources
JMS Connection Details: JNDI Properties
Apache ActiveMQ 6.1.x
IBM WebSphere MQ 7.0
Progress SonicMQ 8.0
Launching Several Virtual DataPort Instances of the Same Installation
How to Configure the Secondary Instances
Deploying Web Services on This Configuration
Creating Multidimensional Base Views Over a Multidimensional Data Source
“Include Empty Rows” Option
“Leaf Members Only” Option
“Leaf Label Only” Option
“Member Captions” Option
“Technical Keys” Option
“Member Leaves” Option
“Include Member Keys” Option
SAP Variables
SAP Metadata Cache
Mapping Multidimensional Data to a Relational Model
Creating Multidimensional Base Views Over a Multidimensional Data Source
Mapping the Result of MDX Queries
Mapping the Result of MDX Queries
OpenAPI 3 Data Sources FAQ
Resource Manager: Available Fields to Evaluate a Rule
Examples of Resource Manager Rules
Restriction of Number of Data Sources
Single User Mode
Explicit Single User Mode
Automatic Single User Mode
Supported JDBC Data Sources
Transforming Incoming/Outgoing Soap/Rest Messages with XSLT Stylesheets
Useful Tools to Debug Issues with Active Directory or Other LDAP Servers
Values of the Attribute “Access Interface”
Change Password Script (VDP)
Virtual DataPort Developer Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
Examples
Access Through JDBC
JDBC Driver Parameters
Connecting to Virtual DataPort Through a Load Balancer
Connecting to Virtual DataPort Using Kerberos Authentication
When the Client Application Does Not Belong to the Domain
Connecting to Virtual DataPort Using OAuth Authentication
Details of the JDBC Interface
Description of Views and Their Fields
Retrieving the Content Type of Blob Values
Working with Datetime Values with the Denodo JDBC Driver
Obtaining the Names of Elements Inside a Struct (Register)
Connecting from Applications that Filter Incoming Serialization Data
Queries with Datetime Literals
Get Auto-Generated Keys in IDU Statements
Enabling JDBC Logging
Access Through ODBC
Configuration of the ODBC Driver on Windows
Install the ODBC Driver on Windows
Set Up a DSN on Windows
Install Multiple Versions of the ODBC Driver on Windows
Configuration of the ODBC Driver in Linux and Other UNIX
Obtain the Appropriate ODBC Driver
Install UnixODBC
Register the Denodo ODBC Driver in UnixODBC
Register a Data Source (DSN) on UnixODBC
Compiling UnixODBC
Troubleshooting Issues
Creating a DSN-Less Connection
Use Kerberos Authentication
Use OAuth Authentication
Details of the ODBC Interface
How the ODBC Interface Reports the Datetime and Interval Data Types
Integration with Third-Party Applications
Supporting Queries with Brackets
Maximum Length of Text Values
Maximum Length of Error Messages
Disabling Transactions
Limit the Number of Incoming ODBC Connections
Backward Compatibility of the ODBC Driver
Access Through an ADO.NET Data Provider
Using Kerberos Authentication
Access Through OLE DB
Developing Extensions
Developing Custom Functions
Creating Custom Functions with Annotations
Creating Custom Functions Using Name Conventions
Compound Types
Custom Function Return Type
Getting Information About the Context of the Query
Dealing with Datetime and Interval Types
Developing Java Stored Procedures
Using Datetime Values in Denodo Stored Procedures
Required Libraries to Develop Stored Procedures
Developing VQL Stored Procedures
Developing Custom Wrappers
Required Libraries to Develop Custom Wrappers
Extending AbstractCustomWrapper
Overriding AbstractCustomWrapper
Dealing with Conditions
Dealing with the ORDER BY Clause
Configuring a Custom Wrapper
Updating the Custom Wrapper Plan
Dealing with Datetime and Interval Types
Developing Custom Input Filters
Required Libraries to Develop Custom Filters
Developing Custom Filters
Custom View Policies
Developing a Custom View Policy
Appendix
Output Schema of the LIST Command
Output Schema of the DESC Commands
Error Codes Returned by Virtual DataPort
Compile the Denodo ODBC Driver
Pre-requisites to Compile the ODBC Driver on Linux
Compile the ODBC Driver (Standard Method)
Compile the ODBC Driver to Obtain Forwardable Tickets
Troubleshooting the Compilation of the Denodo ODBC Driver
Virtual DataPort VQL Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
General Overview of Virtual DataPort
Creating or Defining Data
Defining Base Views
Defining Data Sources and Wrappers
Defining the Views of the Global Schema
Executing Statements
Language for Defining and Processing Data: VQL
Element Names (Identifiers)
Using Back Quotes
Object Descriptions
Data Types
Data Types for Dates, Timestamps and Intervals
Internationalization
Statements
Logical Operators
Comparison Operators
Comparing Literals
Three-valued Logic
Concatenation Operator
Functions for Conditions and Derived Attributes
Syntax Conventions
Syntax of Functions and Condition Values
Comments
Functions
Aggregation Functions
AVG
COUNT
FIRST
GROUP_CONCAT
LAST
LIST
MAX
MEDIAN
MIN
NEST
STDEV
STDEVP
SUM
VAR
VARP
Conversion Functions
ARRAY_TO_STRING
CAST
CREATETYPEFROMXML
REGISTER
Datetime Functions
ADDDAY
ADDHOUR
ADDMILLIS
ADDMINUTE
ADDMONTH
ADDSECOND
ADDWEEK
ADDYEAR
AT TIME ZONE
CONVERT_TIMEZONE
CURRENT_DATE
CURRENT_TIMESTAMP
EXTRACT
FIRSTDAYOFMONTH
FIRSTDAYOFWEEK
FORMATDATE
GETDAY
GETDAYOFWEEK
GETDAYOFYEAR
GETDAYSBETWEEN
GETHOUR
GETMILLISECOND
GETMINUTE
GETMONTH
GETMONTHSBETWEEN
GETQUARTER
GETSECOND
GETTIMEFROMMILLIS
GETTIMEINMILLIS
GETWEEK
GETYEAR
LASTDAYOFMONTH
LASTDAYOFWEEK
LOCALTIME
LOCALTIMESTAMP
MAX
MIN
NEXTWEEKDAY
NOW
PREVIOUSWEEKDAY
SUBTRACT
TO_DATE
TO_LOCALDATE
TO_TIME
TO_TIMESTAMP
TO_TIMESTAMPTZ
TRUNC
JSON Functions
AVRO_TO_JSON
COMPLEX_TYPE_TO_JSON
JSONPATH
JSON_TO_AVRO
JSON_TO_COMPLEX_TYPE
Numeric Functions
ABS
ACOS
ASIN
ATAN
ATAN2
CEIL
COS
COT
DEGREES
DIV
EXP
FLOOR
LN
LOG
MAX
MIN
MOD
MULT
PI
POWER
RADIANS
RAND
ROUND
SIGN
SIN
SQRT
SUBTRACT
SUM
TAN
TRUNC
Other Functions
COALESCE
CONTEXTUALSUMMARY
GETVAR
GETSESSION
HASH
MAP
NULLIF
PIVOTREGISTER
ROWNUM
Spatial Functions
ST_WKBTOWKT
Text Functions
ASCII
BASE64_TO_HEX
CHAR
CONCAT
CONCATLIST
DELETESPACES
HEX_TO_BASE64
INSTR
LEN
LOWER
LTRIM
MAX
MIN
POSITION
PRINTF
PROPERCASE
REGEXP
REGEXP_COUNT
REMOVEACCENTS
REPEAT
REPLACE
REPLACEMAP
RTRIM
SIMILARITY
SPLIT
SUBSTRING / SUBSTR
TRIM
UPPER
Window Functions
AVG
COUNT
CUME_DIST
DENSE_RANK
FIRST_VALUE
LAG
LAST_VALUE
LEAD
LISTAGG
MAX
MIN
NTILE
PERCENTILE_CONT
PERCENTILE_DISC
PERCENT_RANK
RANK
ROW_NUMBER
SUM
STDEV
XML Functions
XMLQUERY
XPATH
XSLT
Converting XML Data into Virtual DataPort Compound Types
Generating Wrappers and Data Sources
Creating Data Sources
JDBC Data Sources
ODBC Data Sources
Multidimensional Data Sources
Data Sources for SOAP Web Services
XML Data Sources
JSON Sources
DF Data Sources
LDAP Data Sources
BAPI Data Sources
Salesforce Data Sources
MongoDB Data Sources
Custom Data Sources
Data Source Configuration Properties
Specifying Paths in Virtual DataPort
Creating Wrappers
Execution Context and Interpolation Strings
Wrapper Metadata
JDBC Wrappers
Multidimensional Databases Wrappers
ODBC Wrappers
Web Services Wrappers
XML Wrappers
JSON Wrappers
DF Wrappers
LDAP Wrappers
BAPI Wrappers
Salesforce Wrappers
MongoDB Wrappers
CUSTOM Wrappers
Wrapper Configuration Properties
Valid Conversions Between Types in Wrappers and VDP Types
Native-type Conversions of a Wrapper to Java Types
Filters
Creating a Base View
Query Capabilities: Search Methods and Wrappers
Query Constraints
Assigning Wrappers to Search Methods
Example of How a Search Method Is Created
Modifying a Base View
Queries: SELECT Statement
FROM Clause
JOIN Operation
INTERSECT Operation
MINUS Operation
FLATTEN View (Flattening Data Structures)
Subqueries
WITH Clause
SELECT Clause
Derived Attributes
WHERE Clause
Conditions with Compound Values
Group BY Clause
Use of Aggregation Functions
HAVING Clause
UNION Clause
ORDER BY Clause
OFFSET, FETCH and LIMIT
CONTEXT Clause
TRACE Clause
CASE Expressions
CASE Clause Examples
Defining a Derived View
Modifying a Derived View
Defining an Interface View
Defining the Statistics of a View
Materialized Tables
Creating Materialized Tables
Inserting Data into Materialized Tables
Temporary Tables
Creating Temporary Tables
Privileges Required
Remote Tables
CREATE REMOTE TABLE Command
Summaries
CREATE SUMMARY VIEW Command
Inserts, Updates and Deletes Over Views
INSERT Statement
UPDATE Statement
DELETE Statement
Return Modified Rows
Limitations
Use of WITH CHECK OPTION
Refreshing Views
REFRESH Statement
Transactions in Virtual DataPort
Stored Procedures
Importing a Stored Procedure
Invoking Denodo Stored Procedures
Predefined Stored Procedures
CACHE_CONTENT
CATALOG_ELEMENTS (deprecated)
CATALOG_FKS (deprecated)
CATALOG_METADATA_VIEWS
CATALOG_PERMISSIONS
CATALOG_PKS (deprecated)
CATALOG_VDP_METADATA_VIEWS
CATALOG_VIEWS (deprecated)
CHECK_METADATA
CLEAN_CACHE_DATABASE
COLUMN_DEPENDENCIES
COMPACT_CACHE
COMPACT_HADOOP_CACHE (deprecated)
COMPACT_METADATA_TABLES
COMPUTE_SOURCE_TABLE_STATS
CREATE_REMOTE_TABLE
CREATE_SCHEMA_ON_SOURCE
CREATE_TAGS_FROM_COLLIBRA
CREATE_TAGS_FROM_VIEW
DROP_REMOTE_TABLE
DROP_SCHEMA_ON_SOURCE
DUAL
GENERATE_SMART_STATS_FOR_FIELDS (deprecated)
GENERATE_SMART_STATS_FOR_FIELDS_BY_TABLENAME
GENERATE_STATS (deprecated)
GENERATE_STATS_FOR_FIELDS (deprecated)
GENERATE_VQL_TO_CREATE_JDBC_BASE_VIEW
GET_ACTIVE_LOGGERS
GET_ASSOCIATIONS
GET_AVAILABLE_STATS_MODES
GET_CACHE_COLUMNS
GET_CACHE_CONFIGURATION
GET_CACHE_TABLE
GET_CATALOG_EFFECTIVE_PERMISSIONS
GET_CATALOG_METADATA_WS
GET_DATABASES
GET_DELEGATED_SQLSENTENCE
GET_ELEMENTS
GET_EXPORTED_KEYS
GET_FOREIGN_KEYS
GET_GLOBAL_SECURITY_POLICIES_TAGS
GET_ICEBERG_VIEW_SNAPSHOTS
GET_JDBC_DATASOURCE_TABLES
GET_PARAMETER
GET_PRIMARY_KEYS
GET_PROCEDURE_COLUMNS
GET_PUBLIC_VIEW_DEPENDENCIES
GET_SERVER_CONNECTIVITY
GET_SESSIONS
GET_SOURCE_CHANGES
GET_SOURCE_COLUMNS
GET_SOURCE_TABLE
GET_STATS_FOR_FIELDS
GET_TYPE_ATTRIBUTES
GET_USER_ACCOUNTS
GET_USER_DEFINED_TYPES
GET_VIEW_COLUMNS
GET_VIEW_INDEXES
GET_VIEW_STATISTICS
GET_VIEW_TAGS
GET_VIEWS
LIST_JDBC_DATASOURCE_TABLES (deprecated)
LOGCONTROLLER
MAINTAIN_METADATA_TABLES
MIGRATE_DATE_TYPES
PING_DATA_SOURCE
REFRESH_SOURCE_TABLE_METADATA
ROLLBACK_ICEBERG_VIEW_TO_SNAPSHOT
SOURCE_CHANGES (deprecated)
VALIDATE_MPP_LICENSE
VIEW_DEPENDENCIES
WAIT
WEBCONTAINER_ELEMENTS (deprecated)
WEBCONTAINER_ELEMENT_STATUS
WEBCONTAINER_META_INF
WRITELOGERROR
WRITELOGINFO
Tags
Defining Other Elements of the Catalog
Creating Folders
Defining a Data Type
Defining a Map
Defining JAR Extensions
Defining JMS Listeners
Defining Kafka Listeners
Creating Databases, Users, Roles and Access Privileges
Creating and Modifying Virtual DataPort Databases
Managing Users
Modifying and Deleting Users
Connecting to Another Database
Modifying the Privileges of a User
Managing User Roles
Ownership of Elements
Global Security Policies
Describing Catalog Elements
Exporting Metadata
Importing Metadata
Listing Elements in the Catalog
Removing Elements from the Catalog
Publication of Web Services
Creating SOAP Web Services
Modifying SOAP Web Services
Creating REST Web Services
Modifying REST Web Services
Connection Parameters
Web Services Authentication
Basic and Digest
SAML 2.0
VDP
Embedded Web Container Management
Deploying and Exporting SOAP and REST Web Services
RESTful Architecture
Associations
Navigational Queries
Help Command
Version Control Systems Integration Commands
Statements to Work with GIT
DVCSCOMMIT
DVCSPUSH
DVCSPULL
DVCSREVERT
VCSCONTENTS
VCSSHOW
REVISIONS
GET_REVISION
UNRESOLVED_ELEMENT
Commands for Environment Management
Other Version Control Commands
VCSDISCARD
Resource Manager
Managing the Plans of the Resource Manager
Managing the Rules of the Resource Manager
Advanced Characteristics
Management of Compound Values
Processing of Compound Types: Example
Changing Settings of Virtual DataPort and the Web Container
Using the Cache
Cache Invalidation
Cache Table Recreation
Displaying the Contents of the Cache
Cache Table Creation Templates
Test Table Creation Templates
Configuring Swapping Policies
Managing Internationalization Configurations
Replacing an I18n Map
Deleting an I18n Map
Execution Context of a Query and Interpolation Strings
Adding Variables to Selection Conditions (GETVAR and SETVAR)
Appendix
Date and Time Pattern Strings
Virtual DataPort API Javadoc
Embedded MPP Guide
Preface
Audience
Overview
Architecture
Kubernetes Deployment
Configuration
Container Image Registry
Container Image Registry Credentials
Private Container Image Registry
External Access to Embedded MPP
Ingress
SSL/TLS
Workers, CPU and Memory
Memory Settings for Queries
CPU and Memory Management in Kubernetes
Denodo Server
Catalogs
Hive
Delta Lake
Iceberg
Embedded Hive Metastore
CPU and Memory Management in Kubernetes
Embedded PostgreSQL
Persistent Volume
Backup
CPU and Memory Management in Kubernetes
External Metastore
External Hive Metastore
AWS Glue Data Catalog
Scaling
Autoscaling
Manually
Resource Groups
Deployment
Object Storage
AWS S3
S3-compatible Storage
Azure Data Lake Gen 2
Azure Blob Storage
Google Cloud Storage
Embedded MPP at Denodo
Bulk Data Load with the Denodo MPP
Configuration
Cost Based Optimization
Upgrade to a New Version
Upgrade to Embedded MPP 9
Cleanup
Troubleshooting
Troubleshooting Deployment
Troubleshooting Usage
Appendix
Other Uses of the Denodo Embedded MPP
Connect from Denodo
Bulk Data Load with the Denodo MPP
Metadata Discovery Tool
Cost Based Optimization
Hive Tables Syntax and Properties
Partition Tables
Delta Lake Tables Properties
Iceberg Tables Properties
Data Catalog Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
Installation and Execution
Launching the Data Catalog
Introducing the Data Catalog
Search
Metadata Search
Metadata Search in Index
Content Search
Browse
Databases
Categories
Tags
Databases
Summary of Databases
Requests of Databases
Associations of Databases
Views
Summary of Views
Schema of Views
Query of Views
Building a Query
Executing a Query
Taking Advantage of Associations
Data Preparation
Loading Data
Preparing Data
Executing the Query
Assisted Query
Generating a Query
Executing the Generated Query
Other Options
Assisted Query Role
Associations of Views
Lineage of Views
Requests of Views
Usage of Views
Data Profiling of Views
Search of Views
Web Services
Summary of Web Services
Schema of Web Services
Schema of SOAP Web Services
Schema of REST Web Services
Query of Web Services
Building a Query
Executing a Query
Taking Advantage of Associations
Requests of Web Services
Usage of Web Services
Workflow Requests
My Requests
Request Management
My Notifications
Administrator Notifications
Email Templates
Queries
My Queries
Deploying Saved Queries
Sharing Saved Queries
Exporting and Importing Saved Queries
VQL Shell
User Settings
Locale Configuration
My Requests
My Notifications
Admin Notifications
Accessibility Mode
Authentication
Authentication with Login and Password
Single Sign-On with Kerberos
Single Sign-On with an Identity Provider
Local Authentication
Authorization
Privileges on the Data Catalog
Catalog Management
Administration
Collaboration
User
Users with Predefined Privileges
Data Catalog Editor
Data Catalog Classifier
Data Catalog Manager
Data Preparation
Data Catalog Content Administrator
Data Catalog Administrator
Data Catalog Exporter
Data Catalog Global Administrator
Data Catalog Local User
Data Catalog Assisted Query User
Administration
Server Set-Up
Configure the Virtual DataPort Servers
Configure the Connection Settings to the Virtual DataPort Servers
Enable Single Sign-On with Kerberos
Use an External Database for the Data Catalog
Configure the Permissions
Configure the AI Integration
Search Engines
Index Servers
Content Search
Metadata Search
Personalization
Elements
Information
Export
Connection
Usage Statistics
Data Preparation
Theme
Requests
Categories Management
Create Categories
Edit Categories
Remove Categories
Assign Categories
Tags Management
Create Tags
Edit Tags
Remove Tags
Assign Tags
Elements Management
Property Groups Management
Manage Property Groups
Manage the Custom Properties of a Property Group
Assign Property Groups
Import and Export Data Catalog Metadata
Export Metadata
Import Metadata
Synchronize with Virtual DataPort
Appendix
Installing the Data Catalog on a Cluster: Sharing the Same Settings Across All the Nodes
Setting Up the Common Database
Configure the Data Catalog to Use the Common Database
REST API
HTTP Basic Authentication (stateless)
OAuth Token (stateless)
OAuth Token (Stateful)
CORS Configuration
Apache Lucene Search Syntax
Terms
Fields
Term Modifiers
Boolean Operators
Grouping
Field Grouping
Escaping Special Characters
Data Catalog Scripts
Change Password Script
Configure Export Key Script
Export Metadata Script
Import Metadata Script
Change Database Settings Script
Diagnostic & Monitoring Tool Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
General Architecture
Installation and Execution
Launching the Diagnostic & Monitoring Tool of the Denodo Platform
Launching the Diagnostic & Monitoring Tool of the Denodo Solution Manager
Introducing the Diagnostic & Monitoring Tool
Authentication
Authorization
Creating Servers and Environments
Creating Servers
Creating Environments
Monitoring
Monitoring Servers
Monitoring - Resources
Monitoring - State
Monitoring - Sessions
Monitoring - Requests
Monitoring - Cache
Monitoring - Data Sources
Monitoring - Threads
Monitoring Environments
Overview
Creating Diagnostics and Diagnostic Intervals
Creating Diagnostics
Creating Diagnostic Intervals
Diagnosing
Diagnosing - Resources
Diagnosing - State
Diagnosing - Sessions
Diagnosing - Requests
Diagnosing - Cache
Diagnosing - Data Sources
Diagnosing - Threads
Diagnosing - Errors
Export and Import
Export
Import
Configuration
User Configuration
Configuration - General
Configuration - Overview
Configuration - Resources
Configuration - State
Configuration - Sessions
Configuration - Requests
Configuration - Cache
Configuration - Data Sources
Configuration - Threads
Configuration - Errors
Server Configuration
Informative Message
Database Configuration
Scheduler Administration Guide
Preface
Scope
Who Should Use This Manual
Summary of Contents
Introduction
General Architecture
Installation and Execution
Administration
Authentication
Scheduler Server Local Authentication
Web Administration Tool Local Authentication
Web Configuration
Kerberos Configuration
Informative Message Configuration
Change Password
Community Resources Configuration
Scheduler Server Configuration
Server Set-up
Management
Import/Export Backups
Index Server
Log Configuration
Creating and Scheduling Jobs
Jobs
Data Sources
Scheduler Index Data Sources
CSV Data Sources
Elasticsearch Sources
JDBC Data Sources
VDP Data Sources
Configuring New Jobs
General Structure of a Job
VDP Extraction Section
VDPCache Extraction Section
VDPDataLoad Extraction Section
VDPIndexer Extraction Section
Exporters Section
Retry Section
Handler Section
Time-based Job Scheduling Section (Triggers Section)
Reports Section
Dependencies Among Jobs
Developer API
Scheduler RMI Client API
Scheduler REST Client API
Public
Private (Deprecated)
Extensions (Plugins)
Exporters
Handlers
Appendix
JDBC Drivers
Use of Configure Scheduler Export Key Script
Use of the Import/Export Scripts for Backup (Scheduler)
Export
Import
Use of the Import/Export Scripts for Backup (Scheduler Index)
Export
Import
Use of the Ping Script (Scheduler)
Use of the Ping Script (Scheduler Index)
ChangePassword Script (Scheduler)
ChangePassword Script (Scheduler Web Administration Tool)
ChangePassword Script (Scheduler Index)
How to Manually Configure the Metadata Database
Scheduler API Javadoc
Solution Manager Installation Guide
Preface
Who Should Use This Document
Pre-Installation Tasks
Hardware Requirements
Minimum Requirements to Run the Solution Manager On Premise
Minimum Requirements to Run the Solution Manager on Amazon AWS
Recommendations to Run the Denodo Platform on Microsoft Azure
Software Requirements
Supported Operating Systems
Supported Browsers
Other Pre-installation Tasks
Select a User Account to Install the Solution Manager
Check that the Required Ports Are Free
Grant Privileges to This User Account
Check the PATH Environment Variable on Windows
Download the Installer
Using the Graphical Installation Wizard
Settings you can Modify when you Select the Custom Installation
Common to All Modules
Virtual DataPort
Solution Manager
Using the Command Line Installer
Unattended Installation of the Solution Manager
Modifying the Solution Manager Installer to Include the Latest Update
Unattended Installation of the Solution Manager
Post-Installation Tasks
Replicate the Encryption Key Across All the Installations
If You Already Started One of the Components of the Denodo Platform / Solution Manager
Install the Latest Update
Install the License
Change the Default Passwords
Solution Manager Server
Design Studio
Enable SSL/TLS in the Solution Manager
Obtaining and Installing an SSL Certificate
Enabling SSL/TLS in Solution Manager Servers
Configure Windows Services
High Availability
Step #1: Configuring the Solution Manager Installations for High Availability
Step #2: Configuring the Load Balancer for High Availability
Step #3: Configuring Access Through the Load Balancer
JVM Memory Configuration
Denodo Platform Control Center
Starting Platform Servers and Tools
Installing Updates and Hotfixes
Virtual Machine and Web Container Configuration
Configuration of the JVM Parameters from the Command Line
Control Center Help
Uninstalling the Solution Manager
Appendix
Configuring the Network Interface Restriction in the Solution Manager
Default Ports Used by the Solution Manager Modules
Transparent Metadata Encryption in the Solution Manager
Providing a Krb5 File for Kerberos Authentication
Session Configuration (Solution Manager Administration Tool, Design Studio, Scheduler, Diagnostic & Monitoring Tool)
Maximum Session Timeout
Idle Session Timeout
Single-User Concurrent Sessions Control
Troubleshooting the Solution Manager Installer
Solution Manager Administration Guide
Preface
Scope
Who Should Use This Document
Summary of Contents
Introduction
Architecture
Starting the Solution Manager
Introducing the Solution Manager Administration Tool
License Management
Install a License
Check Global License Information
Assign Licenses
Assign a License to a Server
Check the License Information for an Environment
How Licenses Work
How a Server Resolves Its License
How Denodo Servers Behave on License Responses
License Usages Table
Configuration
Setting-Up an External Database
Changing the Password of the External Database
VCS Configuration
Setting-Up the Informative Message
Global Monitoring Configuration
Cloud Storage Appenders
JDBC Appenders
Authentication and Authorization
User Management
Authenticating with LDAP
Enabling the LDAP Roles Cache in Solution Manager
Authenticating with Single Sign-On
SAML Configuration
OAuth Configuration
OpenID Configuration
Kerberos Configuration
Authorization
Global Privileges
Privileges Granted to a Role For an Environment
Role Management
Import Roles from an LDAP Server
Denodo Security Token
Architecture
Configure the Denodo Platform Components to Reach the Security Token Server
Security Considerations
External Identity Provider Considerations
Configure the Connection to the Security Token Server from the Denodo Platform
Configure HTTP Timeouts and Limits for the Request of Public Keys
Load Balancer Considerations
Authentication Credentials
Regenerate Autogenerated Credentials
Single Sign-On Sequence
Environments
Clusters
Working Modes
Differences Wizard
Standard Mode
Creating Environments
Configuring Environments
Creating Standard Clusters
Configuring Clusters
Automated Cloud Mode
Configuring Mode
General
Server Access
Default Region & Images
Creating Environments
Automated Cloud Mode (AWS)
Automated Cloud Mode (Azure)
Configuring Environments
AWS Environment
Azure Environment
Creating Clusters
New AWS Cluster
New Azure Cluster
Configuring Clusters
AWS Cluster
Azure Cluster
Managing Clusters
View the Cluster Summary
View the Progress of the Last Action
Enable a Cluster
Disable a Cluster
View All Cluster Events
Start a Cluster
Stop a Cluster
Restart a Cluster
Recreate a Cluster from an Image
Recreate a Cluster from a Server
Install a Denodo Update in a Cluster
Monitor the Cluster with the Diagnostic & Monitoring Tool
Environment Configuration
Configuring Virtual DataPort Properties
Configuring Deployments
Standard Mode Environments Deployments Configuration
Cloud Environments Deployments Configuration
Data Catalog Server Synchronization
Configuring Deployment Scripts (Standard Mode)
Examples of Deployment Scripts
Configuring Environment Logging Level
Configuring Environment Monitoring
Clusters Configuration
Configuring Cluster Load Balancing Variables (Standard Mode)
Configuring Scheduler Properties
Configuring Cluster Logging Level
Servers
Creating Servers
Configuring Servers
Configuring Server Load Balancing Variables (Standard Mode)
Configuring Server Logging Level
Load Balancing Variables
Creating Load Balancing Variables
Assigning Values to Loading Balancing Variables
Promotions
Creating Revisions
General Options
Additional Options
Revision Candidate Elements
Create Revisions from VQL
Create Revisions from Compare Environments Wizard
Compare Environments Wizard
Revision Candidate Elements
Creating Revisions from Compare Environments Wizard
Revisions Table
Create a Revision
Edit a Revision
Edit a Revision from the Compare Environments Wizard
See the Information of a Revision
Remove Revisions
Filter Revisions in the Table
Download the Metadata of a Revision
Validate a Revision
Check the Validation Summary of a Revision
Copy Revision
Deploy Revisions
Check the Environments Where a Revision Was Deployed
Standard Mode Deployments
Deployment Preconditions
Deployment Paradigms
Cache Considerations
Cloud Deployments
Cloud Deployments Options
Cache Considerations
Considerations for Deployments
Cache
Summaries
Denodo Embedded MPP
Deployments Table
Cancel Deployments
Remove Deployments
Filter Deployments in the Table
Check the Progress Summary of a Deployment
See the Errors of a Deployment
See the Revisions of a Deployment
Download Backup Data for a Deployment
Deployment Progress
Monitoring
Monitoring Configuration
Monitoring Virtual DataPort Servers Running Behind a Load Balancer
Import and Export
Import from Solution Manager 8.0
Appendix
REST API
Get the List of Environments
Create an Environment
Get the List of Licenses Available
Update an Environment
Delete an Environment
Update AWS Global Credentials
Update AWS Environment Credentials
Update Azure Global Credentials
Update Azure Environment Credentials
Start Automated Mode Cluster
Stop Automated Mode Cluster
Restart Automated Mode Cluster
Recreate Automated Mode Cluster
Recreate From Servers Automated Mode Cluster
Install Denodo Update Automated Mode Cluster
Get the List of Virtual DataPort Properties Associated to an Environment
Create and Delete Environment Properties
Get the List of Clusters
Start Environment Monitoring
Stop Environment Monitoring
Create a Cluster
Get a Cluster
Get Cluster Status
Update a Cluster
Delete a Cluster
Get the List of Scheduler Properties Associated to a Cluster
Create and Delete Cluster Properties
Get the List of Servers
Create a Server
Get a Server
Update a Server
Delete a Server
Export Solution Manager Catalog and Configuration
Import Solution Manager Catalog and Configuration
Get the List of Revisions
Create a Revision from a VQL File
Download the VQL of a Revision
Download the Scheduler ZIP of a Revision
Get the List of Deployments
Start a New Deployment from a List of Revisions
Get the Progress of a Deployment
Free License Usage
Free License Usage by Server Id
Ping License Manager Server
Ping to Solution Manager Server
Best Practices: Deployment of Updates Across Your Organization
Updating Solution Manager with High Availability
Solution Manager Compatibility with the Denodo Platform Servers
Creating A Custom AWS AMI for the Denodo Platform
Creating A Custom Azure Image for the Denodo Platform
Suggested Reading Materials to Use the Automated Cloud Mode
Architecture of a Deployment of Solution Manager in AWS
Architecture of a Deployment of Solution Manager in Azure
Best Practices for Automated Mode Environments
Maximizing Uptime and High Availability
Infrastructure Deployment Considerations
Health Checks
Backup and Recovery
Restoring Backup
Maintenance
Automated Cloud Mode - Creating a Backup
Creating an IAM Policy
Main Features
Automated VPC Management
Automated Subnet Management
Automated Auto Scaling Group Management
Automated Security Group Management
Creating an Azure Custom Role
Main Features
Automated Resource Group Management
Automated Virtual Network Management
Automated Subnet Management
Automated Network Security Group Management
Minimizing Downtime Option
Automated Auto Scale Set Management
Configuration Files of the Monitoring in Solution Manager
JDBC Logging Configuration
Cloud Storing Configuration
DenodoConnect Components
Denodo Custom Wrapper for IBM Cognos TM1 - User Manual
Denodo Custom Wrapper for SAS - User Manual
Denodo Dashboard - User Manual
Denodo Distributed File System Custom Wrapper - User Manual
Denodo Dynamics 365 Business Central Templates - Quick Use Guide
Denodo Dynamics 365 Customer Engagement Templates - Quick Use Guide
Denodo DynamoDB Custom Wrapper - User Manual
Denodo Embedded MPP - User Manual
Denodo Google Sheets Custom Wrapper - User Manual
Denodo Google Sheets Tool - User Manual
Denodo Governance Bridge for Collibra - User Manual
Denodo Governance Bridge for IGC - User Manual
Denodo Governance Bridge for Microsoft Purview - User Manual
Denodo GraphQL Custom Wrapper - User Manual
Denodo HL7 FHIR Service - User Manual
Denodo Hibernate Dialect - User Manual
Denodo Kafka Custom Wrapper - User Manual
Denodo Log Custom Wrapper - User Manual
Denodo OData 2.0 Service - User Manual
Denodo OData2 Custom Wrapper - User Manual
Denodo OData4 Custom Wrapper - User Manual
Denodo Power BI Custom Connector - User Manual
Denodo RFCReadTable Custom Wrapper - User Manual
Denodo SFTP Exported Files Custom Handler - User Manual
Denodo SSH Custom Wrapper - User Manual
Denodo ServiceNow Templates - Quick guide
Denodo Templates for Adobe Analytics - Quick Use Guide
Denodo Templates for Google Analytics - Quick Use Guide
Denodo Templates for Marketo - Quick Use Guide
Denodo Templates for SharePoint - Quick Use Guide
Denodo Testing Tool - User Manual
Denodo XtraFuncs - User Manual
Generic XML Exporter - User Manual
Quick Start Guides
Denodo Enterprise
Denodo 9 Enterprise for AWS Quick Start Guide
Denodo 9 Enterprise for Azure Quick Start Guide
Denodo Enterprise Plus
Denodo 9 Enterprise Plus for AWS Quick Start Guide
Denodo 9 Enterprise Plus for Azure Quick Start Guide
Denodo Platform
Denodo Platform 9 Container Registry Quick Start Guide
Denodo Platform 9 Helm Charts Quick Start Guide
Denodo Platform Container 9 QuickStart Guide
Denodo Platform for AWS 9 BYOL Quick Start Guide
Denodo Express 9 Quick Start Guide
User Manuals
/
Data Catalog Guide
/
Appendix
/
Data Catalog Scripts
Data Catalog Scripts
¶
The following scripts can help you manage the Data Catalog:
Change Password Script
Configure Export Key Script
Export Metadata Script
Import Metadata Script
Change Database Settings Script
Feedback:
Share:
Add feedback
