Start your trail!
Expert Trails cover different specific topics from start to finish. They include information related to this topic through all the resources that we have in Denodo, including user manuals, KB articles, webinars, tutorials, blog posts, training courses, etc.
Cache
Gain knowledge of caching with respect to how to model the cache with the necessary best practices in mind, the different purposes of cache, and advanced configurations of caching.
Learn more >
DataSource Connectivity
Learn different types of options to access the data source, configuration techniques, and some of the advanced features in data source configuration.
Learn more >
Deployment Topology
Understand the necessary knowledge for defining the data virtualization architecture, not only in the design phase, but also you will get expert knowledge on how to implement it.
Learn more >
Development Lifecycle
Leverage the Development Life Cycle to outline, design, develop, test, deploy and maintain any data virtualization project with greater efficiency and improved quality.
Learn more >
Modeling and Metadata Organization
Become an expert on how to do the modeling and how to organize the metadata in any data virtualization project and learn the best practices at the same time.
Learn more >
Monitoring
Get strong knowledge on how to analyze the infrastructure and Denodo services performance by using advanced monitoring capabilities and tools included within the Denodo Platform.
Learn more >
Operations
Become knowledgeable about all the necessary steps to install and maintain the Denodo Platform, including managing updates, deployments, and backup and recovery processes.
Learn more >
Query Performance Optimization
Understand the curated selection of the different resources available on Denodo Platform optimization techniques to become a master in Denodo Query Performance Optimization.
Learn more >
Security
Become an expert by leveraging the security options available in Denodo, controlling security in a single place, and ensuring that the right data is only accessed by the right users.
Learn more >
Cache
Purpose of caching
Some of the cache purposes are: performance, optimizing frequently used queries, minimizing source system impact, protecting against intermittent system availability...
StartCache Configuration
To make use cache it must be configured according the document Configuring the Cache of a View . A Data Refresh Plan periodically and a Cache Maintenance Process must be defined...
StartModeling
Depending upon your scenario it is really important selecting the correct cache mode:
- Full cache: to delegate complex operations like joins, unions..
- Partial Cache: when you do not have all tuples of the view in cache...
Advanced Configuration
Some additional configurations:
- Bulk Data Load: to optimize loading big amounts of data
- Cache Indexes: Indexes improve the speed of data retrieval operations on database tables
- Specific conditions ...
Summaries
Denodo 8 includes smart query acceleration techniques using a new type of view called Summaries
- Users do not need to explicitly reference the cached view to use the precomputed data
- Summaries are not restricted to the cache data source...
Troubleshooting
Identify the possible reasons when a certain problem arises. Monitoring the cache can also help to debug if any issue occurs when load processes of the cache on the Virtual DataPort server is performed...
Start
Data source Connectivity
Different ways to access data in real-time
The first step in a Denodo project is to create data sources to connect to various data repositories, types: JDBC, ODBC, Web Service, Multidimensional DB, XML, JSON, Delimited File, Excel, LDAP, BAPI...
StartWhat do you need to configure the data sources
Let's focus on different configuration parameters required to create data sources:
- JDBC: JDBC driver, driver class, connection URI.
- ODBC Sources the most common connection type is DSN...
Authentication
There are different authentication methods available: using login and password, Use Pass-through session credentials, using kerberos. OAuth 1.0, OAuth 2.0 ...
StartAdvanced features in different data sources
Advanced configurations to optimize the data source:
- Connection pooling: To minimize the cost of opening connections.
- Source Configuration: specifies certain characteristics of the underlying sources...
Error analysis
How to debug the errors that may occur during the data source connection: Modifying the log level, Logging System...
Start
Deployment Topology
Understanding your scenario
To get some inspiration take a look at some Data Virtualization Use Cases
StartDefining your infrastructure architecture
It is time to decide on the infrastructure architecture. Depending on the analysis done in the previous stage, you may be interested in a pure on-premises architecture, cloud, or hybrid architecture...
StartMaking it real!
Once you have decided on the architecture, it is time to install. The first step is installing the Denodo Solution Manager that will manage licenses for all your environments, then install the Denodo Platform servers.
Are you thinking on a cluster or a cloud machine?...
Monitoring your deployment
The Denodo Platform provides different endpoints to perform a Health Check of the different components (Health Monitoring section of the Denodo Platform Installation Guide). Besides, you can monitor the status of the server in real-time using the Diagnostic and Monitoring Tool or the JMX interface or by using the Denodo Monitor..
Start
Development lifecycle
Discover and analyze the business need
The first step is to discuss with all the stakeholders in the organization to identify the current business problems and gather the need, requirements and expectations for building a new system. The next step is to look out for the possible solutions that solve the current problem with the help of Data Virtualization...
StartArchitect and Design the Solution
The first thing is to design the solution as per the business requirement. Phases:
- Technologies to be used.
- Architecture of the system.
- Development methodology that is suitable.
- Team roles & responsibilities
- Documentation...
Build and Test the Solution
The most common environments that would exist in any Data Virtualization project are:
- Development: where the development of new projects will be executed.
- Testing/QA: to validate the suitability of each new iteration of a project.
- Production: the final operational system...
Deployment
Denodo provides tools to manage large deployments with several environments (Dev, Testing, Production), including clustering and geographical distribution. It offers Solution Manager Promotions which is the recommended way for promoting the metadata between environments. On the other hand, it offers Import/Export Scripts for backup purposes..
StartMaintenance
Maintenance of Denodo Platform can include monitoring the performance of the Denodo Platform servers and software updates and upgrades. The Expert Trail: Monitoring provides more details on various monitoring methods available in Denodo Platform...
StartModeling and Metadata Organization
Define the naming convention and metadata organization
The first step of this stage is to define the naming convention that should be used in the data virtualization project. After having an understanding of the different layers and naming conventions, it should be defined how to organize the projects within the virtual databases...
StartTop-down vs bottom-up modeling
In the Bottom-Up approach it is possible to start from the ''bottom'' by creating the data sources and base views and grow from there. The final views used by consumers or business users might not be defined at this point. In the Top-Down modeling, however, it is possible to create interface views which allows to define a view schema for contract-based development. .
StartBuild the Connectivity
The next step is to proceed by creating elements belonging to the first layer: Connectivity. These are the data sources and base views....
StartCombine the data
The next step is to combine and transform base views to new views, which are called derived views or integration views. These views build the foundation for the upper and final views (business views, report views, data services)...
StartConsume the data
The Denodo Platform provides different ways to consume the data from third party clients. The main connectivity methods are through JDBC, ODBC and the RESTful Architecture, which consists of the GraphQL Service, the OData 4.0 service and the RESTful Web Service...
StartMonitoring
Health Monitoring
Denodo provides several scripts, URLs, and methods aimed at each one of the Denodo Platform and Solution Manager components that help you to determine if servers are alive...
StartQuery Monitoring
Denodo Platform supports three different features that allow customers to monitor the queries that a Virtual DataPort server receives: Query Monitor, VDP Logging System...
StartVirtual DataPort Server Monitoring
Denodo divides the VDP server Monitoring into two main sections: real-time monitoring and historical data diagnosis. Besides, for each section, Denodo provides several features as you can see in the following subsections...
StartDenodo Reporting Tools
Denodo provides: Diagnostic & Monitoring Tool and Denodo Monitor Reports to help users to analyze the information stored in the logs gathered by the Denodo Monitor tool and understand the Virtual DataPort server behavior ..
StartOperations
Installing the Solution Manager
Steps to install the Solution Manager which acts as a single point of contact for licenses and managing the Denodo servers,...
StartInstalling the Denodo Platform
Steps to install Denodo Platform that can be downloaded from the Denodo Support site ...
StartInstalling the Latest Update
It is important to install the latest update release of both the installations to take advantage of latest features in the product and also to get rid of the latest bug fixes and security vulnerabilities...
StartDeployment Process
The Solution Manager Admin Tool helps you make Promotions. A promotion takes under consideration all the tasks that you simply ought to perform to migrate changes within the servers' metadata from one environment to another ...
StartUpgrade Process
The process of upgrading to a new major version of the Denodo Platform involves installing the new version in a separate computer - or in a different directory of the same computer - and transferring the metadata and settings from the previous version to the new installation. ...
StartBackup and Recovery Policies
In a Denodo production installation, it is recommended to have a backup server that will be connected to the Load Balancer when some of the main servers are not working for an unexpected reason....
StartMonitoring the Denodo Servers
The Expert Trail: Monitoring provides information on monitoring...
StartResource Management
In an environment with multiple concurrent user sessions that run queries, not all user sessions have the same importance. The Resource Manager allows you to classify sessions into groups based on the attributes of the session...
StartQuery Performance Optimization
Modeling
Best Practices and general guidelines on how to design views that are optimal from a performance point of view focusing on a Big Data/Analytic use case....
StartLeveraging the Optimizer
Denodo Query Optimizer Engine analyzes the metadata and automatically determines the best execution plan to run a query in the most performant way so, all information must be accurate...
StartCaching
Caching in Denodo can be used for several purposes, such as enhancing performance or protecting data sources from costly queries, and/or reusing complex data combinations and transformations....
StartTroubleshooting
learn a method to identify the bottlenecks of a Query in Denodo and the different options and actions to resolve or improve the performance of an existing query...
StartSecurity
Securing Data in Motion; Configuration of Transport Layer Security (TLS)
The first step in making sure that the Denodo Platform is secured is making sure that none of the communications between the Denodo Platform components and other services are decipherable to unwanted listeners...
StartSecuring Data at Rest; Configuring How Data is Stored by the Denodo Platform
The data that Denodo stores during operation must be encrypted. In conjunction with encrypting data in motion, encrypting data at rest will prevent sensitive data in the filesystem or underlying data sources from being exposed...
StartIntegration with a Centralized Authentication Server
Permissions inside the Denodo Platform must be carefully configured to make sure that bad actors do not obtain permission to access the Denodo Platform. Denodo Administrators can configure the Virtual DataPort server to delegate authentication to a Centralized Authentication Server...
StartConfiguring Permissions in Denodo
User and role-based permissions in Denodo allow for the authorization of overall enterprise data to be managed at a single point. Denodo Platform allows LDAP groups to be linked to roles in Denodo, which allows for groups already organized in a central authentication server to be provided permissions in the server...
StartAuditing
The Denodo Platform generates an event for each operation performed on the Virtual DataPort server, which records which users had access to what resources, what changes they made, and when the operations were executed. ...
Start