New Features of Virtual DataPort 8.0

This section lists the new features of Virtual DataPort 8.0.

Design Studio

Virtual DataPort has a new web interface for developers: the Design Studio.

The Design Studio is aimed at developers that create data sources, base views, derived views, publish web services, etc. Its layout is similar to the Administration Tool.

Developers can use the Design Studio or keep using the Administration Tool. All data sources, views, web services… created or modified from the Design Studio can be opened from the Administration Tool and vice versa.

The administrators have to use the administration tool to configure Virtual DataPort. That is, to modify the cache configuration, setting-up the authentication, user management, etc. The following updates of the Denodo Platform will incrementally add these features to the Design Studio.

The Administration Tool will be actively maintained during for the full life of version 8.0 at least.

The Solution Manager includes the Design Studio. You do not have to start it, it starts up automatically when starting the Solution Manager Administration Tool. The installation of the Denodo Platform also includes the Design Studio.

We recommend using the Design Studio of the Solution Manager (https://solution-manager.acme.com:19443/denodo-design-studio/) because the Solution Manager provides a single point of access to all the Denodo components, not just the Design Studio. In addition, users will be able to use more Single Sign On protocols like OAuth or SAML, not just Kerberos, to log in to Design Studio.

Once you set up the environments in Solution Manager, when you log in, you will find the link to open Design Studio on the My applications page. In addition, you can also access directly with the URL “https://denodo-solution-manager.acme.com:19443/denodo-design-studio/#/web-local-login”.

To access the Design Studio of the Denodo Platform, go to “https://denodo-server.acme.com:9443/denodo-design-studio/#/web-local-login”.

General

Storing the Catalog on an External Database

The catalog of Virtual DataPort consists of its settings and the metadata created by users: data sources, views, web services, users, roles, etc. By default, the catalog uses the Apache Derby database that is embedded in Virtual DataPort.

You can now configure the catalog to use an external database instead of the embedded one.

See more about this in the section Storing the Metadata on an External Database of the Administration Guide.

Graphical Support to Upload Drivers and Other Libraries

The Administration Tool now provides support to upload to Virtual DataPort the JDBC drivers, the SAP Java Connector (JCo), the Essbase libraries, etc.

From now on, before creating a connection to a database for which Virtual DataPort does not provide the driver, upload it from the dialog File > Extensions management.

Important

In Denodo 8.0, do not copy a driver or other library to the installation; instead, upload it from this dialog. That way, the library will be copied to the appropriate directory and if Virtual DataPort is configured to store the catalog on an external database, all the servers sharing the same database will have access to this library.

See more about this in the section Importing Extensions of the Administration Guide.

Connections Only Use One Port

The JDBC driver, the administration tool and the new Design Studio now use a single port to communicate with Virtual DataPort (by default, 9999). In previous versions, the communication requires two ports (9999 and 9997). This change simplifies the configuration of firewalls.

For JMX connections, you do need to open two ports in the firewall as in previous versions. These ports have changed. By default, the application establishes the connection with the port 9997 and the port 9995 is the auxiliary port. The following applications of the Denodo Platform connect to these two ports:

  • Denodo Diagnostic & Monitoring Tool

  • Denodo Monitor

  • The monitoring feature of Solution Manager.

There are no changes regarding ODBC connectivity.

Managing Configuration Properties

You can now read the value of configuration properties of Virtual DataPort (“<DENODO_HOME>/conf/vdp/VDBConfiguration.properties”) and the embedded web container (“<DENODO_HOME>/resources/apache-tomcat/conf/tomcat.properties”) using the new stored procedure GET_PARAMETER. This is more convenient that connecting to the computer to obtain the value of a parameter.

Rename Elements

You can now rename data sources, JMS listeners and web services (SOAP and REST). To do this, on the Server Explorer of the Administration Tool or the Design Studio, right-click the element and click Rename. You already could rename views in previous version.

Execution Engine

Smart Query Acceleration: “Summary Views”

In analytical environments, most queries involve combining one or more facts tables with one or more dimensions and then, calculate an aggregation.

For these scenarios, Denodo 8 includes smart query acceleration techniques using a new type of view called Summary. Summaries store common intermediate results that the query optimizer can then use as an starting point to accelerate analytical queries.

See more about this in the section Smart Query Acceleration Using Summaries of the Administration Guide.

Data Sources

Hadoop Distribution is Not Required for Bulk Data Load

In previous versions, to enable Bulk Data Load on some data sources, you need to download the Hadoop distribution.

In this new version, Virtual DataPort includes the necessary libraries to perform bulk data load to HDFS-based databases so you do not need to download anything. This benefits the data sources for these databases:

  • Hive 2.0

  • Impala

  • Presto

  • Spark

This makes enabling bulk data load simpler, particularly on Windows where the Hadoop distribution requires the file winutils.exe.

Consider this when upgrading from a previous version to Denodo 8.0:

  • The data sources for these databases imported from a previous version will work without having to modify their configuration, provided the Hadoop distribution is located in the same path.

  • In the previous versions, you had to indicate the Hadoop executable location.

  • The Administration Tool no longer shows the option Hadoop executable location for new data sources.

  • To maintain backward compatibility, the Administration Tool still shows the option Hadoop executable location for data sources that were created with this option (in the VQL statement, that is the parameter HADOOP_EXECUTABLE_LOCATION).

Authentication and Security

Global LDAP Authentication

You can now set a global LDAP configuration, for the entire server, instead of having it to define on just one database. If you enable this, client applications can connect to any database and the credentials will be validated using the same LDAP configuration regardless of the database the application is connecting to.

In previous versions, you either had to configure LDAP authentication in all databases or instruct the client applications to establish the connection with a particular database.

In the Administration Tool, this configuration is located on the dialog Server configuration, tab Server authentication. The section LDAP Authentication of the Administration Guide explains how to enable this.

In addition, you can reuse this “global LDAP configuration” for Kerberos and SAML 2.0 instead of having to provide the same LDAP settings. This simplifies the configuration since most of the time, Virtual DataPort is configured to obtain the roles of the users from the same system (e.g. Active Directory) regardless of the authentication method.

Apply Changes to OAuth 2.0, SAML 2.0 and Kerberos Settings Without Restarting

You no longer need to restart Virtual DataPort in these situations:

  • When enabling or disabling OAuth 2.0 authentication or changing any of its settings. In previous versions, you have to restart when enabling/disabling this. In previous versions, you already can change the OAuth settings without having to restart.

  • When enabling or disabling SAML 2.0 authentication or changing any of its settings. In previous versions, you have to restart. In previous versions, you already can change the OAuth settings without having to restart.

  • When enabling or disabling Kerberos authentication or changing any of its settings. For example, you can change the keytab or selecting/clearing the check box Avoid domain name for authentication and you do not need to restart.

Denodo Stored Procedures

Procedure Create Remote Table Provides More Information

The stored procedure “CREATE_REMOTE_TABLE” now provides more information about each step it performs. In version 7.0, it returns three rows with one column, with the status of each step. Now, it provides additional information for each step.

See more about this in the section CREATE_REMOTE_TABLE of the VQL Guide.

Administration Tool

Easier Access to OData and GraphQL Services

The context menu of the Server Explorer of the Administration Tool has two new options: Open OData service and Open GraphQL service. This makes it easier to access these services.

Warning When Deleting the Implementation of an Interface View

When you delete a view, the administration tool lists the views that depend on the one you are deleting, so you confirm that you want to delete all these views.

Now, this dialog also warns about interface views that will be invalid due once its implementation view is deleted.

See more about this in the section Views Affected by Modifications of the Administration Guide.

Database Is Optional in the Server URI

When connecting to Virtual DataPort from the Administration Tool or the Design Studio, the database is optional in the URI (e.g. “//denodo-server.acme.com:9999”). When you do not specify the database, you will connect to the first database over which you have the privilege CONNECT.

In previous versions, you always have to provide the database.

This does not apply to JDBC connections nor other types of connections.

Northbound Connections

New Denodo GraphQL Service

Denodo now provides a GraphQL service. GraphQL is a data query language, and a runtime for executing those queries against your data. Its main features are:

  • It gives clients the power to ask for exactly what they need.

  • It gets many resources in a single request, reducing the number of API requests.

See more about this in the section Denodo GraphQL Service of the Administration Guide.

This new service complements the existing offering of an OData4 service and the RESTful web service.

RESTful Web Service Supports Kerberos Authentication

The RESTful Web Service now can be configured to use SPNEGO authentication (i.e. Kerberos over HTTP). In previous versions, the REST web services can use this authentication method but not the RESTful web service.