Changes in Virtual DataPort 7.0¶
This section lists the changed introduced in Virtual DataPort 7.0 compared to Virtual DataPort 6.0.
Privileges¶
CONNECT Privilege¶
In Denodo 7.0 you must grant the CONNECT privilege to a user over a database in order to allow this user to access any of the elements of that database. The other privileges over the database or its elements are disregarded if the user/role does not have the privilege CONNECT.
In previous versions, the CONNECT privilege was only checked when establishing the connection.
Because of this change, grant the users/roles the CONNECT privilege over the databases in which they already have other privileges.
Privileges of Users with the Role “serveradmin”¶
Denodo 6.0 has an option to grant LDAP/Kerberos users that have the role “serveradmin” the permission to grant/revoke privileges to users and roles, including themselves. This was located in tab Privileges of the Server configuration.
In this version this option is not available. Therefore, you need to add these users to the “assignprivileges” group of Active Directory.
Changes in VQL Syntax¶
The statement ALTER DATABASE
no longer admits the parameter I18N
.
In the parameter PASSWORD
of the CONTEXT
it is now mandatory to indicate the password encrypted and add the token ENCRYPTED
. For example:
SELECT *
FROM view1
CONTEXT(USERNAME = 'admin', PASSWORD = 'd4GvpKA5BiwoGUFrnH92DNq5TTNKWw58I86PVH2tQIs/q1RH9CkCoJj57NnQUlmvgvvVnBvlaH8NFSDM0x5fWCJiAvyia70oxiUWbToKkHl3ztgH1hZLcQiqkpXT/oYd' ENCRYPTED)
This affects the views that:
The data is obtained from a data source of type JDBC, web service, BAPI or multidimensional data sources.
And the data source was created with the option “with pass-through session credentials”.
And you want to connect to the source with other credentials than the ones you used to connect to the Server.
Changes in Date Types¶
Denodo 7.0 has new types to represent datetime values.
localdate
: Represents a date as a year, a month and a day, without taking into account the time zone.time
: Represents a time as hour, minute, second and fraction of a second.timestamp
: Represents date and time, without taking into account the time zone.timestamptz
: Represents date and time with time zone.intervaldaysecond
: A duration in terms of days, hours, minutes and seconds.intervalyearmonth
: A duration in terms of years and months.
The type date
is now deprecated.
The function current_date()
(and its equivalent, CURRENT_DATE
) now returns a value of type localdate
.
The function now()
(and its equivalent, CURRENT_TIMESTAMP
) now returns a value of type timestamptz
.
In Denodo 7.0, Virtual DataPort does not delegate functions with input parameters of type timestamptz
to databases that do not support this data type. This is because the result of most functions operating with values of type timestamptz
depends on the time zone of the query (i.e. the i18n of the query). For example:
SELECT gethour(TIMESTAMP WITH TIME ZONE '2020-01-01 00:00:00 +00:00')
This return 0 if the query is with the i18n rfc_restful
(GMT) and 15 if running with i18n us_pst
. As databases without a data type with time zone
usually do not have functions to convert between time zones, Virtual DataPort does not delegate these functions
to ensure consistency with the expected result of the function.
An exception to this rule is the function now()
(and its equivalent CURRENT_TIMESTAMP
), which is still delegated to all the sources in 7.0 because:
To keep backward compatibility
Its high impact in existing projects. That is, a lot of queries use this function so it is better to delegate it to all databases even though, there may be scenarios where the result of the query is not correct due to the change in the time zone.
Take into account that, as the function
now()
returns atimestamptz
value, the constructs that apply a function over the functionnow()
will not be delegated to databases that do not support the typetimestamptz
.
In Denodo 7.0, the setting “Date type mapping” of the SOAP web services only affects fields of type date (deprecated)
, not timestamp
nor timestamptz
nor localdate
.
One of the effects is that in web services with “Date type mapping” set to “date”, if a field projects the function now()
,
the response of the web service will now contain
a timestamp with time zone, unlike in Denodo 6.0 that only returns a date.
To continue returning only a date (without the time component), use current_date()
instead of now()
.
The section data types for dates and timestamps explains in detail these new date types.
We recommend modifying the definition of the views to use the new types because their behavior is more consistent and easier to use. The main reason being that, by default, “date” is a timestamp with time zone.
Date-time values are displayed with the following format, regardless of the language settings of the computer where the Administration Tool runs:
Data Type |
Format |
---|---|
localdate |
yyyy-MM-dd (<year>-<month>-<day>) For example: “2018-12-31”. |
timestamp |
yyyy-MM-dd HH:mm:ss.S (<date> <time>) For example, “2018-12-31 23:58:05.15673” |
timestamptz |
yyyy-MM-dd HH:mm:ss.S XXX (<date> <time> <time zone>) For example, “2018-12-31 23:59:05.15673 -08:00” |
time |
HH:mm:ss.S (<hour>:<minute>:<second>.<millisecond>) For example: “23:15:05.15673” |
date (deprecated) with subtype DATE |
yyyy-MM-dd (<year>-<month>-<day>) For example: “2018-12-31”. |
date (deprecated) with subtype TIMESTAMP or subtype TIMESTAMP WITH TZ |
yyyy-MM-dd HH:mm:ss.S XXX (<date> <time> <time zone>) For example, “2018-12-31 23:59:05.15673 -08:00” |
date (deprecated) with subtype TIME |
HH:mm:ss.S (<hour>:<minute>:<second>.<millisecond>) For example: “23:15:05.15673” |
The tool also uses these formats when storing the results of a query to a file.
If you want a query to display a datetime value with a different format, use the function FORMATDATE.
Search Fields that Use the Type Date¶
During the process of exporting the VQL from a Denodo 6.0 installation, we recommend setting the following properties:
SET 'com.denodo.exportMigrationCompatibility80' = 'true';
SET 'com.denodo.vdb.catalog.exportMigrationCompatibility.migrateDateTypes' = 'true';
As explained in that section, with these properties, the Denodo server 6.0 analyzes the date
fields to see if they can be exported to one of the new datetime types.
You can also use the procedure MIGRATE_DATE_TYPES in Denodo 7.0 at any time to migrate the types automatically at any time.
This section explains how to search what views still have fields of type date
and how to change them.
Step #1
Execute these VQL statements on the new Denodo 7.0 installation:
-- Materialized table that returns information about base views, interface views
-- and materialized tables, but not derived views.
CREATE MATERIALIZED TABLE view_summary AS
SELECT database_name
,name
,view_type
FROM GET_VIEWS()
WHERE view_type IN (0, 2, 3)
CONTEXT('queryTimeout' = '120000');
-- Materialized table that returns information about the columns of
-- type "date".
CREATE MATERIALIZED TABLE view_date_column_summary AS
SELECT database_name
,view_name
,column_name
,column_vdp_type
,column_sql_type
FROM GET_VIEW_COLUMNS()
WHERE column_vdp_type = 'date' CONTEXT('queryTimeout' = '120000');
Step #2
Execute the following query. It returns the fields of type “date” of the base views, interface views and materialized views. We do not include derived views yet because in many of them, the type is inherited from an underlying view.
SELECT v.database_name
,v.name
,c.column_name
,c.column_vdp_type
,c.column_sql_type
FROM view_summary v
INNER JOIN view_date_column_summary c ON v.database_name = c.database_name
AND v.name = c.view_name CONTEXT('queryTimeout' = '120000');
Step #3
For each field of the result, we recommend changing its type to one of the new data types:
For base views, select the type that matches the type of the column in the table of the underlying database.
For interface views, modify the definition of the field.
For materialized views, recreate them with one of the new types instead of “date”.
Step #4
Execute the following query. It returns the fields of type “date”, including derived views.
SELECT database_name
,view_name
,column_name
,column_vdp_type
,column_sql_type
FROM GET_VIEW_COLUMNS()
WHERE column_vdp_type = 'date' CONTEXT('queryTimeout' = '120000');
Step #5
The type of a field of a derived view is still date
if:
The field is defined as such by one of its underlying views.
Or if the field is the result of an expression that returns a
date
. For example, if a field is the result of the expressionTO_DATE
, the type of the field isdate
. In this case, you should use one of the new functions instead (TO_LOCALDATE, TO_TIME, TO_TIMESTAMP or TO_TIMESTAMPTZ depending on the type of the input field).
Subtracting “Date” Expressions Now Returns a Long¶
The result of subtracting two expressions of type date
(<expression 1> - <expression 2>
) is the number of days between <expression 1> and <expression 2>. The result is a positive number if <expression 1> is a more recent datetime than <expression 2>. Otherwise, the result is a negative number.
In Denodo 6.0, this number is of type int
. In Denodo 7.0, this value is of type long
. On very rare cases, this change may cause a query to fail. If it does, modify the view or the query.
See more about this in the section Arithmetic Operators for Datetime and Interval Values of the VQL Guide.
Backward Compatibility Properties¶
This section explains how to restore the behavior of previous versions of Denodo, regarding the values of type “date”. Although they are available, we recommend not using them and instead, switch to the new datetime types.
The execution engine may implicitly convert a text value into a datetime value. In previous versions, the result of this conversion is a value of type “date”. In Denodo 7.0, literals may be converted to one of the new datetime types, not to “date”.
To keep the behavior of previous versions, execute the following from the VQL Shell:
SET 'com.denodo.vdb.parser.datetime.literals.transformToOldToDateFunction' = 'true';
In Denodo 7.0, custom functions that in its Java source code have an input parameter or return an object of the Java class
java.util.Calendar
, now return a “timestamptz” value. In previous versions, they return a “date”.To keep the behavior of previous versions, execute the following from the VQL Shell:
SET 'com.denodo.vdb.compatibility.datetime.custom.mapCalendarToDateType' = 'true'
Temporary tables use the new datetime types. In previous versions they use “date”. To keep the behavior of previous versions, execute the following from the VQL Shell:
SET 'com.denodo.vdb.parser.datetime.createsqltable.mapDateTimeToOldDateType' = 'true'
Changes in Stored Procedures¶
Procedure VIEW_DEPENDENCIES¶
Now when you call this stored procedure and the target has a dependency of the type storedprocedure
, it returns the value
Predefined Storedprocedure
when this procedure is a predefined one.
Procedure GET_CATALOG_METADATA_WS¶
Now you can call this stored procedure to get information related to Web Services.
Procedures GET_ELEMENTS and GET_VIEWS¶
These procedures now differentiate materialized tables from base views. In previous versions, these procedures considered materialized views as base views.
In the result of these procedures, the value of the field subtype
is now materialized
and the value of view_type
is now 3
.
Changes in Cost Optimization¶
As a result of some internal changes in the calculation of the cost optimizations, some query plans involving data movements and other operations can change from the previous versions.
Subtype of DECIMAL Fields¶
When a new base view with a DECIMAL
field is created, we are going to try to use the best subtype that fits the field
size, INTEGER
, BIGINT
OR DECIMAL
.
To restore the behavior of previous versions, execute this from the VQL Shell:
SET 'com.denodo.vdb.admin.model.wrapper.convertExactDecimaltoIntegerSubtype' = 'false'
JDBC Data Sources¶
Oracle¶
Starting with Denodo 7.0 GA, right after a JDBC data source opens a connection to Oracle, it executes the following command on that connection:
ALTER SESSION SET NLS_DATE_FORMAT= 'YYYY-MM-DD';
By doing this, the pattern of the datetime values does not depend on the configuration of Oracle. Note that if in a previous version of Denodo you created a base view from a SQL query, the conditions of this query over datetime fields may not have the right format and the query will fail. In that case, modify the query so the datetime value follows the pattern “YYYY-MM-DD” (<year>-<month>-<day>). E.g. hire_date >= DATE '2018-02-03'
.
Apache Hive¶
The Apache Hive driver is no longer distributed.
The adapters for Apache Hive 0.7, 0.10, 0.11 and 0.12 have been removed. If you were using any of these, use the adapter for Apache Hive 0.13 instead.
Note
If you are connecting to Cloudera’s or Hortonworks’ Hive, use the adapter specific to them. Do not use the generic Apache Hive adapters.
Impala JDBC Drivers¶
Due to license restrictions, the JDBC driver of Impala is no longer distributed.
DenodoConnect Component: Salesforce Wrapper¶
Starting with Denodo 7.0, Denodo has a native data source to Salesforce.com. Users that retrieve data from Salesforce are advised to stop using the DenodoConnect Component and use the new data source because the DenodoConnect component will not be actively maintained.
Denodo JDBC Driver Compiled with Java 8¶
Starting with Denodo 7.0, the Denodo JDBC driver is compatible with Java 8, 9, 10 and 11. The client applications that use the Denodo JDBC driver and that run with earlier versions of Java need to be updated.
The driver of Denodo 6.0 is compatible with Java 7 and Java 8; the driver of Denodo 5.5, with Java 6.
You cannot use the JDBC driver of earlier versions of Denodo to connect to Denodo 7.0 nor use the JDBC driver of version 7.0 to connect to earlier versions.