You can translate the question and the replies:

Conversion Error Timestamp -> Text with HIVE

Hi, I want to create a Selection View over a HIVE data source and encountered an error within the WHERE condition that I fail to solve. Users should be able to enter a date range using the fields "from_timestamp" and "to_timestamp" of type "timestamp". These inputs have to be converted into strings since the source has partitioning over year-month. Because of this, we need to convert timestamps to a text: 'yyyy-MM'. Field in the baseview: | Name | Type | Example | | -------- | -------- | -------- | | month | text | '2023-01' | Following documentation, I wanted to use formatdate() to convert the timestamp to a string, and then use substr() to cut out the "yyyy-MM" part. This works fine if i test it in VQL Shell: `select substr(formatdate('yyyy-MM-dd HH:mm:ss', TIMESTAMP '2023-01-20 10:16:45.673'), 1, 7)` But it some how does not, when applied in the view itself: Where condition in Denodo: `viewname.month >= substr(formatdate('yyyy-MM-dd HH:mm:ss', from_timestamp), 1, 7)` Query pushed to source: `WHERE t0.month >= SUBSTR( date_format( from_utc_timestamp( unix_timestamp('2023-01-10 10:16:45.000', 'yyyy-MM-dd HH:mm:ss'), 'yyyy-MM-dd HH:mm:ss') ), 1,7)` Because of this query HIVE complains: > SemanticException [Error 10015]: Line 1:3390 Arguments length mismatch ''yyyy-MM-dd HH:mm:ss'': date_format requires 2 argument(s), got 1), Why does the query pushed to HIVE only contain 1 argument for date_format() instead of 2? Is there setting that prevents Denodo to push down type conversions within the WHERE condition? Since the conversion works in VQL Shell just pushing the result of ` substr(formatdate('yyyy-MM-dd HH:mm:ss', from_timestamp), 1, 7) ` should work.
user
21-04-2023 05:49:57 -0400
code

3 Answers

Hi, I’d use only the function [FORMATDATE](https://community.denodo.com/docs/html/browse/8.0/en/vdp/vql/functions/datetime_functions/datetime_functions#formatdate) that returns a string containing a datetime-type formatted using the given pattern. I’ve tried using that function like: FORMATDATE(‘yyyy-MM’, date_field) and it retrieves the data in a string with format 'yyyy-MM’. I didn’t need to use the [SUBSTR](https://community.denodo.com/docs/html/browse/8.0/en/vdp/vql/functions/text_functions/text_functions#substring-substr) function. Related to the data source settings that you’ve mentioned, in the Administration Tool go to the data source and then in Configuration>Source Configuration you have to go to “Delegate scalar function list” and enable “Custom”. There, I’d copy the list of functions enabled and paste it into a text editor and delete the functions that you would like to not be delegated. Finally, replace the function list with the new list customized. It is recommended to use the default configuration because it’s important for optimization reasons since it allows Virtual DataPort to delegate to the data source as much processing as possible to optimize response times and minimize traffic through the network. You can see more information in[ the Data Source Configuration Properties section](https://community.denodo.com/docs/html/browse/8.0/en/vdp/administration/creating_data_sources_and_base_views/data_source_configuration_properties/data_source_configuration_properties) of the VDP Administration Tool. Also, if you need to review the query and you have an active Denodo Support user, I’d recommend opening a Support Case in [the Denodo Support Site](https://support.denodo.com/) to analyze it in detail with the Denodo Support team. Hope this helps!
Denodo Team
24-04-2023 07:56:39 -0400
code
Hi, thank you for the in-depth answer. I followed your advice, got rid of SUBSTR and your proposal with changing "Delegate scalar function" solved the issue. Without the change of the Delegate scalar function, there is still some weird behaviour that I do not understand. I defined: `WHERE month >= formatdate('yyyy-MM',from_timestamp)` and Denodo forwarded this to HIVE: ``` WHERE t0.`month` >= date_format( from_utc_timestamp( unix_timestamp('2023-03-24 14:16:49.267', 'yyyy-MM-dd HH:mm:ss'), yyyy-MM') ) ``` [date_format()](https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.date_format.html) still seems to be missing the second argument. Was this not the case for you?
user
24-04-2023 08:30:18 -0400
Hi, I’ve tried executing a query in Hive using the function FORMATDATE(‘yyyy-MM’, timestamp_field) and Denodo forward the query correctly. If you have an active Denodo Support user, I’d recommend opening a Support Case in the [Denodo Support Site](https://support.denodo.com/) to analyze the issue in detail with the Denodo Support team. Hope this helps!
Denodo Team
26-04-2023 07:46:20 -0400
code
You must sign in to add an answer. If you do not have an account, you can register here