You can translate the question and the replies:

Handling of Array by Spark Simba driver

Hi Team, I am using spark driver in denodo to query the data. While i directly query objects in S3, the column datatype is shown as an array, but when we onboard it to denodo it is onboarded as "text" and I don't see a dataype as an array in denodo. Due to this when I execute the base view I am getting the JDBC wrapper error. I am able to select all the rows when I select columns execpt the array column through VQL. I am not able to flatten the column as it does not show as an array type, any suggestions? Thanks, Vijay K

#array #customfunction

user
13-05-2021 23:04:18 -0400

1 Answer

Hi, I am able to read this data into a Denodo Array by passing the onboarded ‘text’ string to the [Split() function](https://community.denodo.com/docs/html/browse/8.0/en/vdp/vql/appendix/syntax_of_condition_functions/text_processing_functions#split), and using that result. Like so : ``` Split(<separator value>, <your array text>) ``` This extra function call is necessary, as the Denodo Spark Dialect is currently limited by handling compound types as strings when performing Projection. For more information, refer to the[ Denodo Dialect for Spark User Manual](https://community.denodo.com/docs/html/document/denodoconnects/8.0/en/Denodo%20Dialect%20for%20Spark%20SQL%20-%20User%20Manual). Hope this helps!

Denodo Team
14-05-2021 17:12:12 -0400

ADD ANSWER

You must sign in to add an answer. If you do not have an account, you can register here