Quick question about Json parsing. I know there are currently no native functions (please let me know if that is incorrect) for parsing json on the read, like you would have natively with HIve or Presto (using something like get_json_object).
I could write my own function, but before doing that, I am curious if there are any best practices that you would suggest using in the tool. Ideally we would have denodo parse it on the fly, without the user having to write any code to parse it.
I know there are 'flatten' options and 'json' type connections, but I have not gotten those to work. Here is what the data footprint looks like from my end:
1. We get data each day from a rest API. That data is just one big json dictionary file. That json string gets stored as one text type columm in Hive.
2. We have processes that fetch that new data and parse out all the columns into fields.. then that new parsed data is added to a table with all the rest of the parsed data.
3. There are a few hundred million rows of parsed data in the cleansed, 'final' table.
Ideally I would like it if I could just have denodo look at that json field from hive and auto-parse all the events into its own table. Then it would save me from having to ETL the data around.
Is anything like this a possibility? Thanks!