You can translate the question and the replies:

Spark fires multiple queries for a single denodo query

Hi, I'm connecting to denodo from AWS Glue (with Spark 2.4.3). When I issue a query to denodo from Glue, it fires multiple queries to denodo. eg: Denodo Query: **select * from (select sum(col1) as col1, col2, col3 from interface view where filters group by col2, col3);** What is actually fired to Denodo: **Query 1: select * from (select sum(col1) as col1, col2, col3 from interface view where filters group by col2, col3); Query 2: select sum(col1) from (select sum(col1) as col1, col2, col3 from interface view where filters group by col2, col3); Query 3: select col2 from (select sum(col1) as col1, col2, col3 from interface view where filters group by col2, col3); Query 4: select col3 from (select sum(col1) as col1, col2, col3 from interface view where filters group by col2, col3);** Basically there are four queries. The actual query + three more queries which is equal to number of columns in the select clause. I use below driver configuration to connect to Denodo from Glue. ``` df = spark \ .read \ .format("jdbc") \ .option("url",ddl_endpoint) \ .option("driver","com.denodo.vdp.jdbc.Driver") \ .option("query",sql) \ .option("user", source_dbparams['user']) \ .option("password",source_dbparams['password']) \ .load() ``` Anything I need to know about this behaviour? Edit: The denodo query is completely delegated.
user
27-01-2022 00:44:05 -0500
code

2 Answers

Hi, In order to replicate your scenario, I have connected to Denodo from AWS Glue by following the steps as per in the Knowledge Base Article,[**How to connect to Denodo from AWS Glue**](https://community.denodo.com/kb/en/view/document/How%20to%20connect%20to%20Denodo%20from%20AWS%20Glue) then executed some sample queries and not able to reproduce your scenario(i.e.) it has been executed as a single query. Upon further internally reviewing I came to know that, if it is partitioned in spark then spark executes parallel queries to the source for faster performance. Hence, you can verify with the spark for this behavior. If you have any further questions and you are a valid support user, you may open a Support Case at the Denodo Support Site and the Support Team will assist you further. Hope this helps!
Denodo Team
27-01-2022 07:32:34 -0500
code
Thanks for the quick update Denodo team. I will take a look at the Spark logs and verify if this needs a case with Denodo. As mentioned in the Denodo document, I've put in the similar steps to connect and extract data from Denodo. Thanks again.
user
27-01-2022 23:52:59 -0500
You must sign in to add an answer. If you do not have an account, you can register here