You can translate the question and the replies:

Denodo + Spark SQL

I had created spark conenction in denodo. Datbase Adapter: Spark SQL 1.6 Driver Class : org.apache.hive.jdbc.HiveDriver Database URI : jdbc:hive2://xyx.abc.com:10000 But while running queries;still the job is running on mapreduce. In yarn for the job application type is showing as MAPREDUCE.
Spark-sql connection
user
10-02-2017 08:20:57 -0500
code

4 Answers

Hi, As you have mentioned, you are using the Hive JDBC driver to connect Denodo using Spark SQL. This JDBC driver in general uses a map reduce operation to calculate relational queries.  So any query running in Hive, expect for a full data scan like (SELECT * From <table_name>;), will always be triggered using the map reduce for query execution. Hope this helps you!
Denodo Team
16-02-2017 08:58:06 -0500
code
Try port 10016 on HDP.
Denodo Team
30-04-2018 20:24:10 -0400
code
Hello Support Team, So here if we want to force Denodo to use Spark engine, is it possible through Denodo? Thanks
user
05-09-2018 11:52:49 -0400
Hello! Hive on Spark requires server level setup and configuration on Hive end (requires start and stop of Hiveserver2) to enable Spark Execution Engine. Though Denodo platform can connect and communicate to Hive on Spark without issues, but cannot force the HiveServer2’s behaviour i.e. The Execution engine choice is solely with Hive and Denodo Platform has no way to enforce it. So, for you to use The Spark engine instead of MapReduce please setup the Hive on Spark as explained in [Hive on Spark: Getting Started](https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started). I find the tutorials from [Big Data](https://community.denodo.com/tutorials/browse/bigdata/index) helpful for Data Integration with Big Data Systems. Hope this helps!
Denodo Team
12-11-2018 04:38:38 -0500
code
You must sign in to add an answer. If you do not have an account, you can register here