You can translate the question and the replies:

Best way to do mass exports

Hi guys! I'm trying to find out the best way to do a mass export from denodo (like 20MM records). I'm just starting my research so any help would be useful. Thanks in advance! Best, Fabian
mass exports best practices
user
21-08-2017 13:29:30 -0400
code

3 Answers

Hi, If I need to export datasets from my denodo views I use one of the possible ways, * In Virtual DataPort Admin tool, by clicking on "Save" button from query results panel I will be able to export the data as a file. * In Denodo Scheduler, by creating a new VDP type job and pointing it to VDP data source. I will be able to provide a query and by selecting the CSV exporter in the exporters section to export the result set into a file. Have a look at the official documentation for creating a scheduler job [General Structure of a Job](https://community.denodo.com/docs/html/browse/6.0/scheduler/administration/creating_and_scheduling_jobs/configuring_new_jobs/general_structure_of_a_job#example-of-exporters-section) from Scheduler Administration Guide. Keep in mind that if you have a Denodo Express License then it supports only a maximum of 10,000 rows in the export. Hope this helps you!
Denodo Team
22-08-2017 09:02:31 -0400
code
Yes, I understand that. I'm working in a project where our customer has the full denodo license. However, each time I try to get a limit of 50K rows, my VDP session gets crashed, the VDP client runs out of memory and starts eating my whole CPU. So, I think it is not a valid solution when trying to export large amounts of rows. I would try with the Scheduler but just wanted to know if there are any other ways to pull out that amount of data... Thanks for your help! Best, Fabian
user
 Edited on: 13-10-2017 10:51:10 -0400
Hi, If you are experiencing these problems, you should try to reconfigure the JVM heap space of your Virtual DataPort admin tool, to help you to export huge dataset from your Denodo views. To configure your JVM heap space you can follow these steps: 1. Launch Denodo Platform Control Center. 2. Navigate to Configure > JVM options. 3. Modify the JVM configuration parameters for "Virtual DataPort Administration Tool" and increase the JVM heap space. I would advise you to have a look at the section "Recommended settings for the JVM when Denodo Platform needs to deal with huge datasets" in the KB article titled [Denodo Admin and Development Best Practices](https://community.denodo.com/kb/view/document/Denodo%20Admin%20and%20Development%20Best%20Practices?category=Best+practices) Hope this helps!
Denodo Team
23-08-2017 09:02:43 -0400
code
You must sign in to add an answer. If you do not have an account, you can register here