You can translate the question and the replies:

Flat file data source compressed zip file

I am using Dendo 8 platform. I have a zip file containing multiple delimited csv files. I create a delimited file datasource and during source configuration I select "Decompress (zip format)" option. Test connection is successful. However after I save and create base view, it only show me the result for the first csv file in the zip file. How to make Denodo read all the csv files in the compressed zip file so that the it is able to create the base views for all the csv files in the zip file? I also have another question related to above as well. I can only see ".zip" and ".gzip" type filters. Can Denodo able to read from ".tar" file as a data source? Currently I do not see an option for this.
user
20-01-2021 22:18:57 -0500

1 Answer

Hi, When creating a base view using a ZIP compressed file, the first CSV file will only be processed which is expected at the moment. As an alternate workaround, I would perform the following based on my requirement: * Extract all the CSV files from the ZIP. * Create a Delimited File data source, set the **Data Route** to 'Local' and set the file path to the location of the Unzipped folder. * If the schema of the CSV files is the **same** then I would: * Provide the path to the folder along with the pattern of files type in the File Name pattern. * You can refer to the [Paths Pointing to a Directory](https://community.denodo.com/docs/html/browse/8.0/vdp/administration/creating_data_sources_and_base_views/path_types_in_virtual_dataport/local_path#paths-pointing-to-a-directory) section of the Virtual DataPort Administration Guide. * Note: with this method only one base view would be created common to all the files of the same type in the folder. * If the files do not have a common schema, I would: * Set an [Interpolation Variable](https://community.denodo.com/docs/html/browse/8.0/vdp/administration/creating_data_sources_and_base_views/path_types_in_virtual_dataport/path_types_in_virtual_dataport#paths-and-other-values-with-interpolation-variables) to create base views on the given file name input only. However, if I still need to extract data from different CSV files packed in a ZIP or if I need to extract files based on any other format like **.tar**, then I would suggest you to create a Custom Input filter to extract files from the archive and process it in Virtual DataPort. You can take a look at [Developing Custom Input Filters](https://community.denodo.com/docs/html/browse/latest/vdp/developer/developing_extensions/developing_custom_input_filters/developing_custom_input_filters) section of the Virtual DataPort Developer Guide for more information. Hope this helps!
Denodo Team
21-01-2021 03:24:03 -0500
You must sign in to add an answer. If you do not have an account, you can register here