Temporary File Generation on the Denodo Side¶

When a bulk data load is initiated from the client application (in this case, the Virtual DataPort server), the system first prepares the data you want to cache using bulk load by generating temporary files on the server.

• These temporary files are created in the directory defined by the Work path located in the Read & Write tab of the data source configuration.

• By default, the server creates these files in its installation directory.

• If a different location is required, a custom directory can be specified in the Work path field.

These files act as intermediate staging files and contain the formatted data that will be loaded into Apache Doris and will be deleted once the data is loaded on the database. With the default configuration, an Apache Doris data source uses the bulk data load features of this database. To make sure this is enabled, open the data source and in the tab Advanced - section Driver properties, check that the value of allowLoadLocalInfile is true (the default value).

Loading Data into Apache Doris (LOAD DATA LOCAL INFILE)¶

After the temporary files are created in the Work path, the client (VDP server) issues an Apache Doris LOAD DATA LOCAL INFILE command to ingest the data into the database. The command references the temporary file stored in the Work path and defines how the data should be parsed and mapped into the target table. This process is done transparently to the user.