I'm trying to read delimited files that are being saved to a partially known folder structure such as this:
The root folder is known and will not change.
The subfolder is partially known. E,g I want denodo to read data from within any subfolder that has the word 'folder' in it, (subfolder A & subfolderB).
The third level folder (e.g 579718) is completely unknown, and it likely to change.
I then want denodo to recursively look for any .txt file within those third level folders (and even any subfolders within them), regardless of what name the folder has.
In short, it would be ideal if I could insert a regular expression into the "File Path" field within the delimited file source, as well as the file name pattern.
I've found this to be possible when pulling data from S3 storage using the denodo-hdfs-customwrapper-7.0, but not on any local file system delimited source wrappers.
I hope it makes sense what I'm trying to achieve. Thanks in advance!