S3 Path¶
Use this type of path to obtain the data from a file or a set of files located in a S3 bucket.
Find information about the Filters tab in Compressed or Encrypted Data Sources; filters work the same way for any type of path (local, HTTP, FTP…).
Configuration¶
In URI, enter the path you want to obtain the data from. It can point to a file or a directory and you can use interpolation variables (see section Paths and Other Values with Interpolation Variables).
In Custom properties you can set the same properties that you would put in the Hadoop configuration files like core-site.xml to configure the S3A Hadoop connector.
Paths Pointing to a Directory¶
When you create a base view over a data source that points to a directory, Virtual DataPort infers the schema of the new view from the first file in the directory and it assumes that all the other files have the same schema.
Only for delimited-file data sources: if the path points to a directory
and you enter a value in File name pattern, the data source will
only process the files whose name matches the regular expression entered
in this box. For example, if you only want to process the files with the
extension log, enter (.*)\.log.
Note
For XML data sources, if a Validation file has been provided, all files in the directory have to match that Schema or DTD.
Authentication¶
There are two ways to configure the credentials:
AWS IAM credentials: specify the AWS access key ID and the AWS secret access key. Optionally, you can enter the AWS IAM role ARN. The AWS user will assume the role to get the necessary privileges to connect to the bucket. The role ARN is the Amazon Resource Name of the IAM role you want to assume. Specify the role ARN like
arn:aws:iam::<awsAccountId>:role/<roleName>.Denodo AWS instance credentials: automatically obtains the credentials of the database from the AWS instance where this Virtual DataPort server is running. Optionally, you can also specify an AWS IAM role ARN to get the necessary privileges to connect to the database.