Access Through Arrow Flight-SQL¶
Arrow Flight SQL is a protocol designed for high-performance, distributed database query execution using Apache Arrow’s columnar data format. It provides a standard interface, similar to ODBC or JDBC, for applications to interact with SQL databases. By leveraging Arrow’s in-memory format and the Flight RPC framework, it enables efficient data transport and processing, particularly in big data and cloud environments. It supports a variety of platforms and tools, ensuring compatibility across different systems and enhancing query speed and scalability.
Current Restrictions¶
The system supports username/password authentication for client connections.
Parameterized queries are not currently supported.
Type mapping: Virtual DataPort to Apache Arrow¶
In our system, we map Virtual DataPort data types to Apache Arrow types based on the information provided in the field’s data type and defined subtype (if applicable). This mapping ensures that data from Virtual DataPort is appropriately represented when transported in the Apache Arrow format for efficient in-memory processing.
Mapping: Virtual DataPort Types to Arrow Types¶
Virtual DataPort Type |
Source Type |
Arrow Type |
|---|---|---|
BLOB |
BINARY |
BINARY |
BLOB |
BINARY |
|
LONGVARBINARY |
BINARY |
|
VARBINARY |
BINARY |
|
BOOLEAN |
BIT |
BOOLEAN |
BOOLEAN |
BOOLEAN |
|
DATE(DEPRECATED) |
TIMESTAMP_W_TZ |
TIMESTAMP(MICROSECOND, TZ) |
DATE |
DATE(DAY) |
|
TIME |
TIME(MICROSECOND, 32 bits) |
|
TIMESTAMP |
TIMESTAMP(MICROSECOND) |
|
DECIMAL |
DECIMAL |
DECIMAL(256 bits) |
DOUBLE |
FLOATINGPOINT(DOUBLE) |
|
FLOAT |
FLOATINGPOINT(SINGLE) |
|
NUMERIC |
DECIMAL(256 bits) |
|
REAL |
FLOATINGPOINT(SINGLE) |
|
DOUBLE |
DECIMAL |
DECIMAL(256 bits) |
DOUBLE |
FLOATINGPOINT(DOUBLE) |
|
FLOAT |
FLOATINGPOINT(SINGLE) |
|
NUMERIC |
DECIMAL(256 bits) |
|
REAL |
FLOATINGPOINT(SINGLE) |
|
FLOAT |
DECIMAL |
DECIMAL(128 bits) |
DOUBLE |
FLOATINGPOINT(DOUBLE) |
|
FLOAT |
FLOATINGPOINT(SINGLE) |
|
NUMERIC |
DECIMAL(128 bits) |
|
REAL |
FLOATINGPOINT(SINGLE) |
|
INT |
BIGINT |
INT(64 bits) |
DECIMAL |
DECIMAL(128 bits) |
|
INTEGER |
INT(32 bits) |
|
NUMERIC |
DECIMAL(128 bits) |
|
SMALLINT |
INT(16 bits) |
|
TINYINT |
INT(8 bits) |
|
INTERVALDAYSECOND |
INTERVAL_DAY_TO_SECOND |
INTERVAL(MONTH_DAY_NANO) |
INTERVALYEARMONTH |
INTERVAL_YEAR_TO_MONTH |
INTERVAL(MONTH_DAY_NANO) |
LOCALDATE |
DATE |
DATE(DAY) |
LONG |
BIGINT |
INT(64 bits) |
DECIMAL |
DECIMAL(256 bits) |
|
INTEGER |
INT(32 bits) |
|
NUMERIC |
DECIMAL(256 bits) |
|
SMALLINT |
INT(16 bits) |
|
TINYINT |
INT(8 bits) |
|
TEXT |
CHAR |
UTF8 |
CLOB |
UTF8 |
|
DATALINK |
UTF8 |
|
LONGVARCHAR |
UTF8 |
|
LONGNVARCHAR |
UTF8 |
|
NCHAR |
UTF8 |
|
NCLOB |
UTF8 |
|
NVARCHAR |
UTF8 |
|
TIME |
TIME |
TIME(MICROSECOND, 32) |
TIMESTAMPTZ |
TIMESTAMP_W_TZ |
TIMESTAMP(MICROSECOND, TZ) |
TIMESTAMP |
TIMESTAMP |
TIMESTAMP(MICROSECOND) |
XML |
CLOB |
UTF8 |
SQLXML |
UTF8 |
|
LONGVARCHAR |
UTF8 |
|
LONGNVARCHAR |
UTF8 |
|
VARCHAR |
UTF8 |
|
NCLOB |
UTF8 |
|
NVARCHAR |
UTF8 |
|
ARRAY |
ARRAY |
LIST |
STRUCT |
STRUCT |
STRUCT |
Note
Mapping for types TIME, TIMESTAMPTZ and TIMESTAMP can differ if connection parameters timePrecision or
timeStampPrecision are specified. See Access using Flight SQL for more information about parameters.