Apache Doris into Apache Arrow through ADBC over the MySQL wire, using MySQL Connector/ODBC 9.4 (adbcBridge 0.1.0) #67263
Replies: 1 comment
|
A follow-up with one new measurement. On 2026-09-05 I ran the same seven-step ADBC workload (connect, SELECT 1, DDL + inserts + read, 1,000-row Native driver: stops at connect with Table for all 28 PostgreSQL-wire and MySQL-wire databases, with each native driver's first error: https://adbcbridge.org/notes/native-adbc-drivers-on-wire-compatible-databases/ |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
adbcBridge is a small Apache-2.0 ADBC driver written in C11 that loads an ODBC driver and exposes it through the Arrow ADBC C ABI: block-cursor reads straight into Arrow record batches, bulk ingest, metadata, partitioned parallel reads. I ran Apache Doris through it as one of 46 databases in a single compatibility workload and wanted to share the entry here, since the quirks it records are Doris-specific.
The first-party
flightsqlADBC driver already covers Doris, and themysqlADBC driver may work over its wire protocol too. This entry is a third, documented route — the MySQL wire through MySQL Connector/ODBC — and records exactly what it needed.What was verified (Apache Doris 2.1.0, MySQL 5.7.99 wire, FE + BE in one container; driver: MySQL Connector/ODBC 9.4):
_binary '<raw bytes>'literal itsPREPONCLIENT=1path inlinesNO_SSPS=1); through 8.4.0 the same entry failed only the non-BMP text check —héllo 🚀stored byte-exact, read back ashéllo ???on every read path including pyodbc, because 8.4.0 substitutes?for a non-BMP character before either C type sees itWhat the compatibility entry records: Doris reports no transaction support (
SQL_TC_NONE), so_binaryparameter literals are rewritten as text andNO_SSPS=1is required — the FE prepares only a pointSELECTon astore_row_columnunique-key table or anINSERT, and a server-side preparedINSERTexecutes only for parameters bound fromSQL_C_CHARor to a matching non-character SQL type — a character parameter bound from any other C type,SQL_C_WCHARincluded, gets a bareNullPointerExceptionfrom the FE. Every OLAP table has to declare how its rows are distributed, so generated ingest DDL appendsDISTRIBUTED BY RANDOM BUCKETS AUTOplusenable_duplicate_without_keys_by_default, without which Doris refuses any table whose first column isTEXT/STRING(what a generated string column is),FLOATorDOUBLE; a leadingVARCHAR(n)is accepted. Keyed on@@version_commentsinceversion()is a bare MySQL number. No binary column type, noDOUBLE PRECISIONspelling;ANSI_QUOTESis accepted but ignored, so identifiers are backtick-quoted. Ingest 2.2k rows/s — anINSERTis a load transaction whatever it carries, so multi-row batching is worth ~300x (7 rows/s without it); fetch 1.36M rows/s. Full entry: docs/COMPATIBILITY.md.Trying it (Python; Rust, Go, Java and C# are on the docs site):
Links: repository · docs · upstream notes · PyPI
It is a 0.1.0. If the entry says something wrong about Doris, or you run a version or driver I didn't, an issue on the repository with the details is the most useful thing you could send.
All reactions