Support DuckDB syntax: FROM-first, SEMI/ANTI joins, COPY, ATTACH, MACRO, PRAGMA and DuckDB 2.0 additions by hayssams · Pull Request #2643 · JSQLParser/JSqlParser

hayssams · 2026-09-15T14:04:35Z

Adds support for nineteen DuckDB constructs that JSqlParser rejected (or silently mis-parsed), including the new DuckDB 2.0 syntax. One commit per feature, each with tests that assert both the AST and a stable round-trip through toString() and the deparser.

Features, with the DuckDB documentation that specifies them

Feature	Example	DuckDB reference
`GLOB` pattern matching	`WHERE b GLOB 'y*'`	Pattern matching
Standalone `SEMI` / `ANTI` joins	`FROM t ANTI JOIN u ON t.id = u.id`	FROM clause
`MAP(key, value)` column type	`CREATE TABLE t (m MAP(VARCHAR, INTEGER))`	MAP type
`INSERT ... BY NAME` / `BY POSITION`	`INSERT INTO t BY NAME SELECT 1 AS b, 2 AS a`	INSERT
`USING SAMPLE` sizes and methods	`USING SAMPLE 10 PERCENT (bernoulli)`	Samples
`DESCRIBE <query>`	`DESCRIBE SELECT * FROM t`	DESCRIBE
`PRAGMA`	`PRAGMA table_info('t')`, `PRAGMA memory_limit = '1GB'`	Pragmas
`INSTALL` / `LOAD` extensions	`FORCE INSTALL h3 FROM community`, `LOAD httpfs`	Extensions
`ATTACH` / `DETACH`	`ATTACH 'file.db' AS mydb (TYPE SQLITE, READ_ONLY)`	ATTACH/DETACH
`PREPARE` / `DEALLOCATE`	`PREPARE q AS SELECT * FROM t WHERE a = $1`	Prepared statements
`COPY` (import and export)	`COPY (SELECT * FROM t) TO 'out.parquet' (FORMAT PARQUET)`	COPY
`CREATE MACRO` (scalar and table)	`CREATE MACRO add(a, b := 5) AS a + b`	CREATE MACRO
FROM-first queries	`FROM t SELECT a`, `FROM t WHERE a > 1`	FROM clause
`UNPIVOT` statement form	`UNPIVOT monthly_sales ON jan, feb INTO NAME month VALUE sales`	UNPIVOT
2.0 `CONNECT` / `DISCONNECT`	`CONNECT 'postgres://localhost/mydb'`	A preview of DuckDB v2.0
2.0 `CREATE TRIGGER ... REFERENCING OLD/NEW TABLE` with a statement body	`CREATE TRIGGER t AFTER UPDATE ON x REFERENCING OLD TABLE AS o NEW TABLE AS n FOR EACH STATEMENT INSERT INTO ...`	A preview of DuckDB v2.0
2.0 `CREATE EXTENSION REPOSITORY`	`CREATE EXTENSION REPOSITORY my_repo WITH PREFIX 'https://…'`	A preview of DuckDB v2.0
2.0 `APPROX NEAREST n BY SIMILARITY` joins	`JOIN products t APPROX NEAREST 2 BY SIMILARITY array_cosine_similarity(q.vec, t.vec)`	A preview of DuckDB v2.0
2.0 recursive CTE `USING KEY`	`WITH RECURSIVE tbl (a, b) USING KEY (a, avg(b)) AS (...)`	A preview of DuckDB v2.0

Notes

ANTI JOIN previously parsed silently as a table alias, so valid SQL produced a wrong AST. Join.isAnti() now records it, and LEFT/RIGHT/FULL SEMI|ANTI are accepted too.
New Statement types (PRAGMA, INSTALL/LOAD, ATTACH/DETACH, CONNECT/DISCONNECT, PREPARE/DEALLOCATE, COPY, CREATE MACRO, CREATE EXTENSION REPOSITORY) are wired through StatementVisitor, StatementVisitorAdapter, StatementDeParser, TablesNamesFinder, StatementFeatureVisitor and StatementValidator. COPY, PREPARE and UNPIVOT expose their nested query to TablesNamesFinder.
FROM-first is represented as a PlainSelect with a fromFirst flag, so the clause order round-trips; SelectDeParser follows the same order. Bare FROM t queries and BigQuery pipe syntax are untouched.
UnPivotQuery mirrors the existing PivotQuery, added as default visitor methods so no third-party visitor breaks.
New keywords are non-reserved and stay usable as identifiers; regression tests cover glob, anti, load, install, attach, detach and pragma.

Test plan

New tests in DuckDBTest, each feature with a regression case for the syntax it could have shadowed (POSITION(...), MySQL ALGORITHM COPY, Oracle CONNECT BY, PostgreSQL CREATE EXTENSION, EXECUTE FUNCTION triggers, FROM-clause UNPIVOT (...), pipe queries).
Full suite: 7050 tests, 1 failure — SpecialOracleTest.testAllSqlsParseDeparse, which fails on an unmodified master in this environment too.
JavaCC still reports 0 errors and 1 warnings, the same warning as on master.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds GLOB as a LikeExpression keyword so `col GLOB 'pattern'` and `col NOT GLOB 'pattern'` parse and deparse. GLOB stays a non-reserved keyword and remains usable as an identifier. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

DuckDB allows SEMI/ANTI JOIN without a LEFT prefix, and ANTI as a join type at all. ANTI previously parsed silently as a table alias, producing a wrong AST from valid SQL. Adds Join.isAnti(), the LEFT/RIGHT/FULL SEMI|ANTI prefixes and the standalone form, keeping ANTI usable as an identifier outside a join prefix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

DuckDB parameterises a type with other types, e.g. MAP(VARCHAR, INTEGER). Type arguments previously accepted identifiers and literals but not data type tokens. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

DuckDB matches an INSERT's source columns either positionally or by name. BY previously bound as a table alias, swallowing the clause. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

DuckDB writes the sample size first and the method behind it: USING SAMPLE 10%, USING SAMPLE 10 PERCENT (bernoulli), USING SAMPLE 10 ROWS (system, 377), USING SAMPLE reservoir (50 ROWS). The method is now optional, RESERVOIR is a known method, and a bare sample size accepts a unit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

DESCRIBE now accepts a query as well as a table, so DESCRIBE SELECT ..., DESCRIBE FROM ... and DESCRIBE WITH ... parse. The described query is reachable from TablesNamesFinder and rendered by the deparser. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds PRAGMA name, PRAGMA name(arguments) and PRAGMA name = value, as used by DuckDB and SQLite to read and set database settings. PRAGMA stays a non-reserved keyword. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds [FORCE] INSTALL extension [FROM repository] and LOAD extension, where the extension and repository may be an identifier or a string literal. INSTALL and LOAD stay non-reserved keywords. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

manticore-projects · 2026-09-15T14:12:11Z

Run './gradlew spotlessApply' to fix all violations.

Adds ATTACH [DATABASE] [IF NOT EXISTS] path [AS alias] [(options)] and DETACH [DATABASE] [IF EXISTS] name, with options such as READ_ONLY or TYPE SQLITE kept as strings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds CONNECT target [AS alias] and DISCONNECT [name], which route a session's queries to a remote database. Oracle's CONNECT BY is unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds PREPARE name AS statement and DEALLOCATE [PREPARE] name. EXECUTE of a prepared statement already parsed. The prepared statement is a nested Statement, so TablesNamesFinder reaches its tables. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds COPY table [(columns)] FROM path [(options)] and COPY {table | (query)} TO path [(options)]. Options such as FORMAT PARQUET or AUTO_DETECT true are free-form keywords, so they are collected verbatim. COPY stays valid as MySQL's ALGORITHM value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds CREATE [OR REPLACE] [TEMPORARY] MACRO name (parameters) AS body for both scalar macros and table macros (AS TABLE SELECT ...). Parameters may carry a default value, as in add(a, b := 5). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds CREATE EXTENSION REPOSITORY [IF NOT EXISTS] name [WITH PREFIX prefix], which registers a custom repository to install extensions from. PostgreSQL's CREATE EXTENSION is unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

DuckDB writes CREATE TRIGGER ... REFERENCING OLD TABLE AS o NEW TABLE AS n FOR EACH STATEMENT <statement>, i.e. the standard header with a SQL statement instead of EXECUTE FUNCTION. The standard-form parser now accepts a statement body, and such triggers are recognised and rendered as standard rather than MySQL-style triggers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

hayssams · 2026-09-15T14:45:53Z

@manticore-projects
Fixed
Thanks!

DuckDB lets the FROM clause lead, with the SELECT clause optional: FROM tbl SELECT a and FROM tbl WHERE a > 1. PlainSelect records the order in a fromFirst flag, and both the AST's own rendering and SelectDeParser emit the clauses accordingly. Bare FROM queries and the pipe syntax are unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds the simplified statement form UNPIVOT tbl ON columns [INTO NAME name VALUE values] as UnPivotQuery, the counterpart of the existing PivotQuery. The FROM-clause form UNPIVOT (...) is unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds JOIN t APPROX NEAREST n BY SIMILARITY expression, the vector similarity join of DuckDB 2.0. APPROX no longer binds as a table alias in that position. Also adds the lookahead the UNPIVOT statement's INTO clause needed to keep the grammar free of choice conflicts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

Adds WITH RECURSIVE tbl (a, b) USING KEY (a, avg(b)) AS (...), which aggregates a recursive CTE by key instead of accumulating every row. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqinwHBKuAtPEmtXr3b5P2

manticore-projects · 2026-09-17T00:20:03Z

Please resolve conflicts.

hayssams · 2026-09-17T07:27:46Z

@manticore-projects
Just fixed the conflicts
thanks

@OverRide

…TA lookahead The merge of master resolved 26 conflict hunks by concatenating both sides while dropping the closing braces, @OverRide annotations and return statements that git had hoisted out of the conflict markers, leaving 6 visitor/deparser classes and the grammar unparseable. Also exclude LOAD DATA from the ExtensionStatement lookahead so BigQuery's LOAD DATA statement is not swallowed by DuckDB's LOAD extension alternative. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

manticore-projects · 2026-09-17T08:13:25Z

Merci beaucoups!

guizmaii · 2026-09-18T01:40:55Z

@hayssams Thanks for this! I was waiting for this feature for more than a year now :)

@manticore-projects Is it possible to release a new version with this feature, please?

manticore-projects · 2026-09-18T01:47:23Z

@manticore-projects Is it possible to release a new version with this feature, please?

Done, though it may take a few minutes to appear on Maven Central.

guizmaii · 2026-09-18T02:00:58Z

Thanks!

hayssams · 2026-09-18T11:08:05Z

Thanks @manticore-projects
Thanks for pushing this @guizmaii

guizmaii · 2026-09-21T00:49:52Z

@manticore-projects I don't see any new release after 5.4 in Maven Central.

Did the release process fail?

manticore-projects · 2026-09-21T01:34:33Z

@manticore-projects I don't see any new release after 5.4 in Maven Central.

Did the release process fail?

Manticore Artifacts: https://central.sonatype.com/artifact/com.manticore-projects.jsqlformatter/jsqlparser

guizmaii · 2026-09-22T05:30:17Z

@manticore-projects Why are there 2 namespaces, both being updated: com.manticore-projects.jsqlformatter and com.github.jsqlparser ?

Why is com.github.jsqlparser following the changelogs, while there's no changelog for com.manticore-projects.jsqlformatter? 🤔

manticore-projects · 2026-09-22T05:44:35Z

Because I run various projects depending on JSQLParser and depend often on the latest features, while @wumpz releases only from time to time. (There has been com.github.jsqlparser update for more than 1 year until recently).
Also, I believe in rolling releases and see the JSQLParser version number as rather meaningless while @wumpz has as more traditional view on this.

https://manticore-projects.com/JSQLParser/changelog.html

hayssams · 2026-09-22T05:54:03Z

@guizmaii
It definitely would have been better to see JSQLParser released frequently from JSQLParser but since nothing goes into manticore.jsqlparser before going into jsqlparser.jsqlparser I always found it safe to use manticore releases in my dependencies.

hayssams and others added 8 commits September 15, 2026 14:18

hayssams and others added 7 commits September 15, 2026 16:13

hayssams force-pushed the feature/duckdb-syntax branch from f74c137 to a765cf7 Compare September 15, 2026 14:14

hayssams and others added 4 commits September 15, 2026 18:05

hayssams force-pushed the feature/duckdb-syntax branch from a765cf7 to c04286d Compare September 15, 2026 16:05

Merge branch 'master' into feature/duckdb-syntax

aee7cbc

manticore-projects merged commit c945986 into JSQLParser:master Sep 17, 2026
10 checks passed

Uh oh!

Conversation

hayssams commented Sep 15, 2026

Features, with the DuckDB documentation that specifies them

Notes

Test plan

Uh oh!

manticore-projects commented Sep 15, 2026

Uh oh!

hayssams commented Sep 15, 2026

Uh oh!

manticore-projects commented Sep 17, 2026

Uh oh!

hayssams commented Sep 17, 2026

Uh oh!

Uh oh!

manticore-projects commented Sep 17, 2026

Uh oh!

guizmaii commented Sep 18, 2026

Uh oh!

manticore-projects commented Sep 18, 2026

Uh oh!

guizmaii commented Sep 18, 2026

Uh oh!

hayssams commented Sep 18, 2026

Uh oh!

guizmaii commented Sep 21, 2026

Uh oh!

manticore-projects commented Sep 21, 2026

Uh oh!

guizmaii commented Sep 22, 2026

Uh oh!

manticore-projects commented Sep 22, 2026

Uh oh!

hayssams commented Sep 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants