[WIP] Build free-threaded (cp313t/cp314t) wheels and declare GIL-free support - #24
Open
dance858 wants to merge 2 commits into
Open
[WIP] Build free-threaded (cp313t/cp314t) wheels and declare GIL-free support#24dance858 wants to merge 2 commits into
dance858 wants to merge 2 commits into
Conversation
Closes #18. The extension now calls PyUnstable_Module_SetGIL(Py_MOD_GIL_NOT_USED) on free-threaded CPython, so importing it no longer re-enables the GIL. The build matrix gains an `abi` axis ("" / "t") for 3.13 and 3.14, opts in via CIBW_ENABLE=cpython-freethreading, and the wheel test asserts the GIL stays disabled after import on the t builds. Audit backing the declaration (engine d8cb9b8 + these bindings): - nm on libdnlp_diff.a shows no writable data/bss symbols; the only globals in the engine are the SP_TRACK_MEMORY byte counters, which the wheel build does not enable. No function-local statics, no non-reentrant libc. - All version counters (values_version, *_seen) are per-matrix/per-expr. - Every problem/expr owns its buffers; forward, update_params and the problem constructor memcpy their inputs, and every wrapper copies results into fresh NumPy arrays, so no Python memory is aliased across calls. - The only static state in the bindings is the NumPy-init flag and the module def, both written once at import under the import lock. - expr refcounts are plain ints, so a single expr/problem capsule must not be used from two threads at once -- the same contract as with the GIL. Verified locally on Homebrew python3.14t: import keeps the GIL disabled and 16 threads x 200 evaluations on distinct problems are bit-identical to a single-threaded reference (forward, jacobian, hessian). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aQs1u37EG6XTHFrcaBRkA
NumPy >= 2.5 ships no cp313t wheels on any platform, so the 3.13t builds had to compile NumPy from source: macOS failed outright, Windows took six minutes for the NumPy build alone, and Linux aarch64 under QEMU was still running after an hour. Only 3.14t is built now. The macOS 3.14t job built and tested both wheels fine (GIL stays disabled on x86_64 and arm64) but then failed in the twine step with "pip: command not found". Call pip and twine via python -m so the step does not depend on a pip launcher being on PATH. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aQs1u37EG6XTHFrcaBRkA
Transurgeon
reviewed
Sep 6, 2026
Comment on lines
+35
to
+45
| # "" builds the default (GIL) interpreter, "t" the free-threaded one. | ||
| # Only 3.14t: NumPy >= 2.5 ships no cp313t wheels on any platform, so a | ||
| # 3.13t build would have to compile NumPy from source. | ||
| abi: [ "", "t" ] | ||
| exclude: | ||
| - python-version: "3.11" | ||
| abi: "t" | ||
| - python-version: "3.12" | ||
| abi: "t" | ||
| - python-version: "3.13" | ||
| abi: "t" |
Member
There was a problem hiding this comment.
instead of this, we should maybe just add 3.14t?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #18.
What
bindings.c: on free-threaded CPython, callPyUnstable_Module_SetGIL(module, Py_MOD_GIL_NOT_USED)after module creation so importing_sparsediffengineno longer re-enables the GIL. No-op on regular builds.abimatrix axis (""/"t"), buildcp314twheels, setCIBW_ENABLE=cpython-freethreading, and extend the wheel test to assertsys._is_gil_enabled()is stillFalseafter import on thetbuild. Artifact names get thetsuffix.pip/twineviapython -min the wheel-check step (one macOS job failed there withpip: command not found).No 3.13t wheel. NumPy >= 2.5 ships no
cp313twheels on any platform, so a 3.13t build has to compile NumPy from source in the isolated build env. In the first CI run that failed on macOS, took 6 minutes on Windows, and was still running after an hour on Linux aarch64 under QEMU. Free-threading users on 3.13 would hit the same NumPy problem at install time anyway.Why it is safe to declare GIL-free
Audit of the engine (submodule at d8cb9b8) and the bindings:
nmonlibdnlp_diff.ashows no writable data/bss symbols. The only globals in the engine source are theSP_TRACK_MEMORYbyte counters, which the wheel build does not enable. No function-local statics, no non-reentrant libc calls.values_version,jacobian_csc_seen,csc_seen,transpose_seen) live inside the owning matrix or expr node.variableforward,problem_update_params,new_parameterandnew_problemmemcpytheir inputs; every wrapper copies results into fresh NumPy arrays. No Python memory is aliased across calls.Caveat, documented in the README:
exprrefcounts are plain ints, so one expr/problem capsule must not be used from two threads at once. That is the same contract as under the GIL, which only serialized individual calls and never protected against interleaved use of one object.Verification
-Wall -Wextra(regular headers, and withPy_GIL_DISABLEDdefined).cp314textension against Homebrewpython3.14t3.14.7: import keeps the GIL disabled (the exact check CI now runs).cp314twheels built and passed the GIL check on Linux x86_64, Windows, and macOS x86_64 + arm64 (universal2).🤖 Generated with Claude Code
https://claude.ai/code/session_013aQs1u37EG6XTHFrcaBRkA