0.0.244

[joocer]

Restore Windows wheels, and fix what turned up on the way

Windows wheels stopped at 0.0.230. This restores them and fixes the issues found reviewing the release path.

Why Windows builds stopped

PyPI and git together tell the story:

• June 2025 — c1d0937 dropped cp313 from the Windows matrix. 0.0.230 on PyPI confirms it: win_amd64 for cp39/310/311/312, no cp313. Never fixed, just excluded.
• Feb 2026 — 9ed6ec0/96d3854 added 3.14/3.14t and set the Windows matrix back to ['3.9'…'3.13','3.14','3.14t'], re-including the known-broken 3.13. Since upload had needs: build-windows, that job failing blocked the whole publish. 98ed6f0 (0.0.233) deleted the Windows job — and that same day carries seven version bumps (0.0.233→0.0.239).

The underlying build bug: setup.py fed ["-O2", "-march=native"] to MSVC, which only emits D9002/LNK4044 "unknown option" diagnostics — every Windows wheel we ever shipped was built unoptimised. Combined with downloading VS2019 Build Tools (incl. VC 14.0/14.1) onto a runner that already has VS2022, that's the most likely cp313 breakage: CPython 3.13+ extensions need MSVC 14.3x.

Without the CI logs (long expired) the 3.13 root cause can't be proven. The fix removes the whole class rather than guessing.

Changes

Windows — MSVC /O2 and split LINK_FLAGS; VS2019 download deleted; wheels verified to contain both native extensions and carry the win_amd64 tag; Windows added to the regression suite so this fails in CI, not at release.

Release workflow — ~100 lines of per-artifact staging replaced with one pattern: dist-* download (that duplication is why macOS cp314t went missing from 0.0.243 unnoticed); free-threaded 3.14t removed; the dead raspberrypi job removed (last aarch64 wheel: Feb 2024, and it carried its own independent PyPI publish); Rust toolchain and setuptools_rust removed — there is no Rust here.

nbytes() — two bugs, one fix:

• DataFrame(rows=..., schema=...) reported 0 bytes forever; the len()==0 guard was dropped in 0.0.144 (2024).
• Polling nbytes() while appending was quadratic. 0.0.243's fix instead paid a full msgpack serialise on every append. Now a watermark sizes only rows appended since the last call — no serialisation in append, linear polling.

   rows   0.0.242 O(n²)   fixed O(n)   speedup
  1,000          0.025s       0.001s      21.5x
  4,000          0.385s       0.005s      84.2x
 16,000          6.137s       0.019s     322.0x
 32,000         25.050s       0.043s     585.2x

4× rows → 16× time. 128k extrapolates to ~7 minutes; it is now 0.225s.

Row.nbytes() also no longer enforces MAXIMUM_RECORD_SIZE — sizing a frame must not fail because one row is too big to serialise (arrow() handles it fine). as_bytes still refuses. Verified nbytes() == len(as_bytes) exactly for normal rows.

JSON output — 133e940 ("remove orjson") swapped orjson.dumps → json.dumps in both Row.as_json and the JSONB write path, silently adding ", "/": " padding to bytes we store. Compact separators restored in both. simdjson is now used for loads only — its dumps delegates to stdlib without separator control and would reintroduce the padding.

sdist was unbuildable — it shipped without requirements.txt, so setup.py fell back to egg-info/requires.txt, whose [simdjson] extras section is not a valid requirement specifier:

error in orso setup command: 'install_requires' must be a string or iterable of
strings containing valid project/version requirement specifiers... [simdjson]

pip install orso failed outright on any platform without a wheel. Added MANIFEST.in; fallback no longer parses extras sections. Verified end-to-end in a clean venv.

Other — PyArrow type map cached instead of rebuilt per column per call (2.7× faster schema conversion, ~17% off .arrow() for small wide frames; output verified identical across all 52 type/nullable/precision cases). Optional integration imports guarded with pytest.importorskip (opteryx and polars have no Windows or 3.14 wheels). 3.14 added to the regression matrix — we ship 3.14 wheels but never tested them. varchar_array.pyx removed (not built, not imported) and the disabled macOS workflow deleted.

Testing

421 passed, 2 skipped.

Not addressed

• aarch64 is now unbuilt. Removing the dead job changes nothing shipped (nothing since Feb 2024), and the sdist fix means ARM users can now build from source — which they previously could not. rugo already uses the free hosted ubuntu-24.04-arm runner; that would slot in as a normal matrix entry.
• True is accepted for an INTEGER column (stored as True, not 1) since isinstance(True, int); {"d": 2} is rejected for a DOUBLE column. Both pre-existing.
• workflow_dispatch + Test PyPI dry-run, Trusted Publishing (OIDC) instead of PYPI_API_TOKEN, and cibuildwheel to replace the hand-rolled docker/build.sh/auditwheel matrix.