Skip to content

Property-based testing with Hypothesis across the numeric surface - #46

Merged
aarmey merged 4 commits into
mainfrom
hypothesis-prototype
Sep 12, 2026
Merged

Property-based testing with Hypothesis across the numeric surface#46
aarmey merged 4 commits into
mainfrom
hypothesis-prototype

Conversation

@aarmey

@aarmey aarmey commented Sep 12, 2026

Copy link
Copy Markdown
Member

Summary

Extends property-based testing (Hypothesis) across the rest of the library's numeric surface, replacing fixed seeded-rng fixtures with generated shapes/values/densities where that's a strict generalization, and removing the tests it makes redundant.

Added tests/_hypothesis_strategies.py (shared dense_matrices, signed_dense_matrices, axis_key strategies, plus the deadline=None/too_slow-suppressed settings needed because the compress/matmul/select kernels are @numba.njit(cache=True)) and:

  • test_property_ops.py — scalar mul/div, neg, transpose, log1p, matvec/matmat vs. a dense reference.
  • test_property_reductions_arith.py — per-axis sum/mean/max/min and elementwise +/-/multiply.
  • test_property_indexing.py — general (row_key, col_key) indexing (slices, fancy lists with duplicates/negatives, boolean masks) against dense reference indexing. Generalizes the duplicate-index "fan out" regression and the manual 25-iteration fuzz loop that used to cover it by hand.
  • test_property_transpose.py_transpose_major correctness (matches dense, nnz-preserving, involutive); replaces test_construct.py.
  • test_property_normalization.py — the default normalized() view, every RECIPES entry, and select(), against a plain-numpy reference.

That last one caught a real gap in the test oracle, not the library: the reference formula didn't mirror _compute_row_scale's target <= 0.0 guard (a non-positive median row total skips depth normalization entirely). The old fixed-seed fixtures never happened to construct a matrix that hit it; Hypothesis found it within the first run. Fixed in the new reference.

Removed

test_roundtrip.py and test_construct.py (fully subsumed, deleted outright); trimmed the redundant parts of test_ops.py, test_reductions_and_arith.py, test_indexing.py, test_general_indexing_nnz_astype.py, test_select_minor_fanout.py, test_vcs_norm.py, test_vcs_norm_recipes.py, and test_norm_selection.py — keeping their error-path, structural, memory-budget, and identity/caching tests, which aren't "holds for any input" properties.

Deliberately left alone

Not good property-testing candidates: constructor validation (test_base.py), dtype-narrowing/boundary tests keyed on INT32_MAX (test_index_dtypes.py), tracemalloc memory-budget tests (test_reduction_memory.py, and the chunk-budget mechanics in test_chunked_transpose.py), and AnnData/HDF5/Zarr I/O, categorical encoding, and compression-backend dispatch (test_anndata.py, test_anndata_class.py, test_compression.py, test_metadata_encoding.py, test_rapid_load.py, test_ivcsc.py) — these test file-format/I/O contracts and specific error messages, not numeric properties.

Test plan

  • uv run ruff check . / uv run ruff format --check (on touched files) / uv run ty check all clean
  • uv run pytest -q — 1189 passed, 45 skipped

aarmey and others added 2 commits September 12, 2026 06:52
Add hypothesis as a dev dependency and a small standalone test file
exercising VCSC/VCSR round-trip (from_scipy -> toarray) and matmul
against a dense reference. Unlike the fixed-shape fixtures in
conftest.py, Hypothesis chooses shapes/values/densities itself and
shrinks any failure to a minimal example.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add a shared tests/_hypothesis_strategies.py (dense_matrices,
signed_dense_matrices, axis_key) and property-based test files covering
the numeric-correctness ground previously spread across fixed
seeded-rng fixtures:

- test_property_ops.py: scalar mul/div, neg, transpose, log1p,
  matvec/matmat.
- test_property_reductions_arith.py: per-axis sum/mean/max/min and
  elementwise +/-/multiply.
- test_property_indexing.py: general (row-key, col-key) indexing
  (slices, fancy lists with duplicates/negatives, boolean masks)
  against dense reference indexing -- generalizes the duplicate-index
  "fan out" regression and the manual 25-iteration fuzz loop that used
  to cover it by hand.
- test_property_transpose.py: _transpose_major correctness
  (match dense, nnz-preserving, involutive), replacing test_construct.py.
- test_property_normalization.py: the default normalized() view, every
  RECIPES entry, and select(), against a plain-numpy reference. This
  caught a real gap in the *test* oracle: the reference formula didn't
  mirror _compute_row_scale's `target <= 0.0` guard (median row total
  <= 0 skips depth normalization entirely), a case the old fixed-seed
  fixtures never happened to hit.

Removed the tests these strictly generalize, keeping error-path,
structural, memory-budget, and identity/caching tests in place (those
aren't properties that hold for arbitrary input, or need designed
data, so they stay as example-based tests):
test_roundtrip.py and test_construct.py deleted outright (fully
subsumed); trimmed test_ops.py, test_reductions_and_arith.py,
test_indexing.py, test_general_indexing_nnz_astype.py,
test_select_minor_fanout.py, test_vcs_norm.py, test_vcs_norm_recipes.py,
and test_norm_selection.py to drop only their redundant parts.

Left untouched (not good property-testing candidates): constructor
validation (test_base.py), dtype-narrowing/boundary tests keyed on
INT32_MAX (test_index_dtypes.py), tracemalloc memory-budget tests
(test_reduction_memory.py, and the chunk-budget mechanics in
test_chunked_transpose.py), and AnnData/HDF5/Zarr I/O, categorical
encoding, and compression-backend dispatch (test_anndata.py,
test_anndata_class.py, test_compression.py, test_metadata_encoding.py,
test_rapid_load.py, test_ivcsc.py) -- these test file-format/I/O
contracts and specific error messages rather than "holds for any
matrix" properties.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@aarmey aarmey changed the title Prototype: property-based testing with Hypothesis Property-based testing with Hypothesis across the numeric surface Sep 12, 2026
@aarmey
aarmey merged commit 070d5fc into main Sep 12, 2026
6 checks passed
@aarmey
aarmey deleted the hypothesis-prototype branch September 12, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant