Property-based testing with Hypothesis across the numeric surface - #46
Merged
Conversation
Add hypothesis as a dev dependency and a small standalone test file exercising VCSC/VCSR round-trip (from_scipy -> toarray) and matmul against a dense reference. Unlike the fixed-shape fixtures in conftest.py, Hypothesis chooses shapes/values/densities itself and shrinks any failure to a minimal example. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add a shared tests/_hypothesis_strategies.py (dense_matrices, signed_dense_matrices, axis_key) and property-based test files covering the numeric-correctness ground previously spread across fixed seeded-rng fixtures: - test_property_ops.py: scalar mul/div, neg, transpose, log1p, matvec/matmat. - test_property_reductions_arith.py: per-axis sum/mean/max/min and elementwise +/-/multiply. - test_property_indexing.py: general (row-key, col-key) indexing (slices, fancy lists with duplicates/negatives, boolean masks) against dense reference indexing -- generalizes the duplicate-index "fan out" regression and the manual 25-iteration fuzz loop that used to cover it by hand. - test_property_transpose.py: _transpose_major correctness (match dense, nnz-preserving, involutive), replacing test_construct.py. - test_property_normalization.py: the default normalized() view, every RECIPES entry, and select(), against a plain-numpy reference. This caught a real gap in the *test* oracle: the reference formula didn't mirror _compute_row_scale's `target <= 0.0` guard (median row total <= 0 skips depth normalization entirely), a case the old fixed-seed fixtures never happened to hit. Removed the tests these strictly generalize, keeping error-path, structural, memory-budget, and identity/caching tests in place (those aren't properties that hold for arbitrary input, or need designed data, so they stay as example-based tests): test_roundtrip.py and test_construct.py deleted outright (fully subsumed); trimmed test_ops.py, test_reductions_and_arith.py, test_indexing.py, test_general_indexing_nnz_astype.py, test_select_minor_fanout.py, test_vcs_norm.py, test_vcs_norm_recipes.py, and test_norm_selection.py to drop only their redundant parts. Left untouched (not good property-testing candidates): constructor validation (test_base.py), dtype-narrowing/boundary tests keyed on INT32_MAX (test_index_dtypes.py), tracemalloc memory-budget tests (test_reduction_memory.py, and the chunk-budget mechanics in test_chunked_transpose.py), and AnnData/HDF5/Zarr I/O, categorical encoding, and compression-backend dispatch (test_anndata.py, test_anndata_class.py, test_compression.py, test_metadata_encoding.py, test_rapid_load.py, test_ivcsc.py) -- these test file-format/I/O contracts and specific error messages rather than "holds for any matrix" properties. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends property-based testing (Hypothesis) across the rest of the library's numeric surface, replacing fixed seeded-rng fixtures with generated shapes/values/densities where that's a strict generalization, and removing the tests it makes redundant.
Added
tests/_hypothesis_strategies.py(shareddense_matrices,signed_dense_matrices,axis_keystrategies, plus thedeadline=None/too_slow-suppressed settings needed because the compress/matmul/select kernels are@numba.njit(cache=True)) and:(row_key, col_key)indexing (slices, fancy lists with duplicates/negatives, boolean masks) against dense reference indexing. Generalizes the duplicate-index "fan out" regression and the manual 25-iteration fuzz loop that used to cover it by hand._transpose_majorcorrectness (matches dense, nnz-preserving, involutive); replacestest_construct.py.normalized()view, everyRECIPESentry, andselect(), against a plain-numpy reference.That last one caught a real gap in the test oracle, not the library: the reference formula didn't mirror
_compute_row_scale'starget <= 0.0guard (a non-positive median row total skips depth normalization entirely). The old fixed-seed fixtures never happened to construct a matrix that hit it; Hypothesis found it within the first run. Fixed in the new reference.Removed
test_roundtrip.pyandtest_construct.py(fully subsumed, deleted outright); trimmed the redundant parts oftest_ops.py,test_reductions_and_arith.py,test_indexing.py,test_general_indexing_nnz_astype.py,test_select_minor_fanout.py,test_vcs_norm.py,test_vcs_norm_recipes.py, andtest_norm_selection.py— keeping their error-path, structural, memory-budget, and identity/caching tests, which aren't "holds for any input" properties.Deliberately left alone
Not good property-testing candidates: constructor validation (
test_base.py), dtype-narrowing/boundary tests keyed onINT32_MAX(test_index_dtypes.py),tracemallocmemory-budget tests (test_reduction_memory.py, and the chunk-budget mechanics intest_chunked_transpose.py), and AnnData/HDF5/Zarr I/O, categorical encoding, and compression-backend dispatch (test_anndata.py,test_anndata_class.py,test_compression.py,test_metadata_encoding.py,test_rapid_load.py,test_ivcsc.py) — these test file-format/I/O contracts and specific error messages, not numeric properties.Test plan
uv run ruff check ./uv run ruff format --check(on touched files) /uv run ty checkall cleanuv run pytest -q— 1189 passed, 45 skipped