fix: honor distance_threshold=0 in semantic cache and message history#650
Conversation
A per-call distance_threshold of 0 was resolved with a truthiness check (`distance_threshold or self._distance_threshold`) and therefore silently replaced by the instance default. Callers requesting exact-match-only lookups received fuzzy cache hits instead. 0 is a valid threshold: set_threshold() accepts 0 <= x <= 2 and VectorRangeQuery.set_distance_threshold() rejects only negatives. Only None should fall back to the configured default. Affects SemanticCache.check(), SemanticCache.acheck(), and SemanticMessageHistory.get_relevant(). Adds regression tests covering the sync and async cache paths.
There was a problem hiding this comment.
Pull request overview
This PR fixes an API correctness bug where passing distance_threshold=0 (intended to mean exact-match-only) was silently ignored due to a truthiness fallback, causing unexpected fuzzy cache hits. The change aligns per-call overrides with the existing contract that 0 is a valid distance threshold.
Changes:
- Update
SemanticCache.check()andSemanticCache.acheck()to fall back to the instance default only whendistance_threshold is None(so0is honored). - Update
SemanticMessageHistory.get_relevant()to use the sameNone-only fallback semantics fordistance_threshold. - Add integration regression tests covering the sync/async semantic cache
distance_threshold=0behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
redisvl/extensions/cache/llm/semantic.py |
Fix per-call threshold override logic in check() / acheck() to honor distance_threshold=0. |
redisvl/extensions/message_history/semantic_history.py |
Fix per-call threshold override logic in get_relevant() to honor distance_threshold=0. |
tests/integration/test_llmcache.py |
Add integration regression tests for check() / acheck() when distance_threshold=0. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Addresses review feedback: the falsy-override fix in semantic_history.py had no regression test on that code path.
|
Fixed this up in 5620780. Added Verified against the pre-fix source: with |
vishal-bala
left a comment
There was a problem hiding this comment.
LGTM! Thanks for contributing 🙌
Fixes #649
Problem
The per-call
distance_thresholdoverride was resolved with a truthiness check:0.0is falsy, so a caller passingdistance_threshold=0(exact-match-only) had it silently replaced by the instance default and received fuzzy cache hits instead.0is a valid threshold everywhere else:set_threshold()accepts0 <= x <= 2andVectorRangeQuery.set_distance_threshold()rejects only negatives.The failure mode is a false cache hit so the caller gets a cached response to a prompt they never asked, believing fuzzy matching was off.
Fix
Fall back to the configured default only when the override is
None. Applied inSemanticCache.check(),SemanticCache.acheck(), andSemanticMessageHistory.get_relevant().Tests
Added
test_check_with_zero_distance_thresholdandtest_acheck_with_zero_distance_thresholdtotests/integration/test_llmcache.py. Each stores a prompt, asserts a near-paraphrase does not hit atdistance_threshold=0, and asserts it does hit at the cache default (so the test fails for the right reason rather than by accident).Verified against the real suite (Python 3.12, Redis 8.4 via the repo's compose fixture):
AssertionError: assert [{'entry_id': ...}] == []tests/integration/test_llmcache.py: 62 passedScope note
I left the
ttl = ttl or self._ttloccurrences alone. A TTL of 0 isn't meaningful in Redis and the suite already rejects it (test_bad_ttl), so the falsy-fallback is harmless there. Happy to widen the scope if maintainers prefer consistency.Note
Low Risk
Small, targeted behavior fix in lookup paths with clear regression tests; no API or schema changes beyond honoring an already-valid threshold value.
Overview
Fixes a bug where per-call
distance_thresholdoverrides useddistance_threshold or self._distance_threshold, so0(exact-match-only in Redis COSINE distance) was treated as unset and replaced by the instance default—causing false cache hits and fuzzy context when callers intended strict matching.SemanticCache.check(),SemanticCache.acheck(), andSemanticMessageHistory.get_relevant()now fall back to the configured default only when the override isNone.Integration tests cover sync/async cache checks and semantic history retrieval: near-paraphrases must not match at
distance_threshold=0, while the same queries still match at the default threshold.Reviewed by Cursor Bugbot for commit 5620780. Bugbot is set up for automated code reviews on this repo. Configure here.