Skip to content

feat(models): make retrieval and fusion score semantics explicit (#90) - #142

Closed
Muhammad-AbdullahGhani wants to merge 1 commit into
grayhatdevelopers:mainfrom
Muhammad-AbdullahGhani:feat/clear-ranking-scores
Closed

Muhammad-AbdullahGhani wants to merge 1 commit into
grayhatdevelopers:mainfrom
Muhammad-AbdullahGhani:feat/clear-ranking-scores

Conversation

@Muhammad-AbdullahGhani

Copy link
Copy Markdown
Contributor

Related issue

Closes #90

Summary

  • Add explicit ranking and score semantics across models, CLI, evidence delivery, and MCP tools:
    • SearchHit: added channel_rank, score_kind='ordering_only', score_direction='higher_is_better', score_conversion='negated_distance', distance_metric='cosine', and distance_direction='lower_is_better'.
    • FusedMoment: added combined_rank, score_kind='ordering_only', score_direction='higher_is_better', scoring_method='reciprocal_rank_fusion', contributing_channels, and channels_run.
    • FusionProvenance: added score_kind='ordering_only', score_direction='higher_is_better', and scoring_method='reciprocal_rank_fusion'.
    • EvidenceBoardCandidate: added score_kind='ordering_only' and score_direction='higher_is_better' when candidate score is present.
  • Backwards compatibility: all new fields include defaults and validators so existing callers and consumers remain fully compatible.
  • CLI: updated emit_search table caption and headers (Score (RRF ↑), Contributing Channels) to clarify that scores are ordering-only values rather than calibrated probabilities.
  • MCP: documented in search_moments and evidence_index that candidate scores and ranks are uncalibrated ordering values where higher is better.

Validation

  • pytest tests/test_search.py tests/test_search_fusion.py tests/test_query_service.py tests/test_evidence_delivery.py -p no:asyncio (37 passed)
  • pytest tests/test_cli.py -k search -p no:asyncio (2 passed)
  • ruff check src/vidxp/application_models.py src/vidxp/capabilities/search.py src/vidxp/search_fusion.py src/vidxp/cli_support.py src/vidxp/evidence_delivery.py src/vidxp/mcp.py tests/test_search.py tests/test_search_fusion.py (All checks passed)
  • ruff format --check src/vidxp/application_models.py src/vidxp/capabilities/search.py src/vidxp/search_fusion.py src/vidxp/cli_support.py src/vidxp/evidence_delivery.py src/vidxp/mcp.py tests/test_search.py tests/test_search_fusion.py (8 files already formatted)

@SaadBazaz

Copy link
Copy Markdown
Member

Hey @Muhammad-AbdullahGhani,
Thanks for the Pull Request.
As a starter, we require all contributors to "star" and "fork" the repo, in order to determine if there really is a human behind the wheel or an autonomous agent.
Please star the repo for our review to proceed.
Thanks!

@Muhammad-AbdullahGhani

Copy link
Copy Markdown
Contributor Author

Hi @SaadBazaz, I have starred and forked the repo..

@tulayha

tulayha commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Thanks for working on this. We’re continuing with #144 for #90, so I’m closing this overlapping implementation.

This version hardcodes cosine even though indexes can use other distance metrics, and adds duplicate rank and channel fields that need to stay synchronized. #144 reads the configured metric and uses the existing fields, though it still needs changes from review.

If you’d like to contribute further to this issue, feel free to coordinate with @AhmedIrfan7 on #144. I’ve left specific feedback there that you could work through together.

Your bounded-moment fix in #141 is merged. Thanks for following through on that.

@tulayha tulayha closed this Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make retrieval and fusion scores clear to callers

4 participants