Skip to content

feat(backtest): add benchmark-relative information ratio metrics - #264

Closed
ZhangWT02 wants to merge 1 commit into
OpenByteInc:mainfrom
ZhangWT02:feat/information-ratio
Closed

ZhangWT02 wants to merge 1 commit into
OpenByteInc:mainfrom
ZhangWT02:feat/information-ratio

Conversation

@ZhangWT02

Copy link
Copy Markdown

Summary

Adds benchmark-relative Information Ratio analytics to the Strategy V2 backtest service.

The implementation calculates benchmark-relative metrics from the existing portfolio and benchmark curves. CDI data ingestion is intentionally excluded from this first scoped implementation.

Related issue

Changes

  • Align portfolio and benchmark returns over identical start and end timestamps.
  • Report annualized portfolio, benchmark, and active returns.
  • Calculate annualized tracking error using sample standard deviation (ddof=1).
  • Calculate and classify the Information Ratio.
  • Report status, observation count, frequency, and annualization factor.
  • Require at least two aligned return observations to calculate tracking error and Information Ratio.
  • Exclude invalid, non-finite, and non-positive curve observations.
  • Handle insufficient history and zero tracking error explicitly.
  • Preserve negative Information Ratios.
  • Add deterministic unit tests and Strategy V2 integration coverage.
  • Document interval-alignment, annualization, edge-case behavior, and CDI benchmark suitability.

Annualization uses the existing Strategy V2 convention: 252 trading days for non-crypto markets, 365.25 days for crypto markets, corresponding session lengths for intraday frequencies, and 52 periods for weekly data.

CDI ingestion, the canonical CDI benchmark identifier, and Brazilian business-calendar behavior are intentionally deferred to a follow-up design.

Test plan

  • Tested locally with docker compose up -d --build
  • Backend logs show no errors
  • Relevant pytest tests pass

Results:

  • Docker Compose stack started successfully.
  • All long-running Compose services reported healthy.
  • Database migration completed successfully with exit code 0.
  • Focused pytest result: 24 passed.
  • git diff --check passed.

API documentation (if routes/schemas changed)

  • Regenerated docs/api/openapi.yaml (cd backend_api_python && python scripts/export_openapi.py)
  • Updated docs/agent/agent-openapi.json if /api/agent/v1 routes changed
  • Breaking API changes called out below (oasdiff will fail CI otherwise)

No routes or declared OpenAPI schemas were changed. This PR additively includes benchmarkRelativeMetrics in the existing Strategy V2 backtest result payload.
No /api/agent/v1 routes were changed, and there are no breaking API changes.

Screenshots (if UI change)

Not applicable — no UI changes.

Roadmap contribution (if applicable)

  • A maintainer confirmed the claimed scope before implementation
  • This PR implements one reviewable slice rather than an entire epic
  • User-facing behavior and compatibility notes are documented

The issue author confirmed that the metric-only scope is acceptable and invited a draft PR. Maintainer feedback on the concrete API and test coverage remains welcome.

@brokermr810

Copy link
Copy Markdown
Member

Thank you for the scoped implementation and test coverage. We reviewed the calculation path and found that computing from the forward-filled display benchmarkCurve can create artificial tracking error when a long intraday backtest fetches the benchmark at a coarser frequency.

An alternative implementation has now landed directly on main in commit 87cfeeb. It calculates at the benchmark's native frequency, samples portfolio equity at the same timestamps, uses the matching annualization factor, and filters invalid market-calendar gaps. The relevant Strategy V2 and metric suite passes with 237 tests.

Closing this draft unmerged because the replacement is already on main. Issue #179 remains open for CDI ingestion, the canonical CDI identifier, and the remaining Brazil-specific data/calendar design.

@ZhangWT02

Copy link
Copy Markdown
Author

Thank you for the review and for explaining the native-frequency issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants