Skip to content

Declare each rule's parameter space, so the trials budget is derived rather than remembered #528

Description

@eaitbrahim

From the 2026-08-23 Jesse-comparison PRD (docs/superpowers/specs/2026-08-23-strategy-api-expressiveness-prd.md §4.1). The one genuinely new finding from that comparison — everything else it surfaced was already filed.

The gap

n_trials — the input to keel's own overfitting correction — is currently a number a human remembers to record.

keel already has the machinery that consumes it:

  • research/deflate.py::expected_max_sharpe(n_trials) — the Sharpe expected from the luckiest of N zero-skill trials.
  • research/independence.py"two rules that fire together are one rule counted twice, consuming trials budget twice".
  • A hand-maintained ledger (keel/research/ledger.py) recorded through the CLI with explicit DECISIONS/PROVENANCE, plus source comments reasoning per-choice about whether something "increments the trials budget (§73.12)" (analysis/indicators.py:388, analysis/levels.py:114).

So the statistical correction is rigorous and the number feeding it is manual. That asymmetry is the bug.

What Jesse does

A strategy declares its own search space:

def hyperparameters(self):
    return [
        {'name': 'stop_loss_atr_rate', 'type': float, 'min': 0.1, 'max': 2.0, 'default': 2},
        {'name': 'up_length',          'type': int,   'min': 3,   'max': 30,  'default': 21},
        {'name': 'up_coeff',           'type': float, 'min': 0.1, 'max': 3.0, 'default': 0.71},
    ]

Jesse uses it to drive an optimiser. That is not why it is interesting here. It is interesting because the size of the space a sweep could explore becomes a property of the rule rather than of the operator's diligence — which makes the trials count derivable.

Proposal

Add a declared parameter space to Rule — most naturally an extension of the existing describe(), which already returns per-parameter help text, so the names and their documentation live together rather than in two places that can disagree.

Each entry carries at minimum: name, type, and the range considered legitimate for that parameter.

Then:

  • The trials arithmetic reads the declaration instead of a hand-entered number.
  • A sweep script reads the declaration rather than restating it, so an explored range cannot silently exceed the declared one.
  • keel rules describe-style output can show what a parameter is allowed to be, not just what it currently is.

Explicit non-goal — do not build an optimiser

Declaring the space is cheap and improves a correction that already exists. Running an optimiser against a cost-bound engine would manufacture exactly the overfitting the deflated-Sharpe machinery exists to detect — and the 144-cell sweep already demonstrated the shape of that: 8 winning cells out of 864 trials, all in one asset, in a liquidity surge.

The declaration must not become a licence to search it.

Acceptance criteria

  • Every registered rule kind declares its parameter space, with ranges that reflect what has actually been considered legitimate.
  • n_trials for a rule family is derivable from the rule; no hand-entered count is required for the standard case.
  • A sweep reads the declared space rather than restating ranges in a script.
  • The declaration is covered by a test asserting it stays in sync with the constructor's real parameters — a declaration that drifts from the signature under-counts trials, which is worse than none.
  • The existing ledger's DECISIONS/PROVENANCE recording is unchanged; this feeds it, it does not replace it.
  • No optimiser ships as part of this.

Notes

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions