fix: pool speculative-decoding metrics instead of averaging per-sample ratios

[keepkeen]

What

The speculative-decoding dashboard metrics (rollout/spec_accept_rate, rollout/spec_accept_length) are computed as a mean of per-sample ratios instead of the pooled batch statistic, and are biased — only ever downward.

Root cause

metrics["spec_accept_rate"]   = sum(sample.spec_info.spec_accept_rate   for sample in all_samples) / num_samples
metrics["spec_accept_length"] = sum(sample.spec_info.spec_accept_length for sample in all_samples) / num_samples

spec_accept_rate is itself accept/draft per sample and spec_accept_length is completion/verify_ct. Averaging those ratios:

1. weights a 10-token response the same as a 4,000-token one;
2. counts samples whose counters were never populated as hard 0.0s — the properties return 0.0 on a zero denominator, and SpecInfo.add only runs when terminal meta info arrives, so aborted / partial-rollout samples legitimately carry zero counters.

Both effects deflate the number. On a batch of 7 short samples + 1 long one, the reported accept length is −42% off the pooled truth and the accept rate −74%; with a quarter of the batch unscored, −25%. Anyone tuning --sglang-speculative-* off these dashboards is optimizing against a deflated signal.

Why pooling is the intended semantic

• Sample.SpecInfo deliberately accumulates the four raw counters across partial-rollout turns — the comment says "cannot directly use spec info from sglang because of partial rollout" — i.e. the raw numbers are kept precisely so a correct ratio can be formed at report time. The old code threw them away and used the per-sample properties instead.
• The sibling _compute_prefix_cache_metrics, five lines below, already pools correctly (total_cached_tokens / total_prompt_tokens).

Fix

Sum the four counters over the batch and report Σaccept/Σdraft and Σcompletion/Σverify. When a batch has no counters at all, the keys are omitted (matching _compute_top_p_kept_vocab_metrics) rather than reporting a fake 0.0 — which also removes the latent ZeroDivisionError on an empty sample list.

Test

New tests/test_spec_metrics.py, registered in the cpu-unittest job: pooled math on a skewed batch, zero-counter samples not diluting, no-counter batches reporting nothing, and the sglang_speculative_algorithm=None gate. Three of four cases fail on main.

slime/ray/rollout.py imports sglang / sglang_router / wandb at module level; the test stubs those imports the same way tests/test_agent/test_agent_rollout_cpu.py stubs transformers, which is also what makes this the first of the slime/ray/rollout.py metric helpers exercisable in CPU CI.