[Klaud Cold] Add agentic spec-decode golden synthetic-AL rule: checklist item + verifier Check 10 / 新增 agentic 投机解码黄金合成 AL 规则:清单条目与验证器 Check 10#2160
Conversation
…ier Check 10 / 新增 agentic 投机解码黄金合成 AL 规则至审阅清单与验证器 Check 10 Agentic-workload spec-decode configs (EAGLE/MTP) must simulate acceptance at the committed golden AL from golden_al_distribution/ — SGLang via SGLANG_SIMULATE_ACC_* env vars, vLLM via synthetic rejection sampling in --speculative-config. New checklist template item (EN+ZH) and verifier Check 10 with golden-curve value matching. agentic 工作负载的投机解码配置(EAGLE/MTP)必须按 golden_al_distribution/ 中已提交的黄金 AL 模拟接受 —— SGLang 通过 SGLANG_SIMULATE_ACC_* 环境变量, vLLM 通过 --speculative-config 中的合成拒绝采样。新增清单模板条目(中英) 及验证器 Check 10(含黄金曲线取值核对)。 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
LGTM — policy/prompt-template change that adds the AgentX simulated-AL rule to the merge checklist and teaches the sign-off verifier a matching Check 10.
What was reviewed
- Confirmed
golden_al_distribution/exists with the referencedqwen3.5_mtp.yamlandkimik2.5_eagle3.yaml, so Check 10's "read the committed golden AL YAML" step has real files to hit. - Verified count/rowcount references stay consistent (
0-10, "eleven" rows) and that the English template is byte-identical betweenPR_REVIEW_CHECKLIST.mdand_zh.mdper the sync rule. - Checked the
_zhper-item explanation list was renumbered end-to-end (old 6-11 → new 7-12) with no dangling references. codeowner-signoff-verify.ymlchange is header-comment-only — no expression touched, so the 21k workflow-expression cap is not in play.
Extended reasoning...
Overview
This PR only touches four files: docs/PR_REVIEW_CHECKLIST.md, docs/PR_REVIEW_CHECKLIST_zh.md, .github/codeowner-signoff-verify-prompt.md, and .github/workflows/codeowner-signoff-verify.yml. It codifies a new fairness rule (agentic spec-decode configs must simulate acceptance at the committed golden AL) as one new checklist item, one new Check 10 in the verifier prompt, and one bullet in the workflow header comment. The English template block is intentionally kept verbatim across both language docs per the existing sync rule, and the _zh per-item explanation list is renumbered 6→7 … 11→12 with a new item 6 inserted.
Security risks
None. There is no code change: docs, a prompt template file, and a workflow YAML comment. The verifier workflow itself still checks out the trusted default branch (the security-critical bit is unchanged), and no permissions, secrets, or execution paths are modified. The prompt template is rendered via envsubst from the default-branch checkout, exactly as today.
Level of scrutiny
Low. This is a policy/documentation change — the runtime effect is only that a future sign-off verification run will additionally read golden_al_distribution/*.yaml and check for the SGLang/vLLM simulated-AL knobs on agentic spec-decode configs. The referenced YAMLs already exist, so there is nothing missing for the check to key off of. Worst case if the wording were wrong: the verifier occasionally flags a config that a human would agree is fine, which is a soft failure a reviewer can override.
Other factors
The description notes end-to-end verification of Check 10 requires a throwaway [DO NOT MERGE] PR post-merge (same pattern as #2156, since the verifier reads the prompt from the trusted default-branch checkout). That is an accepted limitation of this class of change, not a blocker. Numbering, cross-references, and the English/Chinese template invariant all check out.
Summary
Adds the AgentX fairness rule for speculative decoding on agentic workloads to the merge standard, and teaches the sign-off verifier to enforce it.
The rule: agentic-workload spec-decode configs (EAGLE / MTP / draft models) must run with simulated synthetic acceptance, pinned to the committed golden acceptance-length curve in
golden_al_distribution/for that model, thinking mode, andnum_speculative_tokens. Agentic replay does not reproduce real token-by-token traffic, so measured AL there is not representative; per the AgentX guidelines a submission may choose any supported draft length but may not substitute a different acceptance target.SGLANG_SIMULATE_ACC_LEN: <golden AL>+SGLANG_SIMULATE_ACC_METHOD: match-expected+SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token(as in the [NV] Add GB300 AgentX Qwen3.5 recipes #2121 GB300 Qwen3.5 AgentX recipes —3.39= qwen3.5 thinking_on MTP-3 golden value)--speculative-configwith"rejection_sample_method": "synthetic"and"synthetic_acceptance_length": <golden AL>(as in the [WIP] Kimi K2.6 agentic #2158 Kimi agentic script —2.88/3.24= kimik2.5 EAGLE3-3/4 golden values)Changes
docs/PR_REVIEW_CHECKLIST.md/_zh.md: new template item after the existing spec-decode chat-template item (template kept English verbatim in both, per the sync rule);_zhper-item explanation list extended and renumbered..github/codeowner-signoff-verify-prompt.md: new Check 10 — applicability (agentic paths × spec-decode flags), (a) simulated acceptance enabled, (b) AL value matches the committed golden curve, FAIL shapes (unsimulated agentic AL / golden mismatch / synthetic knobs on non-agentic configs), N/A line; check-count references updated (0-10, eleven rows).codeowner-signoff-verify.yml: header comment lists the new check (comment-only change; largest expression still ~450 chars, no 21k-cap risk).Validated locally: template renders cleanly through the workflow's
envsubststep (guard passes, 11## Checksections, zero leftover placeholders); workflow YAML parses with expression-length audit far under the cap.Like #2156, the verifier reads the prompt template from the trusted default-branch checkout, so this check can only be exercised end-to-end after merge — will be validated with a throwaway
[DO NOT MERGE]test PR per the AGENTS.md flow.中文说明
将 AgentX 关于 agentic 工作负载投机解码的公平性规则纳入合并标准,并让签署验证器强制执行。
规则:agentic 工作负载的投机解码配置(EAGLE / MTP / draft 模型)必须启用模拟合成接受,且 AL 取值固定为
golden_al_distribution/中该模型、thinking 模式与num_speculative_tokens对应的已提交黄金曲线值。agentic 回放无法复现真实逐 token 流量,实测 AL 不具代表性;提交可选择任意受支持的 draft 长度,但不得替换接受目标。SGLANG_SIMULATE_ACC_LEN+SGLANG_SIMULATE_ACC_METHOD: match-expected+SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token(同 [NV] Add GB300 AgentX Qwen3.5 recipes #2121)--speculative-config中"rejection_sample_method": "synthetic"与"synthetic_acceptance_length"(同 [WIP] Kimi K2.6 agentic #2158)变更:清单模板新增条目(中英文档中模板保持英文原文);验证器提示词新增 Check 10(含黄金曲线取值核对);workflow 头部注释同步(仅注释变更,无 21k 表达式上限风险)。本地已验证 envsubst 渲染与 YAML 解析。与 #2156 相同,端到端验证需在合并后通过临时
[DO NOT MERGE]测试 PR 完成。🤖 Generated with Claude Code