docs(runbooks): add EVM gas-vs-execution-time benchmarking runbook

[bdchatham]

Summary

Adds .agent/runbooks/benchmarking-evm-gas-vs-execution-time.md for the gas-repricing project: a self-contained procedure covering both target functionalities —

1. Replay a real historical block X against restored state after block Y, measuring gas + execution time per tx.
2. Execute arbitrary bytecode against state after block Y, measuring the same.

Revision history on this PR: the first version (this PR's initial commit) proposed a from-scratch synthetic single-validator + RPC-follower genesis chain with fabricated state. After more context surfaced on what the actual environment needs to support, this was reworked entirely around real infrastructure:

• Functionality 1 uses a replayer-mode SeiNode (spec.replayer.snapshot.s3.targetHeight) — the same real, documented state-restore-and-forward-replay mechanism behind validating-flatkv-memiavl-parity-via-sharded-replay.md — instead of synthesizing state from genesis. Verified gap: the existing replay/comparator tooling captures gas but has no timing field anywhere in any of its three comparison layers, which is the actual eBPF opportunity.
• Functionality 2 gives two paths: a preferred read-only historical eth_call + state-override against an archive node (flagged as an unverified RPC capability — confirm before relying on it, since it isn't documented in operating-archive-node-byov.md or anywhere else in this repo), falling back to the original differential-loop bytecode technique against a private writable fork of real state-at-Y (rather than an empty synthetic chain).
• Hardware parity with prod validators is confirmed automatic, traced into the controller's own source: internal/platform/platform.go's NodepoolForMode() plus identical nodepoolName/nodepoolArchive/tolerationKey config and Karpenter NodePool definitions (r6i/r7i, Nitro) between harbor and prod. Zero engineer-side configuration needed — just verify it (§2).

Peer-reviewed collaboratively with the systems-engineer agent, which traced Sei's OCC scheduler in sei-cosmos (app/abci.go, tasks/scheduler.go) and found that naive per-tx timing correlation breaks down inside a replayed multi-tx block under OCC's parallel/re-execution scheduling. Fix: disable OCC (occ-enabled = false, concurrency-workers = 1) on the replayer for the measurement run, which serializes execution without corrupting per-tx gas or compute (new §3a) — flagged VERIFY pending confirmation that the SeiNode/seictl spec actually exposes those app.toml keys. Also caught: Path A can't get eBPF-grade timing off the shared archive node (not yours to instrument — the §6 privileged-attach gate applies), and that block-window detection during replay must be continuous/post-hoc rather than reactive, since a block's application window (~25ms at replay speed) commits before RPC/logs can report it.

Retains the original PR's still-valid content: the governance-mutable SSTORE gas trap, the differential-loop bytecode-crafting technique, the privileged eBPF-attach gate (verified against the platform repo: eng-<alias> namespaces enforce no Pod Security Standard today — an unhardened gap, not a green light), and symbol resolution against the real seid/go-ethereum fork (go.mod's replace directive preserves the upstream import path in compiled symbols).

Test plan

• Reviewer confirms the runbook is self-contained per the README's own "Adding a new runbook" convention
• Reviewer spot-checks the cited file paths (x/evm/types/params.go, internal/platform/platform.go, the platform PSS/NodePool/controller-config files) still say what's claimed
• Someone confirms the occ-enabled/concurrency-workers app.toml keys are actually exposed on the SeiNode/seictl render path (flagged VERIFY in §3a) before the team depends on them
• Someone confirms empirically whether Sei's archive-node RPC actually supports a state-overridden historical eth_call (flagged VERIFY in §4, Path A) before building a campaign on that assumption
• Index entry in README.md is accurate

🤖 Generated with Claude Code

[cursor]

PR Summary

Low Risk
Documentation-only addition to .agent/runbooks/ with no runtime, auth, or application code changes.

Overview
Adds a new agent runbook benchmarking-evm-gas-vs-execution-time.md for testing whether Sei’s EVM gas schedule tracks real compute on harbor, without changing sei-chain.

The runbook documents two measurement paths: historical block replay (replayer at height <Y>, time block <X>) and arbitrary bytecode against historical state (eth_call with state override vs. a private writable fork). It calls out failure modes that invalidate naive benchmarks—governance-tuned SSTORE gas, OCC during replay, unverified historical eth_call overrides, and the lack of per-opcode eBPF symbols—and prescribes mitigations (differential-loop bytecode, disabling OCC for per-tx correlation, hardware parity checks via NodepoolForMode).

It also adds a harbor eBPF privileged-attach gate (sign-off, scoped kubectl debug --target=seid) and a tiered instrumentation path (gasUsed/pprof → offcputime/profile on StateTransition.Execute), plus failure-mode and pre-flight tables.

.agent/runbooks/README.md gains an index row so agents can discover when to load this procedure.

^{Reviewed by Cursor Bugbot for commit 6140dd3. Bugbot is set up for automated code reviews on this repo. Configure here.}