test(app): add smoke test

[iamquang95]

Add docker-compose smoke tests (charon parity + pluto/charon node mixing)

Ports charon v1.7.1's testutil/compose smoke-test framework to test-infra/compose, adding a per-node implementation axis so clusters can mix charon and pluto nodes. Each scenario builds a real cluster (keygen → run with simnet mocks → 60s under Prometheus alert gating); any firing alert fails the test.

Scenarios cover charon's full matrix plus pluto variants (all_pluto, mixed_2_charon_2_pluto, pluto_dkg, gated behind -pluto-run). all_pluto passes against feat/simnet-run: 4 nodes reach QBFT consensus, threshold-sign, and broadcast duties with zero errors/warnings.

Deviations from charon (each commented):

• Alert gate matches firing, not active — charon's never fires (upstream no-op bug)
• Prometheus scrape config generated per cluster size (charon scrapes a non-existent bootnode)
• Pluto keygen gets flags, not CHARON_* env vars (pluto CLI gap)
• Path-less relay URL, dkg --shutdown-delay (distroless image), loki/otlp only when monitoring enabled, dropped CI-only sudo chown

Run: PLUTO_REPO= go test ./smoke -v -integration [-pluto-run]

[iamquang95]

@claude review this PR

[claude]

Claude finished @iamquang95's task in 4m 29s —— View job

---

Claude PR Review — test(app): add smoke test

I reviewed the full diff (8ddf273..HEAD), reading the ported Go framework against charon v1.7.1's testutil/compose at /home/runner/work/pluto/pluto/charon/testutil/compose. Overall this is a high-quality, unusually well-documented port — every deviation from charon is justified with a comment tying it to a concrete pluto/charon behavior, and the pure-Go logic (alert-rule generation, warmup gating, impl selection) is well covered by internal tests. I have no blocking findings; the notes below are observations and a couple of small gaps.

What's solid

• getActiveAlerts firing-vs-active fix (alert.go:163-185) is the most important correctness change over charon and is exactly right — charon matches "active", a state Prometheus never emits, so its gate silently passes everything. Pinned by TestGetActiveAlertsFiringOnly.
• Warmup gating (alert.go:76-119) correctly still fails if a cold-start transient is still firing after the window (inWarmup && startupTransientRules[...] → continue, then re-checked once inWarmup is false). TestStartupTransientRulesScoped locks the allowlist to the 3 windowed rules.
• writeAlertRules broadcast-liveness expression (define.go:545-550) — the or on (job) max by (job) (0 * up) injection so absent series (a node that never broadcasts) still alerts is a genuinely subtle fix and is well tested.
• Config.Validate running on every write and load (config.go:188, exercised by TestConfigValidateRejectsUnknownImpl) closes the "typo silently selects charon image" hole nicely.
• Golden coverage maps 1:1 to the 8 TestDockerCompose subtests + TestNewDefaultConfig.

Observations / minor

1. No CI wiring — smoke tests are manual-only. No workflow under .github/ references test-infra/compose or smoke, and the module is a separate go.mod outside the Rust workspace, so cargo test won't touch it. Given it needs Docker + a source build of both repos, manual-only is reasonable, but please confirm that's intended and consider a nightly/opt-in workflow so the framework doesn't silently rot. (I could not execute go test in this review environment to confirm the unit tests pass — CI/local should verify.)

2. flagsCommand comma-quoting is only exercised in gated integration tests. The single-quote-on-comma logic (config.go:286-288) matters most for create dkg's --operator-enrs=enr:...,enr:... and withdrawal/fee-recipient addresses, but there's no golden or unit test for the pluto keygen DKG define path — the TestDockerCompose matrix covers lock create pluto keygen but not a pluto define dkg. A small table-test on flagsCommand (comma value → single-quoted; false/empty → omitted; true → bare flag) would guard this cheaply and without Docker. Fix this →

3. Pluto keygen container gets both flags and CHARON_* env vars (see testdata/TestDockerCompose_lock_create_pluto_keygen_yml.golden). Harmless since pluto ignores the env vars and reads the flags, but the redundancy is slightly confusing — worth a one-line comment in Lock/Define noting the env vars are dead weight for the pluto keygen path (kept only because they share the kvs used to build the flags).

4. Alert reader only drains after the full AlertTimeout. In Auto (auto.go:131-146), execUp blocks until the context deadline, so the for alert := range alerts loop runs after the timeout — a real alert firing at t≈20s still takes the full 2 min to fail the scenario. This is inherited from charon and fine for a smoke suite, just flagging the wall-clock cost (12 non-pluto scenarios × up to 3 min).

5. Alert channel (buffer 100) has no concurrent reader during the run (alert.go:56), and non-success status messages (alert.go:93) are not deduped unlike real alerts. Bounded well under 100 for a 2-min run and inherited from charon, so low risk — noting only for completeness.

None of these are blockers. Nice work — the deviation comments in particular make this easy to audit against upstream.
· branch feat/smoke-test