Skip to content

Add @ForegroundSafe/@BackgroundOnly declared-contract reachability check (J16) - #12474

Draft
dougqh wants to merge 13 commits into
robertomonteromiguel/dd-apm-sdk-review-java-overridesfrom
dougqh/foreground-background-only-check
Draft

Add @ForegroundSafe/@BackgroundOnly declared-contract reachability check (J16)#12474
dougqh wants to merge 13 commits into
robertomonteromiguel/dd-apm-sdk-review-java-overridesfrom
dougqh/foreground-background-only-check

Conversation

@dougqh

@dougqh dougqh commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds datadog.trace.api.function.ForegroundSafe/.BackgroundOnly annotations declaring a one-way foreground/background cost contract, applied to FixedSizeCache/EncodingCache as initial markers.
  • Adds addendum J16 (background-only-contract) to the dd-apm-sdk-review skill's performance lens: flags a foreground call path reaching @BackgroundOnly code, including a declaration-site inheritance-widening violation and a bounded two-hop interface-propagation extension.
  • Documents the contract in both annotations' Javadoc, the J16 addendum, and a new section in docs/instrumentation_design_guidelines.md.
  • Uses CLASS retention so a future bytecode-level checker (APMLP-1645) can consume the markers.
  • Adds .llm-validation fixtures covering the interface-hop and inheritance-widening cases (13 gate cases total, up from 11).

Part of APMLP-1513 / APMLP-1645.

Test plan

  • ./gradlew :internal-api:spotlessJavaCheck passes
  • .llm-validation gate run (13 cases) via the platform CI job
  • dd-apm-sdk-review 8-lens re-review of this branch: APPROVE_WITH_COMMENTS, no P0s (module-placement P1 explicitly deferred — annotations stay in internal-api for now, pending a future move of the whole toolbox into its own component)

🤖 Generated with Claude Code

dougqh and others added 12 commits September 1, 2026 16:09
Documentation-and-tooling markers declaring whether code is cheap
enough for application (foreground) threads or must be confined to a
background thread the tracer paces itself. No application to real
code yet and no checker -- just the annotation types, following the
Strategy/StrategyConsumer marker convention (APMLP-1543).
…ndSafe / @BackgroundOnly

DDCache is cheap (small, bounded probe count) -- safe to call from
application threads. SimpleUtf8Cache and GenerationalUtf8Cache trade CPU
for allocation savings that only pay off when run on a background
thread, so they're confined to the background serializer thread.

APMLP-1544

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ports the @ForegroundSafe/@BackgroundOnly declared-contract check
(originally drafted for the now-retired /perf-review skill) into the
dd-apm-sdk-review performance override, plus a matching
.llm-validation case exercising it against a synthetic Byte Buddy
advice call into a @BackgroundOnly cache.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The prior commit documented the constraint in a Javadoc @link but never
applied the annotation itself, so J16's grep-based resolution had
nothing to match.

APMLP-1544

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Add java-perf-foreground-calls-background-only-001 to the gate preset
  so it actually runs in CI instead of only under --level full.
- Add the missing core performance.md to its files: list.
- Fix the fixture: annotation is class-level not method-level, and the
  diff now points at the real HttpUrlConnectionAdvice location instead
  of a nonexistent module path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Flat SEV-1 skipped the shared SEV->P mapping in reviewers/performance.md,
so a reviewer who correctly applied the mapping would report P1 and fail
the test's own hardcoded "SEV-1/P0" criterion. J16 now escalates from
SEV-2/3 toward SEV-1/2 based on call-site frequency/cost, matching J13-J15,
and gets the Fix: clause the others already have. Test case graded
accordingly, plus a bad_signals entry for a future silent severity
downgrade.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add the Checker contract section (Trigger/Not-a-trigger/Violation/
Compliant/Out-of-scope) that @NoEscape already uses, so the rule is
machine-checkable straight from the annotation's own Javadoc without
needing J16's prose. Also note that @BackgroundOnly is orthogonal to
JSR-305 @threadsafe -- one governs concurrent-caller safety, the other
which category of caller is valid at all -- since both now land on the
same cache classes.

APMLP-1543

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Caching a caller-supplied HTTP header value pulled in an unrelated
data-retention question. The violation under test is purely which
thread reaches @BackgroundOnly code, so swap in a fixed,
non-sensitive instrumentation-internal string instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tory

Listing every currently-annotated class implied a complete, closed set;
reframe as a non-exhaustive sample and point at grep for the real answer.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ride

Covers two guard behaviors the existing violation-only case couldn't:
absence of either marker on the callee is not itself a finding, and a
method-level @ForegroundSafe correctly widens a @BackgroundOnly type for
that method. Both wired into the gate preset (9 -> 11 cases).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…16 test coverage

Adds the declaration-site inheritance-narrowing rule to both annotations'
Javadoc and to the performance lens's J16 addendum (now slugged
background-only-contract for stable cross-referencing), switches both
annotations from SOURCE to CLASS retention so a future bytecode checker
(APMLP-1645) can see them, adds llm-validation fixtures covering the
interface-hop and inheritance-widening cases, and documents the
foreground/background contract in instrumentation_design_guidelines.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@dougqh dougqh added tag: no release notes Changes to exclude from release notes inst: java Core Java language instrumentation tag: ai generated Largely based on code generated by an AI or LLM labels Sep 11, 2026
@dougqh

dougqh commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a6310c5078

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

* @param <K> key type
* @param <V> value type
*/
@ForegroundSafe

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Limit @ForegroundSafe to fully bounded methods

On a cache miss or eviction, computeIfAbsent calls the caller-supplied producer.apply through produceAndStoreValue, so its cost can be blocking or otherwise background-only. This type-level annotation also covers clear() and visit(), which traverse the entire backing array, with visit() additionally invoking an arbitrary consumer. For a slow producer or large cache, the declared “every method is foreground-safe” contract is therefore false and can cause reviewers or the planned checker to accept unbounded work on an application thread; scope the marker to operations whose complete transitive work is bounded or define callback-aware semantics instead.

Useful? React with 👍 / 👎.

1. **Concrete → declared interface, method-scoped.** When a foreground call site is typed to an interface `I` rather than the concrete class, and some implementation `C` of `I` — already in hand, i.e. visible in the diff or files handed to you, never found by searching the repo for implementers — is `@BackgroundOnly` (type- or method-level), read `C`'s `implements`/`extends` clause and match `C`'s `@Override` methods against `I`'s method signatures. Propagate the marker only to *that signature* on `I` — not to every method `I` declares. A foreground call to a different, unrelated method on `I` is not a finding.
2. **Default method transitively calling a flagged signature.** If `I` declares a default method `d()` whose body calls one of the signatures flagged in step 1, `d()` inherits the flag too. This is a single read of `I`'s own source — do not chase the call further than one interface file.

Neither hop proves the call site's interface reference actually resolves to `C` at runtime (that's still unprovable by grep) — treat a hit from either hop as flag-with-confidence the same as the direct case, since a call site that would only ever reach a *safe* implementer of `I` is the rare case, not the default one to assume.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Require evidence that the interface receiver can be background-only

When an interface has both foreground-safe and background-only implementations in the reviewed files, this instruction requires a confident finding merely because a background-only implementation is “in hand,” even if the call-site variable is explicitly initialized with the safe implementation. The text itself admits that neither hop proves the receiver resolves to C, contradicting the later requirement that only a provable boundary crossing count and creating deterministic false-positive reviews. Track the receiver's creation or assignment to the call site and propagate the marker only when C can actually back that reference.

Useful? React with 👍 / 👎.

@datadog-datadog-prod-us1

This comment has been minimized.

…lse-positive

FixedSizeCache's @ForegroundSafe Javadoc overclaimed: computeIfAbsent's
producer and visit()'s consumer are caller-supplied and can be
arbitrarily expensive, so the annotation now scopes its guarantee to
the cache's own bookkeeping only.

J16's interface-propagation hop-1 flagged any interface-typed call
site reachable to a @BackgroundOnly implementer C, even when the diff
visibly binds the receiver to a different, safe implementation --
guaranteed unreachable. Hop-1 now checks for a visible binding first
and only falls back to flag-with-confidence when the receiver's
concrete type isn't resolvable from the diff.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.79 s 14.62 s [+0.1%; +2.2%] (maybe worse)
startup:insecure-bank:tracing:Agent 13.54 s 13.64 s [-1.5%; +0.0%] (no difference)
startup:petclinic:appsec:Agent 17.53 s 17.42 s [-0.3%; +1.6%] (no difference)
startup:petclinic:iast:Agent 17.53 s 17.14 s [-2.1%; +6.7%] (no difference)
startup:petclinic:profiling:Agent 17.54 s 17.08 s [+1.4%; +3.9%] (significantly worse)
startup:petclinic:sca:Agent 17.66 s 17.44 s [+0.3%; +2.1%] (maybe worse)
startup:petclinic:tracing:Agent 16.18 s 16.62 s [-6.8%; +1.5%] (no difference)

Commit: 8163b897 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@dd-octo-sts

dd-octo-sts Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.81 s 14.73 s [-0.5%; +1.6%] (no difference)
startup:insecure-bank:tracing:Agent 13.63 s 13.69 s [-1.3%; +0.4%] (no difference)

Commit: a6310c50 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@pr-commenter

pr-commenter Bot commented Sep 11, 2026

Copy link
Copy Markdown

LLM Validation

LLM Validation Gate — dd-apm-sdk-review

✅ PASS

  • No blocking-case regressions; the quality change is within noise (baseline/candidate confidence intervals overlap).
  • Advisory: the candidate introduced a bad signal on 1 case(s) that the baseline did not — surfaced for review (see Cases); it does not block on its own.
  • Advisory: 1 run(s) could not be evaluated (agent infra error) and were excluded from scoring — re-run if a case shows too few evaluated runs to trust.

Analysis

Changed instruction file(s): .agents/skills/dd-apm-sdk-review/SKILL.md, .agents/skills/dd-apm-sdk-review/reviewers/_common.md, .agents/skills/dd-apm-sdk-review/reviewers/correctness.md, .agents/skills/dd-apm-sdk-review/reviewers/performance.md, .agents/skills/dd-apm-sdk-review/reviewers/report-template.md, .agents/skills/dd-apm-sdk-review/review-without-harness.md, .agents/skills/dd-apm-sdk-review/reviewers/coherence.md, .agents/skills/dd-apm-sdk-review/reviewers/security.md, .agents/skills/dd-apm-sdk-review/reviewers/design.md, .agents/skills/dd-apm-sdk-review/reviewers/maintainability.md, .agents/skills/dd-apm-sdk-review/reviewers/conventions.md, .agents/skills/dd-apm-sdk-review/reviewers/cross-sdk.md, .agents/dd-apm-sdk-review-overrides/repo-context.md, .agents/dd-apm-sdk-review-overrides/reviewers/security.md, .agents/dd-apm-sdk-review-overrides/reviewers/performance.md, .agents/dd-apm-sdk-review-overrides/reviewers/design.md, .agents/dd-apm-sdk-review-overrides/reviewers/conventions.md, .agents/dd-apm-sdk-review-overrides/reviewers/maintainability.md.

1 case(s) dipped but stayed within the pass bar.

  • java-correctness-mapper-state-leak: won 25% of blind comparisons, quality -15.0. ⚠️ Introduced a new bad signal (advisory).
    • Asked: Run dd-apm-sdk-review on the following change. There is no git repository to inspect — this diff is the entire change set; do not attempt any git commands, go
    • Baseline answer: I now have everything needed. I confirmed the full lifecycle and traced the flag's behavior through every path. Note: the skill's SKILL.md and reviewer files are deleted from this worktree (only a mirror README.md survives) and the skill isn't wired into the Skill tool, so I ran the review directly using the
    • Candidate answer: I have everything I need. Since the change was pasted inline, this ran in single-context sequential mode (expected for pasted input, not a capability gap). Grounding checks done: reset() is called per payload flush (PayloadDispatcherImpl:110), the firstSpanWritten pattern matches the sibling TraceMapperV0_5,

Results

  • Pairwise win-rate: 65% [58%–72%] — candidate's share of blind comparisons (90% CI; spanning 50% = no clear difference)
  • Overall quality: 78.5 → 83.5 (/100, +5.1)
  • Bad signals introduced (advisory): 1
  • Candidate criteria coverage (advisory): 39/43 (91%) — expected_criteria the candidate met; does not affect the gate
  • Candidate bad signals (all, advisory): 2 case(s) — trips on the candidate, including ones the baseline also hit
  • Blocking-case regressions: 0
  • Inconclusive runs (excluded — infra error): 1

Cases

Case Mode Quality Δ Win-rate (90% CI) Safety
java-perf-lens-wrong-collection-001 block -2.7 38% [17%–58%] ok
java-perf-pipeline-full-review-002 block +35.0 100% [100%–100%] ok
java-security-crash-handler-before-trust block +0.4 50% [50%–50%] ok
java-correctness-capture-before-send block +0.8 62% [23%–100%] ok
java-correctness-sqs-queue-name-incomplete block +7.1 75% [51%–99%] ok
java-maintainability-resource-leak-streams block +13.3 88% [67%–100%] ok
java-correctness-span-events-list-only block +12.1 100% [100%–100%] ok
java-correctness-mapper-state-leak block -15.0 25% [0%–66%] (1 inconc.) ⚠️ bad signal
java-perf-foreground-calls-background-only-001 block +2.5 62% [42%–83%] ok
java-perf-foreground-calls-unannotated-002 block -5.8 50% [50%–50%] fail (both)
java-perf-foreground-calls-method-override-003 block +2.1 50% [50%–50%] ok
java-perf-foreground-calls-interface-hop-004 block 0.0 50% [50%–50%] ok
java-perf-inheritance-widen-violation-005 block +5.8 75% [51%–99%] ok

Per-dimension scores, token usage, latency, and estimated cost are in the CI job logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inst: java Core Java language instrumentation tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant