Parse observeFullEvaluationData and hash targeting_key in flagevaluations events

[vjfridge]

Background

Feature-flag evaluations are reported to Datadog so users can see how their flags behaved in production. Today, every evaluation carries the raw targeting key (the identifier of the subject being evaluated; often a user email or user ID) and the full evaluation context. Those fields can contain personally identifiable information (PII).

Motivation & High level changes

This PR lets the server decide, per environment, whether that raw data may be observed via a new boolean on the UFC. It is the Java piece of the cross-SDK "Protecting PII in flagevaluations" initiative. dd-trace-java is the pilot SDK; the same pattern will be copied to the other SDKs afterward.

This directly resolves the privacy concern raised on the parent PR #11639, where @AlexeyKuznetsov-DD flagged that flag-evaluation reporting was default-on and shipped the raw targeting_key (user IDs / emails) plus the full evaluation context in clear text, and asked for hashing / opt-in / an explicit privacy sign-off. The no-PII path is now the default.

Server says observeFullEvaluationData	targeting key	evaluation context
true	sent as-is (raw)	included (raw)
false or absent (privacy-preserving default)	SHA-256 hashed, sha256_<hex>	omitted entirely

"Absent" is treated exactly like false, so an older/cached config can never accidentally leak raw values.

How it works

• New field observeFullEvaluationData on the downloaded flag-configuration (UFC) model (top-level boolean, defaults to false when missing).
• Consent is captured at evaluation time, on the evaluation thread, and carried on the event. The OpenFeature hook (FlagEvalLoggingHook) snapshots the gateway's consent value (FeatureFlaggingGateway.isObserveFullEvaluationDataEnabled()) at the moment the flag is evaluated and stores it as FlagEvalEvent.observeFullEvaluationData. Aggregation and serialization then read that per-event snapshot and never consult the gateway — so the hashed-vs-raw decision is pinned to the configuration active when the evaluation happened.
  • Why not read it at drain/flush? CURRENT_CONFIG can be overwritten by a later Remote Config update between when an evaluation happened and when the batch is drained and flushed. Reading the gateway at flush would retroactively apply a different environment's consent to already-collected evaluations. (This was a real bug caught by system-tests: a targeting key came back hashed even though the active UFC said true.) Snapshotting on the evaluation thread closes that window in both directions.
  • Conservative fold: when multiple evaluations merge into one bucket, the consent flag is folded with AND across the events' snapshots. If any evaluation in the bucket's lifetime saw consent off, the whole bucket falls back to hashed/omitted. Fail-closed toward privacy. The bucket key is unchanged (no split on consent).
• When hashing, the targeting key is run through the existing shared SHA-256 helper and prefixed with sha256_; the context field is dropped so it's absent from the JSON (not null, not {}).
• Evaluation logic is unchanged.

SHA-256 hash

The hash is unsalted SHA-256 over the raw UTF-8 bytes (no trimming/case/Unicode normalization), emitted as lowercase hex. Unsalted is a deliberate cross-SDK contract requirement: every SDK must produce the identical digest for a given targeting key so hashed values line up across languages and against the backend. It matches the canonical test vector shared across all SDKs:

"jane.doe@datadoghq.com" -> sha256_b4698f9b6d186781fa8dc59e533578fa2d8379a46b1cf6db85cda6aa9c99e51b

Performance

Consent is read once per evaluation on the hook path — a single AtomicReference.get snapshotted onto the event (no allocation, no per-span hot path) — and folded with a boolean AND on bucket merge. Hashing runs at flush cadence, bounded by the number of distinct evaluation buckets, and uses a ThreadLocal<MessageDigest> (no per-call MessageDigest.getInstance).

Testing

• Hash helper matches the canonical PII vector; preserves whitespace/case; is lowercase 64-char hex.
• Config parsing covers true, false (as a @ValueSource matrix), absent (defaults to false), and explicit null (rejected / fail-closed).
• Serialization covers both raw and hashed paths, asserting on the raw wire bytes that the hashed value is present, the raw PII value never appears, and no per-event context is emitted.
• Consent-capture at evaluation time: hook tests assert the event snapshots the gateway's consent value (true, false, and absent → false) at enqueue time; FlagEvalEvent tests cover the new field and the fail-closed default on the convenience constructors.
• Consent-timing regression guards (both directions): a bucket whose event was snapshotted with consent off stays hashed even if a later RC update turns the gateway on before drain/flush; and symmetrically, an event snapshotted with consent on stays raw even if the gateway is later turned off — proving neither aggregation nor flush consults the gateway. Aggregator fold tests cover mixed-consent buckets (any off → hashed) and all-on buckets (stays raw).

🤖 Generated with Claude Code

[vjfridge]

system-tests validation (L3) — Java SDK ✅

Ran the three new observeFullEvaluationData PII-protection tests from system-tests#7316 locally against this branch (java@1.65.0-SNAPSHOT+81af153ca7).

All three pass:

Test	Result
Test_FFE_EVP_Flagevaluation_ObserveFullData_Absent_Hashed	✅ PASS
Test_FFE_EVP_Flagevaluation_ObserveFullData_False_Hashed	✅ PASS
Test_FFE_EVP_Flagevaluation_ObserveFullData_True_Unhashed	✅ PASS

36 passed, 8 skipped, 2704 deselected, 1 xfailed, 3 xpassed in 301.86s

Weblog: spring-boot | Scenario: FEATURE_FLAGGING_AND_EXPERIMENTATION

Note: an earlier build (d65c79f266) of this PR failed True_Unhashed — the SDK was always hashing regardless of the flag value. Root cause: FlagEvaluationWriterImpl.buildEventList() read observeFullEvaluationData from CURRENT_CONFIG at flush time, which could be overwritten by a subsequent RC update before the 10s flush fired. The fix in this PR (capturing the value per-bucket at enqueue time with a privacy-preserving fold) resolves it.

[vjfridge]

ffe-dogfooding validation (L2) - Java SDK ✅

I ran this branch's dd-java-agent + dd-openfeature build through our OpenFeature dogfooding harness against a staging UFC, exercising all three behaviors of the PII-protection contract:

Protected — observeFullEvaluationData = false (default):

• targeting_key = sha256_ + 64-char lowercase-hex SHA-256 digest (71 chars total)
• context.evaluation omitted
• consistent across every emitted flagevaluation event

Full — observeFullEvaluationData = true (opt-in):

• targeting_key raw (verbatim subject id)
• context.evaluation present with the full evaluation context

Kill switch — DD_FLAGGING_EVALUATION_COUNTS_ENABLED = false:

• zero flagevaluation events emitted regardless of observeFullEvaluationData (SDK logs Flag evaluation EVP writer disabled)

Notes:

• Pointing the app at environments with observeFullEvaluationData false vs true produced exactly the expected payload shapes.
• Hashing was applied even to FLAG_NOT_FOUND (runtime-default) evaluations — the privacy posture is enforced independent of evaluation outcome. 👍
• The SDK posts to the singular /api/v2/flagevaluation EVP route, as expected for the flagevaluation track.

Validation harness changes: ffe-dogfooding#100 — adds the per-SDK PII posture dashboard badge and fixes the singular /api/v2/flagevaluation intake route.

Verdict: the Java SDK is performing as expected for the PII-protection contract — default hashing, opt-in full data, and kill switch all behave correctly. Cross-SDK hash equality is covered in the companion system-tests work.

[datadog-prod-us1-5]

More details

The event-time snapshot, conservative bucket fold, and serialization gates preserve the no-PII default while retaining raw data only for explicitly consented evaluations. No additional tests recommended: production telemetry is unavailable and the PR already covers the relevant consent transitions and wire shape.

Was this helpful? React 👍 or 👎

_{📊 Validated against 3 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit e0e3632 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e0e3632b8c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Bind consent to the evaluated configuration

When Remote Config changes from observeFullEvaluationData=false to true while an evaluation is in progress, DDEvaluator.evaluate can finish using the old configuration while this hook reads the new global value. FeatureFlaggingGateway.dispatch updates CURRENT_CONFIG before notifying the evaluator, and the evaluator retains its configuration in a local variable, so this race can mark an evaluation performed without consent as consented and emit its raw targeting key and context. Carry the consent from the exact ServerConfiguration used by the evaluator, for example through evaluation metadata, instead of rereading the gateway here.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Stop keying redacted buckets by hidden context

perf: When observeFullEvaluationData is false—the new default—a changing context field such as a request ID still contributes to ctxKey and creates a distinct full-tier bucket, even though serialization omits that context. At sufficient traffic this emits many wire-identical rows, exhausts the 10,000-bucket per-flag cap within a flush interval, and forces subsequent evaluations into the degraded tier without targeting information. No-consent evaluations should aggregate without the context dimension that will be discarded.

AGENTS.md reference: AGENTS.md:L77-L77

Useful? React with 👍 / 👎.