fix(ffe): make exposure event test deterministic - #7465
Conversation
|
|
cb38a8a to
7c845b7
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7c845b7d20
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| assert result["value"] == "on", f"Expected 'on', got {result['value']!r}" | ||
|
|
||
| # Exposure delivery is asynchronous; wait before the next setup replaces Remote Config. | ||
| wait_for_exposure_event({self.flag}, self.targeting_key) |
There was a problem hiding this comment.
Keep exposure assertions out of setup
When this file is xfailed by a manifest for unsupported tracer versions (for example the file-level version gates in manifests/dotnet.yml, manifests/golang.yml, and manifests/python.yml), this assertion now runs inside the repository's custom setup loop in pytest_collection_finish before pytest executes the xfailed test. A timeout here is re-raised as a setup/collection failure instead of being reported as an expected test failure, so running those declared-unsupported versions can abort the scenario. Keep the wait/assertion in the test body, or make the setup path avoid failing for xfailed items.
Useful? React with 👍 / 👎.
🎉 All green!🧪 All tests passed 🔗 Commit SHA: aa82f0d | Docs | Datadog PR Page | Give us feedback! |
7e14cfa
into
main
Motivation
The FFE exposure-generation test intermittently timed out while waiting for an exposure. It used the generic
test-flagkey shared with other FFE coverage and did not confirm that the evaluation returned the configured variation. Although the original timeout was not conclusively traced to a flag-key collision, isolating the fixture removes that possible source of cross-test interference and makes evaluation failures distinguishable from exposure-delivery failures.Changes
onvariation before checking its exposure.Validation
django-py3.13CI job; that job failed on an unrelated FFE evaluation-metric test.git diff --checkpassed../format.shproceeded through the Python, YAML, and parser checks, then encountered an unrelated macOS Bash 3set -ufailure inutils/scripts/shellcheck.shwhen expanding an empty$@.