feat(dogstatsd)!: add shared_runtime buffered sink - #2224
Conversation
This stack of pull requests is managed by Graphite. Learn more about stacking. |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 955e832 | Docs | Datadog PR Page | Give us feedback! |
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
8eb7bc6 to
fed7da4
Compare
597364e to
b905f96
Compare
fed7da4 to
7bafd9b
Compare
b905f96 to
30d94f8
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7bafd9bcfa
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
BenchmarksComparisonBenchmark execution time: 2026-07-22 13:08:29 Comparing candidate commit 955e832 in PR branch Found 0 performance improvements and 2 performance regressions! Performance is the same for 140 metrics, 0 unstable metrics.
|
7bafd9b to
8f5b1cf
Compare
0e35cfd to
59810b1
Compare
30d94f8 to
395fb45
Compare
b3d9cd0 to
e981309
Compare
e981309 to
7709edc
Compare
395fb45 to
0c91d2d
Compare
7709edc to
9aa9b85
Compare
# What does this PR do? Refactor of the dogstatsd client crate. Main changes are: - Separated code in 3 modules, `action` `client` and `client/sink` - Renamed `Client` to `DogStatDClient` to avoid renames when importing in other crates - Move `new`, `create_client` and `do_send` to be method of the `DogStatsDClient` - Replaced Mutex in `DogStatsDClient` to use a `OnceLock` and make the client handle clonable # Motivation I've encountered several issues with the current design when sharing the client with the StatsExporter in #2159 and when adding the SharedRuntime backed client #2224 . # Additional Notes Anything else we should know when reviewing? # How to test the change? Describe here in detail how the change can be validated. Co-authored-by: vianney.ruhlmann <vianney.ruhlmann@datadoghq.com>
9aa9b85 to
955e832
Compare
) # Release proposal for libdd-data-pipeline, libdd-telemetry and their dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-ddsketch **Next version:** `1.1.1` **Semver bump:** `patch` **Tag:** `libdd-ddsketch-v1.1.1` ### Commits - refactor: migrate to workspace dependencies, phase 4 (#2296) - chore: moving to workspace-level dependencies, phase 2 (#2270) ## libdd-shared-runtime **Next version:** `3.0.0` **Semver bump:** `major` **Tag:** `libdd-shared-runtime-v3.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^2.1.0 → ^3.0.0 - `libdd-capabilities-impl`: ^3.0.0 → ^4.0.0 ### Commits - feat(shared-runtime): add block_on_with_timeout to BlockingRuntime (#2333) - chore(runtime): restart all workers on invalid state (#2262) - refactor: migrate to workspace dependencies, phase 3 (#2283) - refactor: consolidate core dependencies at workspace level (phase 1) (#2253) ## libdd-dogstatsd-client **Next version:** `5.0.0` **Semver bump:** `major` **Tag:** `libdd-dogstatsd-client-v5.0.0` ### Commits - refactor: consolidate core dependencies at workspace level (phase 1) (#2253) - feat(dogstatsd)!: add shared_runtime buffered sink (#2224) - refactor(statsd)!: make client clonable (#2222) ## libdd-telemetry **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-telemetry-v7.0.0` ### Commits - refactor(rc)!: make conversion from RemoteConfigProduct back and forth generally available (#2325) - feat(telemetry)!: Add Installation signature and AppProduct changes payloads (#2213) - fix(telemetry)!: stop sending Connection: close to the Agent (#2286) - refactor: migrate to workspace dependencies, phase 3 (#2283) - chore: moving to workspace-level dependencies, phase 2 (#2270) - refactor: consolidate core dependencies at workspace level (phase 1) (#2253) - fix: drain the mailbox before stopping (#2258) - feat(telemetry)!: make telemetry worker wasm-compatible for the TraceExporter (#2172) - fix(sidecar): don't double-encode file:// telemetry endpoints (#2230) ## libdd-trace-obfuscation **Next version:** `6.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-obfuscation-v6.0.0` ###⚠️ major bump forced due to: - `libdd-trace-utils`: ^9.0.0 → ^10.1.0 ### Commits - refactor: migrate to workspace dependencies, phase 4 (#2296) - chore(lint): update nightly (#2281) - chore: moving to workspace-level dependencies, phase 2 (#2270) - refactor: consolidate core dependencies at workspace level (phase 1) (#2253) - fix: new clippy lints (#2219) ## libdd-trace-stats **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-stats-v7.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^2.1.0 → ^3.0.0 - `libdd-capabilities-impl`: ^3.0.0 → ^4.0.0 - `libdd-dogstatsd-client`: ^4.0.0 → ^5.0.0 - `libdd-telemetry`: ^6.0.0 → ^7.0.0 - `libdd-trace-utils`: ^9.0.0 → ^10.1.0 ### Commits - feat: add v1 isolated bricks for trace exporter (#2280) - refactor: migrate to workspace dependencies, phase 4 (#2296) - feat(data-pipeline)!: add compression option for agentless export (#2205) - feat(stats)!: per-field cardinality limits (#2211) - refactor: migrate to workspace dependencies, phase 3 (#2283) - feat(stats)!: string field length limit (#2267) - feat(trace_exporter)!: add `restart_after_fork` configuration option [APMSP-3843] (#2263) - refactor: consolidate core dependencies at workspace level (phase 1) (#2253) - feat(stats)!: implement additional metric tags in libdd-trace-stats (#2170) - refactor(statsd)!: make client clonable (#2222) - fix(stats-exporter): disable retries for stats payload sends (#2249) - feat(telemetry)!: make telemetry worker wasm-compatible for the TraceExporter (#2172) - fix!: css obfuscation logic (#2203) ## libdd-data-pipeline **Next version:** `8.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-v8.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^2.1.0 → ^3.0.0 - `libdd-capabilities-impl`: ^3.0.0 → ^4.0.0 - `libdd-dogstatsd-client`: ^4.0.0 → ^5.0.0 - `libdd-telemetry`: ^6.0.0 → ^7.0.0 - `libdd-trace-stats`: ^6.0.0 → ^7.0.0 - `libdd-trace-utils`: ^9.0.0 → ^10.1.0 ### Commits - fix(trace-exporter): mark OTLP client-computed stats (#2245) - feat(data-pipeline): adds new attributes to OTLP trace metrics export (#2316) - feat: add v1 isolated bricks for trace exporter (#2280) - feat(data-pipeline)!: add flush_and_close to the trace buffer (#2313) - feat(telemetry)!: Add Installation signature and AppProduct changes payloads (#2213) - refactor: migrate to workspace dependencies, phase 4 (#2296) - feat(data-pipeline)!: add compression option for agentless export (#2205) - fix(telemetry)!: stop sending Connection: close to the Agent (#2286) - feat(stats)!: per-field cardinality limits (#2211) - refactor: migrate to workspace dependencies, phase 3 (#2283) - feat(stats)!: string field length limit (#2267) - refactor(trace-utils)!: split generic TracerHeaderTags (#2279) - chore: moving to workspace-level dependencies, phase 2 (#2270) - feat(trace_exporter)!: add `restart_after_fork` configuration option [APMSP-3843] (#2263) - feat(data-pipeline): emit additional_metric_tags on OTLP span duration metric (#2264) - refactor: consolidate core dependencies at workspace level (phase 1) (#2253) - feat(dogstatsd)!: add shared_runtime buffered sink (#2224) - feat(stats)!: implement additional metric tags in libdd-trace-stats (#2170) - refactor(statsd)!: make client clonable (#2222) - feat(telemetry)!: make telemetry worker wasm-compatible for the TraceExporter (#2172) - feat(trace_exporter)!: remove deprecated methods (#2250) - feat(trace-utils)!: add v1 decoder (#2174) - feat(trace-utils)!: add encoder v1 to v04 + refactor (#2145) - feat(otlp): add instrumentation scope metadata (#2235) - fix(trace-utils)!: serialize v0.5 span links and events into meta (#980) - fix!: css obfuscation logic (#2203) - test(data-pipeline): gate telemetry-only tests behind the telemetry feature (#2204) [APMSP-3843]: https://datadoghq.atlassian.net/browse/APMSP-3843?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: hoolioh <107922352+hoolioh@users.noreply.github.com>

What does this PR do?
Provide a SharedRuntime based buffered sink. The current
QueuingMetricSinkspawns a thread which isn't forksafe. This provides an alternative that can be used in forking context.Motivation
Currently the dogstatsd client doesn't survive a fork and all metrics submitted after the fork are never sent.
Additional Notes
Anything else we should know when reviewing?
How to test the change?
Describe here in detail how the change can be validated.