Skip to content

feat(stats)!: per-field cardinality limits - #2211

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 42 commits into
mainfrom
oscarld/stats-per-key-cardinality-limit
Jul 29, 2026
Merged

feat(stats)!: per-field cardinality limits#2211
gh-worker-dd-mergequeue-cf854d[bot] merged 42 commits into
mainfrom
oscarld/stats-per-key-cardinality-limit

Conversation

@Eldolfin

@Eldolfin Eldolfin commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Implement per-key cardinality limits to stats computation.

Also:

  • fix overflow key peer_tags: [] -> [tracer_blocked_value]
  • fix encode_grouped_stats meta with empty value: key: -> key (breaking change ?)

Motivation

CSS spec compliance.

Additional Notes

Todo:

How to test the change?

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Jul 8, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 97.39%
Overall Coverage: 74.93% (+0.16%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e8ffd6e | Docs | Datadog PR Page | Give us feedback!

@Eldolfin

Eldolfin commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit branch from a5a4786 to 2845fa8 Compare July 8, 2026 12:50
@Eldolfin
Eldolfin changed the base branch from main to oscarld/fix-css-obfuscation-logic July 8, 2026 12:51
@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit branch from 2845fa8 to fca0f65 Compare July 8, 2026 12:52

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a5a4786925

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
Comment thread libdd-data-pipeline/src/trace_exporter/builder.rs
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
@dd-octo-sts

dd-octo-sts Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 87.20 MB 87.33 MB +.15% (+135.77 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.01 MB 8.01 MB +0% (+48 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 98.45 MB 98.59 MB +.14% (+142.12 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.78 MB 10.78 MB +.04% (+4.53 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.06 MB 26.08 MB +.10% (+28.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.60 KB +.47% (+430 B) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 188.08 MB 188.23 MB +.07% (+152.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 979.11 MB 980.23 MB +.11% (+1.12 MB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.47 MB 8.48 MB +.15% (+13.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.60 KB +.47% (+430 B) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.05 MB 25.08 MB +.15% (+40.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 49.83 MB 49.90 MB +.14% (+71.80 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 22.62 MB 22.65 MB +.12% (+28.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 90.58 KB 91.00 KB +.46% (+434 B) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 192.55 MB 192.71 MB +.08% (+160.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 970.38 MB 971.14 MB +.07% (+778.24 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.54 MB 6.55 MB +.20% (+13.50 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 90.58 KB 91.00 KB +.46% (+434 B) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 26.91 MB 26.94 MB +.14% (+40.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 47.43 MB 47.50 MB +.14% (+71.32 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 77.70 MB 77.82 MB +.14% (+116.47 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.92 MB 8.93 MB +.13% (+12.06 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 93.26 MB 93.38 MB +.12% (+122.60 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.85 MB 10.87 MB +.13% (+15.23 KB) 🔍

@Eldolfin
Eldolfin marked this pull request as ready for review July 8, 2026 15:15
@Eldolfin
Eldolfin requested review from a team as code owners July 8, 2026 15:15

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d82ebfd172

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-data-pipeline-ffi/src/trace_exporter.rs
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs
Comment thread libdd-trace-stats/src/span_concentrator/mod.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/mod.rs
@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit branch from abb8d3a to efe280f Compare July 10, 2026 11:53

@VianneyRuhlmann VianneyRuhlmann left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, although I think we should make sure that this is shipped in the same release as the telemetry PR

Comment thread libdd-data-pipeline-ffi/src/trace_exporter.rs
@gh-worker-ownership-write-b05516
gh-worker-ownership-write-b05516 Bot removed the request for review from a team July 21, 2026 09:55
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Jul 21, 2026
…2170)

# What does this PR do?

- Add additional metric tags to the stats aggregation key. For each configured key, the matching span meta tag is included in the key and populates `ClientGroupedStats.additional_metric_tags`.
- Limit the number of keys for `additional_metric_tags` to 4, not configurable.
- Limit the maximum length of individual additional_metric_tags values to 200 characters, not configurable. Values exceeding this are replaced with `tracer_blocked_value`.

# Motivation

- Allow Serverless Compatibility Layer to send additional metric tags on trace stats. See [RFC](https://datadoghq.atlassian.net/wiki/spaces/APM/pages/6482919540/PENDING+Span-Derived+Primary+Tags+-+V1).
- DataDog/serverless-components#140 to follow.
- https://datadoghq.atlassian.net/browse/SVLS-8787

# Additional Notes

Additional metric tags still need to be implemented in libdd-data-pipeline. Omitted from this PR since the goal of this PR is to support additional metric tags tags for Serverless without affecting other teams.

https://github.com/DataDog/libdatadog/blob/42e3296aa3fa76749caf5d96b5bbf1ff2b1913b6/libdd-data-pipeline/src/trace_exporter/stats.rs#L146


Per-field cardinality limit for additional metric tags will be implemented in a following PR, #2211.
- Limit the number of distinct aggregation entries per bucket that carry additional metric tags to 100 by default, configurable via `additional_metric_tags_max_entries`. Once exceeded, new entries have their tag values masked to `tracer_blocked_value`.

---

Implementing additional metric tags instead of span derived primary tags. See previous PR for span derived primary tags - #1815.

# How to test the change?

- Unit tests
- Deploy to Azure Functions with the following environment variables and validate that additional tags appear on trace metrics
  * `DD_TAGS: custom.primary:val,workload:azure_functions`
  * `DD_TRACE_EXPERIMENTAL_FEATURES_ENABLED: true`
  * `DD_TRACE_STATS_ADDITIONAL_TAGS: custom.primary,workload`

<img width="685" height="411" alt="Screenshot 2026-07-06 at 3 39 47 PM" src="https://github.com/user-attachments/assets/b91495b6-11aa-4051-922f-63655666c4e5" />


Co-authored-by: duncan.harvey <duncan.harvey@datadoghq.com>
@Eldolfin

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2906b45c52

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-trace-stats/src/span_concentrator/mod.rs
@Eldolfin Eldolfin changed the title feat(stats)!: per-key cardinality limits feat(stats)!: per-field cardinality limits Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants