Skip to content

feat(snuba): record gauge for batch size. - #8300

Merged
tryangul merged 2 commits into
masterfrom
rbroughan/record-batch-size-gauge
Aug 13, 2026
Merged

feat(snuba): record gauge for batch size.#8300
tryangul merged 2 commits into
masterfrom
rbroughan/record-batch-size-gauge

Conversation

@tryangul

@tryangul tryangul commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

What

Add gauge metrics for per-batch rows and bytes on ClickHouse inserts.

How

  • Emit new gauges: insertions.batch_flush_msgs and insertions.batch_flush_bytes

Why

  • We're tuning batch sizes for reduced merge pressure and I want to see how many messages we're actually sending per batch at a glance
  • Existing counter metrics (batch_write_msgs, batch_write_bytes) get summed across aggregation intervals, making batch sizes look inflated at wider time windows
  • Gauges report actual per-batch values at any zoom level

Notes

  • Keeps existing counters for throughput

Links

@tryangul
tryangul requested a review from a team as a code owner August 13, 2026 04:13

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1b6e10c. Configure here.

Comment thread rust_snuba/src/strategies/clickhouse/writer_v2.rs Outdated
Comment thread rust_snuba/src/strategies/clickhouse/writer_v2.rs Outdated

counter!("insertions.batch_write_bytes", num_bytes as i64);
counter!("insertions.batch_write_msgs", batch_len as i64);
gauge!("insertions.batch_write_bytes", num_bytes as i64);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a new name.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch

@phacops phacops left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need both the counters and the gauges in that case? I'd be down to remove the counters.

@tryangul
tryangul enabled auto-merge (squash) August 13, 2026 15:02
@tryangul
tryangul merged commit db2acd7 into master Aug 13, 2026
67 checks passed
@tryangul
tryangul deleted the rbroughan/record-batch-size-gauge branch August 13, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants