Expand connection pool benchmark coverage for pool implementation A/B testing by mdaigle · Pull Request #4459 · dotnet/SqlClient

mdaigle · 2026-07-20T21:06:25Z

Summary

Builds on the connection pool benchmarks added in #4447 (now merged to main) to make the suite useful for evaluating the new ChannelDbConnectionPool (UseConnectionPoolV2) against the legacy WaitHandleDbConnectionPool, aligned with the design goals in #3356: parallel connection opening, async best practices (low threadpool pressure), and minimal lock contention.

What's added

Process-level UseConnectionPoolV2 config flag (runnerconfig.jsonc -> Config -> Program). It sets the Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2 AppContext switch once at startup.
- The pool-implementation switch is read and cached when the first pool is created, so it cannot be toggled per benchmark iteration (BenchmarkDotNet runs all cases in-process). Comparing the two pools therefore requires two runs — flag false (legacy) then true (V2). This flag applies to every pool benchmark in the suite, including the existing ConnectionPoolStressRunner and ParallelAsyncConnectionRunner.
ConnectionPoolChurnRunner — single-threaded, no-contention open/close on a warm pool. Isolates the raw per-checkout CPU and allocation cost of the pool's acquire/return path (the low-noise counterpart to the parallel/contention runners; the most sensitive measure of per-op allocations).
ConnectionPoolContentionRunner — steady-state open -> SELECT 1 -> close with pure-sync vs pure-async workers and a large (no-wait) vs small (back-pressure) pool axis. Focuses on the waiter wake path and threadpool pressure (watch Completed Work Items / Lock Contentions).

The existing ConnectionPoolStressRunner and ParallelAsyncConnectionRunner are left untouched — the new flag simply lets them all be run against either pool implementation.

Not micro-benchmarked

Rate limiting (#4396), pruning (#4304), shutdown drain (#4302), and connection resiliency (#4415) are time/behavior-based and belong in reliability tests (#3667), not hot-path microbenchmarks.

Validation

Built PerformanceTests (net8.0, Release) — 0 warnings / 0 errors.
Ran all three pool runners locally against a live SQL Server (Azure SQL Edge) with the flag both false and true; V2 showed ~3.5-4.3x faster parallel burst-open and materially lower async Completed Work Items, consistent with Feature | Redesign the SqlClient Connection Pool to Improve Performance and Async Support #3356's goals.

Checklist

Tests added or updated (benchmark runners)
Public API changes documented — N/A (no public API changes)
Verified against live SQL Server
No breaking changes

Related: #3356, #601, #979, #4447

… testing Adds tooling and benchmarks to evaluate the new ChannelDbConnectionPool (UseConnectionPoolV2) against the legacy WaitHandleDbConnectionPool, aligned with the design goals in issue #3356 (parallel connection opening, async best practices / low threadpool pressure, minimal lock contention). - Add a process-level `UseConnectionPoolV2` config flag (runnerconfig.jsonc -> Config -> Program) that sets the AppContext switch once at startup. The pool implementation switch is read and cached when the first pool is created, so it cannot be toggled per iteration; comparing the two pools requires two runs (flag false, then true). This applies to every pool benchmark in the suite. - Add ConnectionPoolChurnRunner: single-threaded, no-contention open/close on a warm pool. Isolates raw per-checkout CPU and allocation cost (the low-noise counterpart to the parallel/contention runners). - Add ConnectionPoolContentionRunner: steady-state open/query/close with pure sync vs pure async workers and a large (no-wait) vs small (back-pressure) pool axis, focused on the waiter wake path and threadpool pressure. Leaves the existing ConnectionPoolStressRunner and ParallelAsyncConnectionRunner untouched; the new flag lets them all be run against either pool implementation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Expands the Microsoft.Data.SqlClient.PerformanceTests benchmark suite to better compare the legacy WaitHandleDbConnectionPool vs the new ChannelDbConnectionPool (via Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2) by adding a process-level config flag and two additional pool-focused runners.

Changes:

Adds UseConnectionPoolV2 to runnerconfig.jsonc + Config and wires it in Program to set the AppContext switch at process startup.
Introduces ConnectionPoolChurnRunner for single-thread warm-pool checkout/return cost measurement.
Introduces ConnectionPoolContentionRunner for concurrent steady-state open/query/close throughput across sync vs async and pool size contention axes.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/Microsoft.Data.SqlClient/tests/PerformanceTests/runnerconfig.jsonc	Adds `UseConnectionPoolV2` flag and enables configs for the two new pool runners.
src/Microsoft.Data.SqlClient/tests/PerformanceTests/Program.cs	Sets `UseConnectionPoolV2` AppContext switch early and registers the new runners.
src/Microsoft.Data.SqlClient/tests/PerformanceTests/Config/Config.cs	Adds a documented `UseConnectionPoolV2` config field and runner job entries.
src/Microsoft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolContentionRunner.cs	New benchmark runner for steady-state pool contention behavior under sync/async workloads.
src/Microsoft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolChurnRunner.cs	New benchmark runner for low-noise single-thread warm-pool churn cost.

- Split SteadyStateOpenQueryClose into separate async/sync Task.Run paths (Task.Run(Func<Task>) vs Task.Run(Action)) so the sync branch does not create an async state machine, removing overhead unrelated to pool behavior. - Wrap WarmPool connections in try/finally and dispose each SqlConnection after closing to prevent retaining objects until GC each iteration, which would add allocation noise to the benchmark measurements. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/Microsoft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolContentionRunner.cs:113

In the async mode, wrapping each worker in Task.Run forces work onto the thread pool and adds scheduler/queueing overhead that can dominate the measurement and inflate ThreadingDiagnoser metrics (e.g., Completed Work Items). Since this runner is meant to highlight the async waiter/wake differences between pool implementations, it should start the async workers without Task.Run (e.g., an immediately-invoked async lambda) so the only thread-pool usage comes from actual async continuations.

                if (Async is AsyncBehavior.Async)
                {
                    tasks[i] = Task.Run(async () =>
                    {

codecov · 2026-07-20T23:06:02Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.55%. Comparing base (fdebcd2) to head (7b0450e).
⚠️ Report is 27 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4459      +/-   ##
==========================================
- Coverage   65.83%   63.55%   -2.28%     
==========================================
  Files         287      283       -4     
  Lines       43763    66821   +23058     
==========================================
+ Hits        28812    42470   +13658     
- Misses      14951    24351    +9400

Flag	Coverage Δ
CI-SqlClient	`?`
PR-SqlClient-Project	`63.55% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

cheenamalhotra

LGTM, minor improvements will improve benchmark accuracy.

benrr101

I like it - I think it's pretty straightforward!

benrr101 · 2026-07-22T23:32:00Z

+        [GlobalSetup]
+        public void Setup()
+        {
+            Console.WriteLine(


Does BenchmarkDotnet prefer using some other logger for output or does it handle console just fine?

Console.WriteLine is fine here. BenchmarkDotNet runs each benchmark in a separate child process and captures its stdout, so [GlobalSetup] output shows up in the run log. The other runners in this suite already log this way, so this keeps it consistent.

benrr101 · 2026-07-22T23:38:17Z

+        /// means no contention for physical connections; a smaller value forces workers to
+        /// wait for connections to be returned (pool exhaustion / back-pressure).
+        /// </summary>
+        [Params(50, 10)]


Do we also want a case that greater than the max parallelism?

Added — MaxPoolSize is now [100, 50, 10] against Parallelism = 50, so we cover all three regimes: pool larger than the worker count (idle spares, no contention), equal (fully subscribed), and smaller (back-pressure).

Address review feedback: remove the AsyncBehavior param from the churn and contention runners and expose dedicated sync and async benchmarks. The previous single async-signed method forced the sync path to pay async state-machine overhead, distorting per-operation allocation measurements. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Address review feedback: the switch is always set from config (true or false), not only when the flag is enabled. Reword the comment accordingly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Address review feedback: add a MaxPoolSize (100) greater than Parallelism (50) so the contention runner covers all three regimes — pool larger than, equal to, and smaller than the worker count. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/Microsoft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolContentionRunner.cs:121

This benchmark also doesn't need an async state machine at the top level; it can return Task.WhenAll(tasks) directly. That keeps the worker behavior the same while reducing per-invocation allocations/overhead.

        [Benchmark]
        public async Task SteadyStateOpenQueryCloseAsync()
        {
            var tasks = new Task[Parallelism];
            for (int i = 0; i < Parallelism; i++)

Address review feedback: the contention benchmark methods only build a task array and await Task.WhenAll, so they don't need an async state machine. Return the Task directly to avoid an extra allocation per invocation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings July 20, 2026 21:06

mdaigle requested a review from a team as a code owner July 20, 2026 21:06

github-project-automation Bot added this to SqlClient Board Jul 20, 2026

github-project-automation Bot moved this to To triage in SqlClient Board Jul 20, 2026

Copilot started reviewing on behalf of mdaigle July 20, 2026 21:06 View session

Copilot AI reviewed Jul 20, 2026

View reviewed changes

Comment thread ...oft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolContentionRunner.cs

Comment thread ...oft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolContentionRunner.cs

Copilot AI review requested due to automatic review settings July 20, 2026 21:44

Copilot started reviewing on behalf of mdaigle July 20, 2026 21:45 View session

Copilot AI reviewed Jul 20, 2026

View reviewed changes

Comment thread ...icrosoft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolChurnRunner.cs

paulmedynski moved this from To triage to In progress in SqlClient Board Jul 21, 2026

paulmedynski added the Area\Connection Pooling Use this label to tag issues that apply to problems with connection pool. label Jul 21, 2026

paulmedynski added this to the 7.1.0-preview3 milestone Jul 21, 2026

github-actions Bot assigned benrr101 and cheenamalhotra Jul 21, 2026

paulmedynski moved this from In progress to In review in SqlClient Board Jul 21, 2026

cheenamalhotra requested changes Jul 22, 2026

View reviewed changes

Comment thread ...icrosoft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolChurnRunner.cs Outdated

Comment thread ...oft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolContentionRunner.cs Outdated

github-project-automation Bot moved this from In review to Waiting for customer in SqlClient Board Jul 22, 2026

benrr101 previously approved these changes Jul 22, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 23, 2026 21:47

mdaigle dismissed benrr101’s stale review via bfbe1aa July 23, 2026 21:47

Copilot started reviewing on behalf of mdaigle July 23, 2026 21:48 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Comment thread src/Microsoft.Data.SqlClient/tests/PerformanceTests/Program.cs Outdated

Clarify UseConnectionPoolV2 switch comment in perf test setup

f7980d3

Address review feedback: the switch is always set from config (true or false), not only when the flag is enabled. Reword the comment accordingly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 23, 2026 22:03

Copilot started reviewing on behalf of mdaigle July 23, 2026 22:04 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 23, 2026 22:23

Copilot started reviewing on behalf of mdaigle July 23, 2026 22:24 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Comment thread ...oft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolContentionRunner.cs

Copilot AI review requested due to automatic review settings July 23, 2026 22:45

Copilot started reviewing on behalf of mdaigle July 23, 2026 22:46 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Uh oh!

Conversation

mdaigle commented Jul 20, 2026

Summary

What's added

Not micro-benchmarked

Validation

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

codecov Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cheenamalhotra left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

benrr101 left a comment

Choose a reason for hiding this comment

Uh oh!

benrr101 Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

mdaigle Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

benrr101 Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

mdaigle Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov Bot commented Jul 20, 2026 •

edited

Loading

cheenamalhotra left a comment •

edited

Loading