Skip to content

Fix synchronous cancellation race in Channels - #132230

Open
steveisok wants to merge 1 commit into
mainfrom
steveisok-investigate-bounded-channel-loss
Open

Fix synchronous cancellation race in Channels#132230
steveisok wants to merge 1 commit into
mainfrom
steveisok-investigate-bounded-channel-loss

Conversation

@steveisok

Copy link
Copy Markdown
Member

Fixes #129796
Fixes #132094

AsyncOperation used _cancellationRegistration.Token to determine whether completion needed atomic reservation. If cancellation ran synchronously inside UnsafeRegister, the registration had not yet been assigned, so the callback observed a default non-cancelable token and completed without reserving the operation.

A reader or waiter could then be reserved and completed a second time. This caused linked-list assertion failures in RendezvousChannel and could silently lose items during a BoundedChannel direct handoff.

Use an immutable _isCancelable value initialized before registration can invoke the callback. This preserves the non-cancelable fast path, pooling behavior, and registration cleanup without retaining the full CancellationToken on modern .NET.

Adds deterministic coverage for:

  • synchronous cancellation during registration
  • skipping a canceled bounded-channel reader during direct handoff
  • delayed rendezvous waiter removal after the list has been drained

The added field fits existing object padding; BlockedReadAsyncOperation<int> remains 96 bytes.

Validation

  • System.Threading.Channels Debug tests: 1,599 passed, 2 existing stress tests skipped
  • Debug and Release library builds: succeeded with no warnings
  • Release 3,000,000-item repro: 3,000,000 written and read, no missing items

Note

This pull request description was generated with the assistance of GitHub Copilot.

Use stable cancelability state initialized before cancellation registration, and add deterministic regression coverage for bounded and rendezvous channels.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @JulieLeeMSFT, @VSadov
See info in area-owners.md if you want to be subscribed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a race in System.Threading.Channels async operations where synchronous cancellation during token registration could cause completion to proceed without an atomic reservation, enabling double-completion and corrupting waiter/reader linked lists (with downstream assertion failures and potential item loss).

Changes:

  • Make cancelability an immutable construction-time property (_isCancelable) and use it for completion reservation decisions, avoiding reliance on _cancellationRegistration.Token when registration may be a default value.
  • Add test infrastructure to construct synchronously-canceled internal channel async operations and inspect/modify their linked-list state.
  • Add deterministic regression tests for rendezvous waiter removal and bounded-channel direct handoff skipping canceled readers.
Show a summary per file
File Description
src/libraries/System.Threading.Channels/src/System/Threading/Channels/AsyncOperation.cs Introduces _isCancelable and switches completion reservation/assert logic to use it, addressing the synchronous-cancellation registration race.
src/libraries/System.Threading.Channels/tests/TestBase.cs Adds reflection-based helpers to create/cancel internal async operations and manipulate their list links/heads for deterministic regression coverage.
src/libraries/System.Threading.Channels/tests/RendezvousChannelTests.cs Adds a regression test validating delayed removal of a synchronously-canceled waiter doesn’t corrupt the waiting-reader list.
src/libraries/System.Threading.Channels/tests/BoundedChannelTests.cs Adds regression tests for synchronous cancellation during registration and for skipping canceled readers during direct handoff.

Review details

  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment on lines +37 to +60
protected static object CreateSynchronouslyCanceledAsyncOperation(
string typeName,
CancellationToken cancellationToken,
Type genericTypeArgument = null)
{
Type operationType = typeof(Channel<int>).Assembly.GetType(typeName, throwOnError: true);
if (genericTypeArgument is not null)
{
operationType = operationType.MakeGenericType(genericTypeArgument);
}

MethodInfo trySetCanceled = operationType.GetMethod("TrySetCanceled", BindingFlags.Instance | BindingFlags.Public);
var cancellationCallback = new Action<object, CancellationToken>((state, token) =>
{
Assert.True((bool)trySetCanceled.Invoke(state, new object[] { token }));
});

return Activator.CreateInstance(
operationType,
BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic,
binder: null,
args: new object[] { true, cancellationToken, false, cancellationCallback },
culture: null);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

2 participants