Name resolution delay - #12893
Open
AgraVator wants to merge 54 commits into
Open
Conversation
This commit implements the plumbing required to propagate delay reason tokens from load balancing policies up to the transport layer and tracers, as specified in the LB policy delay design.
…dence invariants - Refactor ClientStreamTracer to expose delayTypeStarted(String) and delayReasonAttached(String) - Enhance PickResult with separate delayType and delayReason diagnostic fields - Implement Mark Roth's hybrid telemetry cadence model in DelayedClientTransport.PendingStream - Support channel fallback delay states (client_channel_init, subchannel_state_mismatch, wait_for_ready_failed) - Simplify leaf and container LB policies to emit canonical unified connecting metric labels
# Conflicts: # core/src/main/java/io/grpc/internal/PickFirstLeafLoadBalancer.java
… in OpenTelemetry modules
…ay and Attempt-Level Delay
…for name resolution in ManagedChannelImpl
AgraVator
marked this pull request as draft
July 6, 2026 14:32
…name-resolution-delay branch
…ame-resolution-delay branch
…elay # Conflicts: # core/src/main/java/io/grpc/internal/DelayedClientTransport.java # core/src/main/java/io/grpc/internal/PickFirstLoadBalancer.java # core/src/test/java/io/grpc/internal/DelayedClientTransportTest.java # opentelemetry/src/main/java/io/grpc/opentelemetry/GrpcOpenTelemetry.java # opentelemetry/src/main/java/io/grpc/opentelemetry/OpenTelemetryMetricsModule.java # opentelemetry/src/main/java/io/grpc/opentelemetry/OpenTelemetryMetricsResource.java # opentelemetry/src/main/java/io/grpc/opentelemetry/OpenTelemetryTracingModule.java # opentelemetry/src/test/java/io/grpc/opentelemetry/OpenTelemetryMetricsModuleTest.java # opentelemetry/src/test/java/io/grpc/opentelemetry/OpenTelemetryTracingModuleTest.java # rls/src/main/java/io/grpc/rls/CachingRlsLbClient.java # rls/src/test/java/io/grpc/rls/CachingRlsLbClientTest.java # util/src/main/java/io/grpc/util/RoundRobinLoadBalancer.java # util/src/test/java/io/grpc/util/RoundRobinLoadBalancerTest.java # xds/src/main/java/io/grpc/xds/CdsLoadBalancer2.java # xds/src/main/java/io/grpc/xds/PriorityLoadBalancer.java # xds/src/test/java/io/grpc/xds/CdsLoadBalancer2Test.java # xds/src/test/java/io/grpc/xds/PriorityLoadBalancerTest.java
… and expand unit/stress tests
…nelImpl callback pattern
…y observability (gRFC A66) - Add nameResolutionDelay, lbPolicyDelay, and baselineNoDelay end-to-end tests to GrpcOpenTelemetryTest - Update LoadBalancer.PickResult.withError to set delayType="connecting" and delayReason=error.getDescription() - Fix missing static import checkstyle violation in CdsLoadBalancer2Test
…lay, remove debug prints, restore stress tests and add unit tests for patch coverage
…hStreamTracerFactory tests in LoadBalancerTest
…est and ManagedChannelImplTest
AgraVator
marked this pull request as ready for review
August 3, 2026 14:37
…endingCall, PendingStream, and OTel modules
…e resolution delay tests
Contributor
|
Can you check the missing code coverages from the report and fix them? |
…s in delay observability
…lientTransport and OpenTelemetryTracingModule
…mptDelay, and duplicate cancel handling
…plugins in delay observability
kannanjgithub
requested changes
Aug 13, 2026
kannanjgithub
left a comment
kannanjgithub
left a comment
Contributor
There was a problem hiding this comment.
There are still some missing branches in codecov report. I have commented about missing coverages for OpenTelemetryMetricsModule.
Also address still existing coverages in the report except for cases where it cannot be done without changing class visibility or other reasons.
…edback, and prune redundant tests - Maintain private encapsulation for realChannel, RealChannel, and PendingCall in ManagedChannelImpl. - Synchronize delay lifecycle checks and tracer notifications to establish happens-before boundaries. - Add defensive lifecycle guards in OpenTelemetryMetricsModule to handle abrupt call endings and reject starts after callEnded. - Address codecov and reviewer feedback: - Test delay type transitions while active for both call and attempt delay metrics. - Test that null delay types record no metric in OpenTelemetryMetricsModule. - Test that null delay types fall back safely to empty strings in OpenTelemetryTracingModule. - Cover syncContext cancellation and tracer abort branches in ManagedChannelImpl using public API. - Prune redundant duplicate label tests, reflection helpers, and multi-threaded barrier stress tests.
… fast paths - In ManagedChannelImpl.PendingCall, replace AtomicBoolean delayEnded with boolean field guarded by monitor to avoid per-call heap allocation and CAS overhead. - In OpenTelemetryMetricsModule and OpenTelemetryTracingModule: - Check experiment flag before acquiring monitor on delay start. - Implement lock-free fast exit in record*DelayEnd when no delay is active. - Declare active delay stopwatches/spans as volatile for safe lock-free null reads. - Guard activeDelayType strictly with monitor lock. - In OpenTelemetryMetricsModuleTest, assert grpc.client.call.delay.duration metric recording on delay type transitions.
… volatile, and align with upstream
kannanjgithub
requested changes
Sep 9, 2026
…olve review comments - Unify call- and attempt-level child delay tracing span names to "Delay" - Align "Delay state transition" span event to record only grpc.delay_reason - Remove redundant local variables in OpenTelemetryTracingModule and OpenTelemetryMetricsModule - Eliminate redundant fully qualified Attributes references - Clean up synchronization and remove redundant volatile qualifiers - Update tests to follow repository style and verify updated span names and attributes
kannanjgithub
requested changes
Sep 10, 2026
… coverage - Update grpc.client.call.delay.duration and grpc.client.attempt.delay.duration descriptions to match gRFC A121 - Add assertions for metric descriptions, units, and required labels (grpc.target, grpc.method, grpc.delay_type) - Add test verifying null-safe behavior when delay observability flag is enabled but metrics are not opted in
kannanjgithub
approved these changes
Sep 10, 2026
AgraVator
commented
Sep 10, 2026
Comment on lines
+228
to
+230
| activeCallDelaySpan.addEvent( | ||
| "Delay state transition", | ||
| Attributes.of(AttributeKey.stringKey("grpc.delay_reason"), delayReason)); |
Contributor
Author
There was a problem hiding this comment.
Being discussed at grpc/proposal#556 (comment)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This finishes the remaining work in #12807