ERROR:opentelemetry.context:Failed to detach context (ValueError: Token created in a different Context) still occurs in 2.3.0 during streaming agent run

[hungubqn0310]

Describe the bug

Recurring ERROR opentelemetry.context: Failed to detach context / ValueError: <Token ...> was created in a different Context logged during a normal streaming agent run via the ADK API server. The request appears to keep working (subsequent events/tool calls are still generated right after), but the errors are logged at ERROR level with full tracebacks on effectively every LLM turn, which is noisy in production.

This looks like the same underlying issue reported in #1670 and #860 (both closed), but on a much newer version — the traceback here goes through internal functions that don't exist in those older reports (_use_extra_generate_content_attributes, use_inference_span, record_inference_telemetry, _instrumentation.py::record_agent_invocation), suggesting either the fix for #1670 didn't fully resolve it or it was reintroduced by the telemetry refactor.

To Reproduce

Running the ADK API server (run_adk.py, streaming mode) behind a chat UI, with an LlmAgent that calls tools (product search) mid-conversation. The error appears intermittently during normal multi-turn tool-calling agent runs, most consistent with the underlying stream/task being torn down (e.g. client-side cancel/reconnect, or SSE stream teardown) while nested tracing context managers are still open — each occurrence throws 3 nested GeneratorExit → detach ValueError tracebacks in a row, from three different context-manager layers (_use_extra_generate_content_attributes, use_span/start_as_current_span around _call_llm_with_tracing, and record_agent_invocation around run_async).

Traceback

2026-08-25 07:26:59,230 - opentelemetry.context - ERROR - Failed to detach context
Traceback (most recent call last):
  File "/opt/app/.venv/lib/python3.12/site-packages/google/adk/telemetry/tracing.py", line 696, in _use_extra_generate_content_attributes
    yield
  File "/opt/app/.venv/lib/python3.12/site-packages/google/adk/telemetry/tracing.py", line 634, in use_inference_span
    yield
  File "/opt/app/.venv/lib/python3.12/site-packages/google/adk/telemetry/_instrumentation.py", line 204, in record_inference_telemetry
    yield tel_ctx
  File "/opt/app/.venv/lib/python3.12/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 387, in _run_and_handle_error
    yield llm_response
GeneratorExit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/app/.venv/lib/python3.12/site-packages/opentelemetry/context/__init__.py", line 155, in detach
    _RUNTIME_CONTEXT.detach(token)
  File "/opt/app/.venv/lib/python3.12/site-packages/opentelemetry/context/contextvars_context.py", line 53, in detach
    self._current_context.reset(token)
ValueError: <Token var=<ContextVar name='current_context' default={} at 0x7d7a4893b880> at 0x7d7a05ca0440> was created in a different Context

2026-08-25 07:26:59,231 - opentelemetry.context - ERROR - Failed to detach context
Traceback (most recent call last):
  File "/opt/app/.venv/lib/python3.12/site-packages/opentelemetry/trace/__init__.py", line 619, in use_span
    yield span
  File "/opt/app/.venv/lib/python3.12/site-packages/opentelemetry/trace/__init__.py", line 519, in start_as_current_span
    yield span
  File "/opt/app/.venv/lib/python3.12/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 1359, in _call_llm_with_tracing
    yield llm_response
GeneratorExit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/app/.venv/lib/python3.12/site-packages/opentelemetry/context/__init__.py", line 155, in detach
    _RUNTIME_CONTEXT.detach(token)
  File "/opt/app/.venv/lib/python3.12/site-packages/opentelemetry/context/contextvars_context.py", line 53, in detach
    self._current_context.reset(token)
ValueError: <Token var=<ContextVar name='current_context' default={} at 0x7d7a4893b880> at 0x7d79e7904440> was created in a different Context

2026-08-25 07:26:59,232 - opentelemetry.context - ERROR - Failed to detach context
Traceback (most recent call last):
  File "/opt/app/.venv/lib/python3.12/site-packages/opentelemetry/trace/__init__.py", line 619, in use_span
    yield span
  File "/opt/app/.venv/lib/python3.12/site-packages/opentelemetry/trace/__init__.py", line 519, in start_as_current_span
    yield span
  File "/opt/app/.venv/lib/python3.12/site-packages/google/adk/telemetry/_instrumentation.py", line 127, in record_agent_invocation
    yield tel_ctx
  File "/opt/app/.venv/lib/python3.12/site-packages/google/adk/agents/base_agent.py", line 306, in run_async
    yield event
GeneratorExit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/app/.venv/lib/python3.12/site-packages/opentelemetry/context/__init__.py", line 155, in detach
    _RUNTIME_CONTEXT.detach(token)
  File "/opt/app/.venv/lib/python3.12/site-packages/opentelemetry/context/contextvars_context.py", line 53, in detach
    self._current_context.reset(token)
ValueError: <Token var=<ContextVar name='current_context' default={} at 0x7d7a4893b880> at 0x7d79e6d54840> was created in a different Context

Immediately after these three tracebacks, the same request continues normally (next streaming event and tool declaration are logged right away), so this does not appear to break the response itself — just log noise at ERROR severity.

Environment

• google-adk: 2.3.0
• opentelemetry-api / opentelemetry-sdk: 1.41.1
• Python: 3.12
• OS: Linux, running in Docker (via ADK API server, streaming mode)

Additional context

Functionally harmless so far in our production traffic (no dropped/missing responses observed correlating with these errors), but it pollutes logs at ERROR level on effectively every LLM turn. Related: #1670, #860, #2792.

[surajksharma07]

@hungubqn0310 The ValueError is actually already being caught internally by OpenTelemetry itself, just logged at ERROR so it's noisy but not breaking your responses like you noticed.
As a workaround try adding a small logging.Filter on the "opentelemetry.context" logger to drop just that specific message and see if it quiets things down on your end.
Once you've confirmed it holds up under your normal traffic for a bit happy to have you raise a PR for it.

[hungubqn0310]

@surajksharma07 Thanks for confirming — that matches what I saw too (responses were never actually broken, just the noisy ERROR traceback).

Applied the suggested workaround: added a logging.Filter on the opentelemetry.context logger that drops only the exact "Failed to detach context" message, leaving any other ERROR from that logger untouched. Verified locally that it suppresses just the target message and nothing else gets swallowed.

Rolling it out to our staging/prod traffic now — will report back once it's held up for a bit, and happy to raise a PR at that point. Thanks for the quick turnaround!