Skip to content

Add installed Claude and Codex tracing journeys - #739

Merged
tt-le merged 7 commits into
mainfrom
tien/codex-otel-tracing-integration
Sep 23, 2026
Merged

tt-le merged 7 commits into
mainfrom
tien/codex-otel-tracing-integration

Conversation

@tt-le

@tt-le tt-le commented Sep 18, 2026 •

Copy link
Copy Markdown
Collaborator

What & why

Add installed-product customer journeys for the Claude Code and Codex tracing paths now on main:

  1. Install the checkout wheel and pinned agent version through the existing integration runner.
  2. Configure the agent against the eng-ml inference e2e workspace with a tracing-enabled CodingAgentConfig input.
  3. Complete a real headless file task carrying a unique UUID.
  4. Resolve the workspace's effective tracing destination, wait 30 seconds, and require that table to contain a span with both the UUID and the model that ran.

The workspace deliberately has no published CodingAgentConfig, so these journeys use the integration suite's managed-config stub seam for only that configuration input. The agents, inference, OTLP export, SQL warehouse, and table verification remain real.

Claude Code sends traces directly to the gateway and obtains the current bearer through its otelHeadersHelper; the journey correlates those spans with OTEL_RESOURCE_ATTRIBUTES and verifies gen_ai.request.model. Codex sends credential-free OTLP requests to its loopback proxy, which injects the current Databricks token only on the upstream trace request; its marker uses Codex's supported otel.span_attributes configuration. Inference traffic does not use the Codex proxy.

Both journeys run only in the existing Full lane for their agent and reuse the eng-ml inference workspace and bearer. During setup they get the workspace ID from an authenticated response, read the effective tracing config, and derive the configured <catalog>.<schema>.<prefix>_otel_spans table; the workflow and runner do not carry a hardcoded table name.

Testing

  • Integration contract tests: 15 passed.
  • Integration evidence tests: 6 passed.
  • Ruff check and formatting pass.
  • Runner help and Python compile checks pass.
  • Codex CI run 35630082102 completed the real task and reached the post-ingestion SQL check. The check is currently blocked because the identity behind DATABRICKS_BEARER lacks USE SCHEMA on the configured tracing schema; the test intentionally remains strict rather than skipping table verification.
  • The Claude journey is included in the Full Claude lane for CI validation.

This pull request and its description were written by Isaac.

@tt-le
tt-le force-pushed the tien/codex-otel-tracing-integration branch from e47cf08 to 7f8a34b Compare September 18, 2026 14:20
An error occurred while trying to automatically change base from tien/codex-otel-gateway-proxy to tien/gateway-proxy-parameterize September 18, 2026 14:53
@tt-le tt-le changed the title Add an installed Codex tracing journey Add installed Claude and Codex tracing journeys Sep 22, 2026
@tt-le
tt-le force-pushed the tien/codex-otel-tracing-integration branch from 89449cd to b15d7af Compare September 22, 2026 22:00
@tt-le
tt-le changed the base branch from tien/codex-otel-gateway-proxy to main September 22, 2026 22:00
@tt-le
tt-le force-pushed the tien/codex-otel-tracing-integration branch from b15d7af to faa58b4 Compare September 22, 2026 22:02
tt-le and others added 6 commits September 22, 2026 23:07
Point the tracing journey at the CI trace table
main.aigw_tracing.unity_gateway_otel_spans and assert the marker span
also carries Codex's `model` attribute, so the trace is attributable to
the model that actually ran.

Co-authored-by: Isaac <no-reply@databricks.com>
@tt-le
tt-le force-pushed the tien/codex-otel-tracing-integration branch from 288db31 to 881da93 Compare September 22, 2026 23:17
@tt-le
tt-le enabled auto-merge (squash) September 23, 2026 13:56
@tt-le
tt-le merged commit 9947d37 into main Sep 23, 2026
38 of 41 checks passed
@tt-le
tt-le deleted the tien/codex-otel-tracing-integration branch September 23, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants