Add Engine Telemetry - Phase 1 by aaronburtle · Pull Request #3825 · Azure/data-api-builder

aaronburtle · 2026-09-25T09:10:43Z

Why make this change?

Add engine-owned telemetry for runtime context, configured capabilities, lifecycle, and aggregate usage across REST, GraphQL, and MCP without relying on customer diagnostic pipelines. This establishes a default-off, synthetic-validation implementation, not production collection.

Set these variables before starting DAB in a dedicated process using synthetic test data.

Environment variable	Value	Purpose
`DAB_PRODUCT_TELEMETRY_TEST_MODE`	`1` or `true`	Explicitly enables synthetic telemetry validation.
`DAB_PRODUCT_TELEMETRY_CONNECTION_STRING`	Full connection string for the selected test Application Insights resource	Selects the product telemetry destination, independently of customer-configured observability. A destination alone does not enable collection.
`APPLICATIONINSIGHTS_STATSBEAT_DISABLED`	`true`	Required for this validation phase; disables the SDK's separate Microsoft health/usage stream.
`APPLICATIONINSIGHTS_SDKSTATS_DISABLED`	`true`	Required for this validation phase; disables SDK delivery-statistics collection and its metadata detection.
`DAB_TELEMETRY_OPT_OUT`	Leave unset for collection; `1` or `true` disables it	Optional override that takes precedence over all product telemetry enablement.

The two SDK statistics flags require literal true (case-insensitive, without surrounding whitespace). They are process-wide settings; DAB checks them but never changes them.

Test mode does not generate synthetic data or anonymize real traffic. Use an isolated test workload and destination, and do not commit connection strings.

What is this change?

Adds per-engine lifecycle events for startup, readiness, configuration changes/rejections, first served/successful requests, heartbeats, and shutdown.
Captures immutable, categorical runtime context and configuration snapshots, preserving configured versus effective settings and configuration epochs across reloads.
Measures requests, logical operations, SQL attempts/retries, per-layer cache lookups, embeddings, HTTP outcomes, and request latency as separate units.
Integrates protocol-aware completion for REST, GraphQL, MCP HTTP/SSE, and MCP stdio. GraphQL eligibility respects conditional selections and individual variable-batch members; discovery and health traffic are excluded.
Adds random API identity with safe best-effort persistence and explicit per-run fallback. No configuration hashes, request contents, or customer-defined names are collected.
Aggregates usage into six-hour UTC windows with bounded counters, explicit loss indicators, and bounded memory-only delivery. There is no event disk spool or telemetry network wait in request processing.
Uses Azure.Monitor.OpenTelemetry.Exporter 1.9.0 through a dedicated custom-event pipeline, with explicit destination selection, payload guards, and source-IP suppression. Updates compatible Azure/OpenTelemetry dependencies.
Adds the umbrella DAB_TELEMETRY_OPT_OUT, including suppression of the DAB-added Application Name/version segment while preserving customer content and legacy-switch behavior.
Documents enablement, collected fields, identity reset, validation, and known SDK limitations.

Scope: Collection remains off by default in every build. CLI command telemetry, installation-profile identity, CLI-to-engine linkage, and production activation remain separate work. Arbitrary embedded/customer-exporter coexistence is not claimed.

How was this tested?

Integration Tests
Unit Tests

Sample Request(s)

For a dedicated synthetic fixture, configure these before process startup:

DAB_PRODUCT_TELEMETRY_TEST_MODE=1
DAB_PRODUCT_TELEMETRY_CONNECTION_STRING pointing to the selected test destination
APPLICATIONINSIGHTS_STATSBEAT_DISABLED=true
APPLICATIONINSIGHTS_SDKSTATS_DISABLED=true

Launch with dab start, or dab start --mcp-stdio anonymous for MCP stdio. Setting DAB_TELEMETRY_OPT_OUT=1 overrides enablement.

With a configured Books entity, GET /api/Books or this GraphQL query exercises eligible data usage:

{
  books {
    items { id title }
  }
}

This query must not establish data usage or first-success telemetry because its data selection is excluded:

{
  __typename
  books @skip(if: true) {
    items { id }
  }
}

Introduce isolated, default-off engine telemetry with explicit validation gates, bounded aggregation, and Azure Monitor export. Cover startup, configuration, request and transport completion with synthetic regression tests. Honor the umbrella opt-out and preserve customer telemetry independence. Keep internal design drafts local and ignored, outside the published commit history.

Use closed failure stages and attempt-scoped attribution for web, stdio, late configuration, validation, and hot reload. Preserve first-failure attribution, prior epochs, handler task semantics, and nested disabled-provider isolation. Add startup, recovery, concurrency, and compatibility regression coverage.

Copilot

Copilot review overview

🟡 Changes recommended

Unresolved critical credential-selection and moderate telemetry completion/classification issues remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

Classify duplicate role headers as unknown · New

What changed in this PR

“Add Engine Telemetry - Phase 1” introduces default-off, synthetic engine telemetry across REST, GraphQL, MCP, lifecycle events, aggregation, and Application Insights delivery.

Changes:

Adds bounded lifecycle, request, operation, cache, embedding, database, and HTTP telemetry.
Adds privacy safeguards, identity handling, opt-out controls, and extensive tests.
Review findings remain: one critical Startup issue (1 vote) and three moderate MCP/GraphQL issues (1, 2, and 2 votes).

File	Summary
`src/Service/Utilities/McpStdioHelper.cs`	Integrates MCP stdio lifecycle telemetry.
`src/Service/Telemetry/EngineTelemetrySdkTransportHandler.cs`	Guards SDK transport payloads and responses.
`src/Service/Telemetry/EngineTelemetryHttpMiddleware.cs`	Tracks REST request completion.
`src/Service/Telemetry/EngineTelemetryHttpCompletion.cs`	Coordinates HTTP completion outcomes.
`src/Service/Telemetry/EngineTelemetryHosting.cs`	Hosts telemetry and cache observers.
`src/Service/Telemetry/EngineTelemetryHealthProbe.cs`	Excludes internal health probes.
`src/Service/Telemetry/EngineTelemetryGraphQLListener.cs`	Tracks GraphQL execution and eligibility.
`src/Service/Telemetry/EngineTelemetryExportAttempt.cs`	Stores export-attempt state.
`src/Service/Telemetry/EngineTelemetryEmbeddingEndpointMetadata.cs`	Identifies embedding endpoints.
`src/Service/Telemetry/EngineTelemetryApplicationInsightsExporter.cs`	Implements guarded Application Insights export.
`src/Service/Telemetry/ApplicationInsightsTelemetryDestination.cs`	Validates telemetry destinations.
`src/Service/Telemetry/ApplicationInsightsEventAttributes.cs`	Maps events to SDK attributes.
`src/Service/Startup.cs`	Registers telemetry throughout the host pipeline.
`src/Service/Program.cs`	Adds telemetry bootstrap and shutdown handling.
`src/Service/Controllers/ConfigurationController.cs`	Reports configuration parsing failures.
`src/Service/Azure.DataApiBuilder.Service.csproj`	Adds the exporter dependency.
`src/Service.Tests/UnitTests/PostgreSqlQueryExecutorUnitTests.cs`	Makes credential tests deterministic.
`src/Service.Tests/UnitTests/McpStdioHelperTests.cs`	Tests stdio telemetry failures.
`src/Service.Tests/Telemetry/EngineTelemetryQueryExecutorTests.cs`	Tests database-attempt attribution.
`src/Service.Tests/Telemetry/EngineTelemetryContextTests.cs`	Tests runtime context privacy.
`src/Service.Tests/Telemetry/EngineTelemetryCloudTests.cs`	Adds opt-in cloud smoke coverage.
`src/Service.Tests/Telemetry/ApplicationInsightsTelemetryDestinationTests.cs`	Tests destination validation.
`src/Service.Tests/Telemetry/ApplicationInsightsEventAttributesTests.cs`	Tests payload attribute guards.
`src/Service.Tests/telemetry.runsettings`	Sets SDK statistics opt-outs for tests.
`src/Service.Tests/Azure.DataApiBuilder.Service.Tests.csproj`	Applies telemetry test settings.
`src/Directory.Packages.props`	Updates compatible package versions.
`src/Core/Telemetry/Product/IProductTelemetryControl.cs`	Defines host telemetry control.
`src/Core/Telemetry/Product/IEngineTelemetryExporter.cs`	Defines exporter abstraction.
`src/Core/Telemetry/Product/EngineTelemetryValueFormatter.cs`	Defines stable telemetry spellings.
`src/Core/Telemetry/Product/EngineTelemetryRequestScope.cs`	Tracks request state and completion.
`src/Core/Telemetry/Product/EngineTelemetryOptions.cs`	Defines bounded aggregation options.
`src/Core/Telemetry/Product/EngineTelemetryMeasurementScope.cs`	Tracks operation measurements.
`src/Core/Telemetry/Product/EngineTelemetryMeasurements.cs`	Defines telemetry dimensions and windows.
`src/Core/Telemetry/Product/EngineTelemetryEvent.cs`	Defines immutable event envelopes.
`src/Core/Telemetry/Product/EngineTelemetryDelivery.cs`	Provides bounded asynchronous delivery.
`src/Core/Telemetry/Product/EngineTelemetryContext.cs`	Captures categorical runtime context.
`src/Core/Telemetry/Product/EngineTelemetryCacheObserver.cs`	Observes cache-layer lookups.
`src/Core/Telemetry/Product/EngineTelemetryAggregator.cs`	Aggregates bounded usage windows.
`src/Core/Services/RestService.cs`	Instruments REST operations.
`src/Core/Services/ExecutionHelper.cs`	Instruments GraphQL operations.
`src/Core/Services/Embeddings/EmbeddingService.cs`	Instruments embedding calls.
`src/Core/Resolvers/QueryExecutor.cs`	Instruments database attempts.
`src/Core/Resolvers/CosmosClientProvider.cs`	Applies the product opt-out to Cosmos identity.
`src/Core/Configurations/RuntimeConfigValidator.cs`	Attributes validation failures.
`src/Core/Azure.DataApiBuilder.Core.csproj`	Grants telemetry internals access.
`src/Config/Telemetry/TelemetryFailureStage.cs`	Defines failure stages.
`src/Config/Telemetry/TelemetryFailureContext.cs`	Carries reload failure context.
`src/Config/Telemetry/TelemetryConfigurationPresence.cs`	Captures bounded configuration presence.
`src/Config/Telemetry/ProductTelemetryPolicy.cs`	Defines enablement and opt-out policy.
`src/Config/Telemetry/ApplicationNameTelemetry.cs`	Implements application-name opt-out behavior.
`src/Config/RuntimeConfigLoader.cs`	Captures configuration provenance.
`src/Config/Properties/AssemblyInfo.cs`	Adds telemetry friend assemblies.
`src/Config/ObjectModel/RuntimeConfig.cs`	Stores telemetry provenance metadata.
`src/Config/FileSystemRuntimeConfigLoader.cs`	Integrates reload telemetry attribution.
`src/Config/DabChangeToken.cs`	Preserves reload failure context.
`src/Azure.DataApiBuilder.Mcp/Utils/McpTelemetryHelper.cs`	Instruments MCP tool execution.
`src/Azure.DataApiBuilder.Mcp/Utils/McpProductResponseStream.cs`	Observes MCP response writes.
`src/Azure.DataApiBuilder.Mcp/Utils/McpProductResponseCompletion.cs`	Completes MCP responses at transport boundaries.
`src/Azure.DataApiBuilder.Mcp/Core/McpStdioServer.cs`	Delays stdio completion until response writing.
`src/Azure.DataApiBuilder.Mcp/Core/McpServerConfiguration.cs`	Registers MCP response filters.
`src/Azure.DataApiBuilder.Mcp/Core/McpEndpointRouteBuilderExtensions.cs`	Wraps MCP HTTP response streams.
`.gitignore`	Excludes local telemetry design drafts.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+            string? role = isStdio
+                ? configuration?.GetValue<string>("MCP:Role")
+                : httpContext?.Request.Headers[AuthorizationResolver.CLIENT_ROLE_HEADER].ToString();
+            EngineTelemetryRole roleClass = EngineTelemetrySession.ClassifyRole(
+                role, httpContext?.User.Identity?.IsAuthenticated == true);


aaronburtle added 3 commits September 24, 2026 19:25

Fix conditional GraphQL telemetry and hosting classification

e85f333

aaronburtle added this to the September 2026 milestone Sep 25, 2026

Copilot AI lite review requested due to automatic review settings September 25, 2026 09:10

aaronburtle added this to Data API builder Sep 25, 2026

aaronburtle requested review from Alekhya-Polavarapu, Aniruddh Munde (Aniruddh25), RubenCerna2079, cwebb-msft, Mohit G (gmohit21), mhyon, rusamant, Sourabh Jain (sourabh1007), Souvik Ghosh (souvikghosh04), Stuart Padley (stuartpa) and vadeveka as code owners September 25, 2026 09:10

Copilot started reviewing on behalf of aaronburtle September 25, 2026 09:11 View session

aaronburtle self-assigned this Sep 25, 2026

aaronburtle added telemetry feature requests/ bug reports related to telemetry assign-for-review labels Sep 25, 2026

github-actions Bot assigned Aniruddh Munde (Aniruddh25) and RubenCerna2079 Sep 25, 2026

github-actions Bot removed the assign-for-review label Sep 25, 2026

Copilot AI reviewed Sep 25, 2026

View reviewed changes

Merge main into engine telemetry and preserve initialization semantics

322512c

aaronburtle assigned Souvik Ghosh (souvikghosh04) and unassigned Aniruddh Munde (Aniruddh25) Sep 25, 2026

aaronburtle added 2 commits September 25, 2026 16:53

Fix telemetry test formatting

ec4b678

Fix telemetry host discovery and cross-platform CI regressions

df0d2bc

Navigation Menu

Add Engine Telemetry - Phase 1 - #3825

Add Engine Telemetry - Phase 1#3825
aaronburtle wants to merge 6 commits into
mainfrom
dev/aaronburtle/engine-telemetry-phase1

aaronburtle commented Sep 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

aaronburtle commented Sep 25, 2026

Why make this change?

What is this change?

How was this tested?

Sample Request(s)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants