docs: document the experimental adk.experimental.* metrics - #2175
Conversation
| They are off by default. To turn them on, set the environment variable: | ||
|
|
||
| ```bash | ||
| export ADK_EXPERIMENTAL_TELEMETRY=true |
There was a problem hiding this comment.
Users should be able to clearly understand how to turn this on for their use case. May it make sense adding some example of how to enable this in code as well?
There was a problem hiding this comment.
Good call. Added a snippet for the per-request path right under the env var. It takes precedence over the environment variable, which is usually the reason you'd reach for it:
from google.adk.agents.run_config import RunConfig
from google.adk.telemetry import TelemetryConfig
run_config = RunConfig(
telemetry=TelemetryConfig(adk_experimental_telemetry_opt_in=True)
)| | **`adk.experimental.invoke_workflow.inference_calls`** | Histogram (count) | The number of inference (model) calls made across one workflow invocation. | `adk.experimental.root_agent.name`, `gen_ai.workflow.name`, `gen_ai.workflow.nested` (nested workflows only) | | ||
| | **`adk.experimental.invoke_workflow.tool_calls`** | Histogram (count) | The number of tool calls made across one workflow invocation. | `adk.experimental.root_agent.name`, `gen_ai.workflow.name`, `gen_ai.workflow.nested` (nested workflows only) | | ||
|
|
||
| !!! warning |
There was a problem hiding this comment.
I checked the rendered version - looks like a regular text, do we want to keep it?
There was a problem hiding this comment.
Worth flagging, since this one catches people out. GitHub's markdown view doesn't support the !!! admonition syntax, so the block flattens into a plain paragraph there. MkDocs does render it as a callout, same as the warnings already in logging.md, which you can see live at https://google.github.io/adk-docs/observability/logging/.
| metric across all datapoints double counts. | ||
|
|
||
| `gen_ai.workflow.nested` is set only on nested workflows, so excluding it leaves the outermost workflow alone, and that datapoint covers the whole turn. The workflow metrics carry no agent dimension, since a value spanning a whole workflow cannot be attributed to a single agent. They carry two names instead: `gen_ai.workflow.name` joins to `gen_ai.invoke_workflow.duration`, while `adk.experimental.root_agent.name` identifies the app, and the two disagree when a turn enters at a sub-agent. | ||
|
|
There was a problem hiding this comment.
general preference, probably not for now:
it's not always easy to understand relationships between invocations / workflows / etc. it would be great to be able to see what telemetry produced for 1-2 typical examples.
There was a problem hiding this comment.
Fair point. It belongs in a follow-up PR rather than this section.
Adds an Experimental metrics section under the metrics schema covering the fourteen adk.experimental.* histograms: per-invocation and per-workflow token spend, plus the per-workflow inference and tool call counts. These roll token spend up from the per-model-call grain of gen_ai.client.* to a whole agent invocation or workflow, so a reader can cost one turn without summing model calls. They are off by default; the section states the opt-in (ADK_EXPERIMENTAL_TELEMETRY, or RunConfig.telemetry per request), that the names and meaning may change, and that nested workflow datapoints are folded into their parent so a naive sum double counts. The experimental opt-in alone is not enough for the eight invoke_workflow rows. They are recorded only under telemetry schema v2, which defaults on for Agent Engine and off everywhere else, so a reader who exports only ADK_EXPERIMENTAL_TELEMETRY locally would find those rows empty with nothing on the page to explain it. The section now states the extra ADK_TELEMETRY_SCHEMA_VERSION_OPT_IN=2 requirement, scoped so it does not overstate: invoke_agent rows are unaffected, and a Workflow-engine app records per-node datapoints under either version.
7137f58 to
f65420e
Compare
2 participants
Adds an Experimental metrics section under the metrics schema covering the fourteen adk.experimental.* histograms: per-invocation and per-workflow token spend, plus the per-workflow inference and tool call counts.
These roll token spend up from the per-model-call grain of gen_ai.client.* to a whole agent invocation or workflow, so a reader can cost one turn without summing model calls. They are off by default; the section states the opt-in (ADK_EXPERIMENTAL_TELEMETRY, or RunConfig.telemetry per request), that the names and meaning may change, and that nested workflow datapoints are folded into their parent so a naive sum double counts.
The experimental opt-in alone is not enough for the eight invoke_workflow rows. They are recorded only under telemetry schema v2, which defaults on for Agent Engine and off everywhere else, so a reader who exports only ADK_EXPERIMENTAL_TELEMETRY locally would find those rows empty with nothing on the page to explain it. The section now states the extra ADK_TELEMETRY_SCHEMA_VERSION_OPT_IN=2 requirement, scoped so it does not overstate: invoke_agent rows are unaffected, and a Workflow-engine app records per-node datapoints under either version.