Skip to content

create_gcs_eval_managers_from_uri doesn't extract bucket from path, contradicting its own docstring #6881

Description

@itsatefe

Description

create_gcs_eval_managers_from_uri() in [src/google/adk/cli/utils/evals.py] has a docstring that says:

eval_storage_uri: The evals storage URI to use. Supported URIs: gs://<bucket name>. If a path is provided, the bucket will be extracted.

But the implementation never extracts a bucket from a path; it treats everything after gs:// as the literal bucket name:

if eval_storage_uri.startswith('gs://'):
    gcs_bucket = eval_storage_uri.split('://')[1]

'gs://my-bucket/some/path'.split('://')[1] returns 'my-bucket/some/path', slashes included, not 'my-bucket'.

Repro

from google.adk.cli.utils.evals import create_gcs_eval_managers_from_uri

create_gcs_eval_managers_from_uri("gs://my-bucket/some/path")

Expected behavior

Per the docstring, this should extract my-bucket as the bucket name and (presumably) use some/path as a prefix/subdirectory for eval storage.

Actual behavior

gcs_bucket ends up as the literal string "my-bucket/some/path". This is passed straight into both GcsEvalSetsManager and GcsEvalSetResultsManager:

self.bucket = self.storage_client.bucket(self.bucket_name)
if not self.bucket.exists():
    raise ValueError(f"Bucket `{self.bucket_name}` does not exist...")

storage_client.bucket(name) doesn't validate the name locally, so this only fails later at .exists(), which 404s because GCS bucket names can never contain /. The resulting error —

ValueError: Bucket `my-bucket/some/path` does not exist. Please create it before using the GcsEvalSetsManager.

— is misleading: the real bucket (my-bucket) may well exist; the tool just built an invalid bucket name from the path segment.

Additional note

Even if bucket extraction were implemented, the path portion currently has nowhere to go — _get_eval_history_dir() / _get_eval_sets_dir() hard-code {app_name}/evals/eval_history and {app_name}/evals/eval_sets respectively, with no support for a custom prefix within the bucket.

Environment

  • google-adk version: 2.0.0 (confirmed still present on main as of 2026-08-24)
  • Reproduced in both GcsEvalSetsManager and GcsEvalSetResultsManager

Suggested fix

  • Implement the extraction the docstring promises (e.g. eval_storage_uri.split('://')[1].split('/')[0] for the bucket, with the remainder threaded through as a storage prefix), or

Metadata

Metadata

Assignees

Labels

eval[Component] This issue is related to evaluation

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions