Skip to content

feat(storage): add run_benchmark_tests.sh for automated time-based GC… - #18138

Draft
shradhakatyal wants to merge 4 commits into
mainfrom
add-gcs-read-benchmark-runner
Draft

feat(storage): add run_benchmark_tests.sh for automated time-based GC…#18138
shradhakatyal wants to merge 4 commits into
mainfrom
add-gcs-read-benchmark-runner

Conversation

@shradhakatyal

Copy link
Copy Markdown
  • Adds run_benchmark_tests.sh under packages/google-cloud-storage/cloudbuild/ to automate high-bandwidth GCS read microbenchmarks for large objects (10 GiB file sizes).
  • Dynamically updates config.yaml to test multi-process worker pools (PROCESSES=48) across standard REST and BidiReadObject DirectPath gRPC (zonal buckets).
  • Automatically verifies and pre-seeds required GCS test objects prior to benchmark execution to ensure consistent benchmark conditions.
  • Emits structured JSON metrics via --benchmark-json for automated ingestion into Cloud Build CI

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an automated Google Cloud Storage read microbenchmark runner script (run_benchmark_tests.sh) designed for GitHub CI/CD and high-bandwidth GCE VMs. The review feedback identifies two critical areas for improvement: first, a race condition and a NameError in the Python pre-seeding block where concurrent processes attempt to write to the same payload file and reference an undefined variable; second, a style guide violation where the configuration loaded from config.yaml is not type-validated before indexing.

Comment thread packages/google-cloud-storage/cloudbuild/run_benchmark_tests.sh
Comment thread packages/google-cloud-storage/cloudbuild/run_benchmark_tests.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant