Fix daily AIC snapshot completeness and quota handling - #60136
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"
- "proxy.golang.org"See Network Configuration for more information.
|
|
🧠 Matt Pocock Skills Reviewer failed during the skills-based review. Warning Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding. What happenedThe threat detection engine failed to produce results. Review the workflow run logs for details. Warning Firewall blocked 4 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "codeload.github.com"
- "github.com"
- "patchdiff.githubusercontent.com"
- "registry.npmjs.org"See Network Configuration for more information.
|
|
Warning Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding. What happenedThe threat detection engine failed to produce results. Review the workflow run logs for details.
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch PR file list
|
|
✅ Ponytail Reviewer completed successfully! Cannot run Ponytail review: /tmp/gh-aw/agent/pr-diff.patch is empty. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "ab.chatgpt.com"See Network Configuration for more information.
|
🏗️ ADR Required — draft added for PR #60136I required ADR enforcement because this PR adds more than 100 new lines in default business-logic directories ( Evidence used
Action taken
Next action for the author
Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"
- "proxy.golang.org"See Network Configuration for more information.
|
There was a problem hiding this comment.
🟡 Changes recommended
Outstanding quota, cache validation, workflow filtering, and evals accounting findings block approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Fixes daily AIC accounting with verified activation snapshots, strict quota handling, and complete evals usage collection.
Changes:
- Moves snapshot restoration and publication to activation.
- Adds strict component validation and quota-aware scans.
- Updates evals collection, documentation, tests, and workflow lockfiles.
File summaries
| File | Summary |
|---|---|
pkg/workflow/notify_comment.go |
Updates usage uploads and removes conclusion cache writes. |
pkg/workflow/notify_comment_test.go |
Tests notification and upload changes. |
pkg/workflow/notify_comment_conclusion_helpers.go |
Updates conclusion helpers. |
pkg/workflow/evals_steps.go |
Collects topology-aware evals token usage. |
pkg/workflow/evals_steps_test.go |
Tests evals usage collection. |
pkg/workflow/daily_aic_workflow_guardrail_test.go |
Adds daily AIC guardrail coverage. |
pkg/workflow/compiler_activation_daily_aic.go |
Generates activation restore and publication steps. |
docs/src/content/docs/reference/cost-management.md |
Documents snapshot and quota behavior. |
actions/setup/sh/collect_usage_artifact_files.sh |
Repacks usage artifacts. |
actions/setup/js/restore_aic_scan_cache.cjs |
Restores trusted activation snapshots. |
actions/setup/js/daily_aic_scan.cjs |
Scans and validates historical AIC usage. |
actions/setup/js/daily_aic_component_coverage.cjs |
Validates usage component coverage. |
actions/setup/js/daily_aic_cache_helpers.cjs |
Supports cache validation and reuse. |
actions/setup/js/daily_aic_api_budget.cjs |
Handles API budgets and quota state. |
actions/setup/js/artifact_client.cjs |
Observes artifact API quota responses. |
.github/workflows/workflow-normalizer.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/windows-grower.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/weekly-network-domains-audit.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/weekly-issue-summary.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/super-linter.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/step-name-alignment.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/static-analysis-report.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/spec-extractor.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/smoke-update-cross-repo-pr.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/smoke-pi.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/smoke-copilot-arm.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/smoke-codex.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/smoke-claude.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/slide-deck-maintainer.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/scout.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/schema-consistency-checker.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/safe-output-health.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/repository-quality-improver.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/refactoring-cadence.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/python-data-charts.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/prompt-clustering-analysis.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/portfolio-analyst.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/pdf-summary.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/outcome-collector.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/org-health-report.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/mcp-inspector.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/lockfile-stats.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/linter-miner.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/jsweep.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/instructions-janitor.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/go-fan.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/github-mcp-tools-report.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/github-mcp-structural-analysis.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/firewall-escape.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/feature-grower.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/dead-code-remover.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/daily-max-ai-credits-test.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/daily-formal-spec-verifier.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/daily-arxiv-researcher.lock.yml |
Regenerated workflow lockfile. |
.github/workflows/chaos-pr-bundle-fuzzer.lock.yml |
Regenerated workflow lockfile. |
.changeset/patch-complete-daily-aic-observations.md |
Adds the release note for complete observations. |
Review details
- Files reviewed: 101/321 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Fixed in |
|
✅ Smoke OTEL completed successfully! Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "ab.chatgpt.com"See Network Configuration for more information.
|
…st-scan-observations Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Addressed in |
|
@copilot The compact report still shows this PR needs follow-up. Please inspect any remaining blocking review or automation signals, refresh the branch if needed, and run the pr-finisher skill before handing this back to maintainers. Run: https://github.com/github/gh-aw/actions/runs/34576660531
|
…st-scan-observations # Conflicts: # .github/workflows/release.lock.yml Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Completed the PR-finisher pass. Run |
|
@copilot The compact report still shows this PR needs follow-up. Please inspect any remaining blocking review or automation signals, refresh the branch if needed, and run the pr-finisher skill before handing this back to maintainers. Run: https://github.com/github/gh-aw/actions/runs/34579367023
|
Completed the PR-finisher pass. Run |
|
@copilot These items still need follow-up on this PR:
Please resolve the remaining feedback, refresh the branch if needed, and run the Run: https://github.com/github/gh-aw/actions/runs/34581583531 Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"See Network Configuration for more information.
|
Completed the PR-finisher pass. The threshold thread is resolved, and I replied directly to both GitHub Actions smoke-test threads confirming that no code changes are required. Run |
|
@copilot The compact report still shows this PR needs follow-up. Please inspect any remaining blocking review or automation signals, refresh the branch if needed, and run the Run: https://github.com/github/gh-aw/actions/runs/34589658410
|
Completed the PR-finisher pass. Run |
|
@copilot resolve the merge conflicts on this branch. |
…st-scan-observations # Conflicts: # .github/workflows/release.lock.yml Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Resolved in |
Daily AIC accounting could reuse incomplete observations, repeatedly fetch historical artifacts, and report partial totals as under budget after quota failures. Conclusion jobs also appended to restored caches rather than preserving the activation scan.
Complete observations
Strict accounting
Quota handling
under_budget.Compiler and runtime
✨ PR Review Safe Output Test - Run 34562419664
Warning
Firewall blocked 2 domains
The following domains were blocked by the firewall during workflow execution:
clients2.google.commtalk.google.comTo allow these domains, add them to the
network.allowedlist in your workflow frontmatter:See Network Configuration for more information.