fix(cdk): pin VPC to AgentCore-supported availability zones#358
fix(cdk): pin VPC to AgentCore-supported availability zones#358AshrafBen10 wants to merge 9 commits into
Conversation
…zones (aws-samples#353) AgentCore only supports a subset of physical availability zones per region. AZ names are aliased per-account to physical zone IDs, so the default maxAzs selection can land in a zone AgentCore does not support, causing the AWS::BedrockAgentCore::Runtime resource to fail with NotStabilized. Changes: - Add optional `availabilityZones` prop to AgentVpcProps — when provided it takes precedence over maxAzs so the VPC is pinned to specific AZ names. - Wire up the CDK context key `agentcore:availabilityZones` in agent.ts so affected accounts can set it in cdk.context.json or via -c flag without touching construct code. - Add tests for the new prop (explicit AZs override maxAzs, 3-zone case). Usage for affected accounts: cdk deploy -c agentcore:availabilityZones='["us-east-1b","us-east-1c"]' Or in cdk.context.json: { "agentcore:availabilityZones": ["us-east-1b", "us-east-1c"] } Closes aws-samples#353
Review:
|
krokoko
left a comment
There was a problem hiding this comment.
Thanks for tackling this — the root cause analysis is spot on, and the approach (pinning AgentVpc via CDK context so the template gets literal AZ names instead of Fn::GetAZs) is a valid escape hatch. The CDK wiring (availabilityZones vs maxAzs) also looks correct.
I'm requesting changes because, as shipped, this doesn't yet meet the bar for how we should handle AgentCore's AZ constraint in this sample. AWS documents the requirement in terms of physical zone IDs (e.g. use1-az1 / use1-az2 / use1-az4 in us-east-1), and AZ names are aliased per account — see:
Issue #353 itself preferred auto-selecting supported zone IDs so a fresh deploy succeeds without account-specific guesswork. A manual name pin alone leaves the default path broken for affected accounts.
What to fix to make this right
-
Prefer auto-pin from AgentCore-supported zone IDs (default path)
At synth time (we already needec2:DescribeAvailabilityZones), resolve the account's name↔ID mapping, intersect with AgentCore's supported zone-ID set for the region, and pass those AZ names intoec2.Vpc({ availabilityZones }). Keepagentcore:availabilityZonesonly as an optional override. That matches AWS's model and closes the bug without per-account tribal knowledge. -
Validate the context override
Mirror the pattern inresolveBedrockModelIds(bedrock-models.ts): reject non-arrays / bare strings with a clear synth error naming the key and expected JSON shape. Prefer requiring ≥2 zones (AgentCore HA guidance). -
Document the operator path
Even if the escape hatch remains, add a short note to the deploy / quick-start / troubleshooting docs (discover mapping → pick supported IDs → set context). An override buried only in construct comments won't help someone mid-rollback. -
Harden tests
Assert subnetAvailabilityZoneproperties are the pinned names (not only subnet count), and add an env-agnostic case since productionAgentStacksynthesizes that way. -
Avoid hardcoding only
us-east-1IDs in comments as if universal
Supported sets differ by region and can change; prefer linking the AgentCore AZ table (and/or a small per-region constant map used by the auto-select logic).
Happy to re-review once the default path selects supported zone IDs (with the context override as a safety valve), validation, docs, and stronger tests are in place. Appreciate the careful write-up and the unblock for #353.
…#353) Addresses review on aws-samples#358: a manual AZ pin alone left the default deploy path broken for affected accounts. The stack now auto-selects AgentCore-supported availability zones, with the context override kept as a validated safety valve. - New constructs/agentcore-azs.ts: - AGENTCORE_SUPPORTED_AZ_IDS: per-region map of supported physical zone IDs (from the AWS AgentCore VPC docs), not a us-east-1-only comment presented as universal. - resolveAgentCoreAzOverride: validates agentcore:availabilityZones, mirroring resolveBedrockModelIds (rejects non-array / empty / non- string entries and <2 zones for HA) with a clear synth error. - selectSupportedAzNames: pure name<->id intersection. - resolveAgentCoreAzs: override wins; else, when synth has a concrete account+region, DescribeAvailabilityZones -> intersect -> pin AZ names; env-agnostic synth / unknown region / lookup failure fall back to CDK's default selection (surfaced via a synth warning, not a masked empty result). - main.ts is now async and threads the resolved names into AgentStack (new AgentStackProps.availabilityZones) -> AgentVpc. - Docs: DEPLOYMENT_GUIDE 'Known deployment issues' + QUICK_START note and troubleshooting row cover the discover-mapping -> pick-IDs -> set- context operator path. - Tests: agent-vpc asserts subnet AvailabilityZone == pinned names plus an env-agnostic Fn::GetAZs fallback; agentcore-azs covers the map, override validation, selection, and resolver branches. Adds @aws-sdk/client-ec2 for the synth-time lookup (loaded lazily).
|
@krokoko thanks for the detailed review — pushed 1. Auto-pin from supported zone IDs (default path). New One nuance worth flagging: this repo's pipeline pre-synthesizes env-agnostic in 2. Validate the override. 3. Docs. Added a "Known deployment issues" subsection in 4. Harden tests. 5. Per-region map, not us-east-1-only. Implementation note: added |
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #358 +/- ##
=======================================
Coverage ? 89.33%
=======================================
Files ? 225
Lines ? 54676
Branches ? 5212
=======================================
Hits ? 48847
Misses ? 5829
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
isadeks
left a comment
There was a problem hiding this comment.
Re-review of a7f04df — the auto-pin rework
The rework from a manual pin into synth-time auto-selection is a real improvement and maps cleanly onto krokoko's five CHANGES_REQUESTED points. I checked it out and ran it — agentcore-azs.test.ts + agent-vpc.test.ts 33/33, tsc --noEmit clean, and I verified the behaviors below by synth/unit probes rather than reading. It also resolves both nits from my earlier review (context validation + env-agnostic test). Nice work, and the loud-fail resolveAgentCoreAzOverride mirroring resolveBedrockModelIds is the right shape.
One finding is worth fixing before merge because it breaks the exact operator recovery path this feature exists for; the rest are minor.
1. The documented -c 'agentcore:availabilityZones=[...]' escape hatch throws (should-fix — it's the mid-rollback recovery path)
resolveAgentCoreAzOverride rejects anything that isn't Array.isArray(...). But CDK delivers a -c key=value context value as a raw string, not parsed JSON — so the CLI form throws:
$ cdk deploy -c 'agentcore:availabilityZones=["us-east-1b","us-east-1c"]'
Error: Context 'agentcore:availabilityZones' must be a JSON array of availability-zone names …;
got "[\"us-east-1b\",\"us-east-1c\"]".
Reproduced both ways: the cdk.context.json form (a real array) works; the -c CLI form throws. This matters because the PR description's "Usage" block leads with the -c form, and an operator hitting the AgentCore AZ rollback — the whole reason this feature exists — is the one most likely to reach for -c mid-firefight. It's the same crash I flagged in my June review (then .forEach is not a function); the new validator turns it into a clearer message but still doesn't accept the input the docs advertise.
Fix: when the context value is a string, JSON.parse it before the array check (fall back to treating a bare string as a single malformed entry so the error still fires for true typos). Then the -c and cdk.context.json forms behave identically. (resolveBedrockModelIds has the same structural limitation, but its docs don't advertise a -c form, so this PR is the one that creates the mismatch — worth fixing here and arguably worth a follow-up for the shared pattern.)
2. Auto-pin silently widens the VPC from 2 AZs to all supported zones (often 3) (document / confirm intent)
selectSupportedAzNames returns every matching zone, so on a typical us-east-1 account auto-pin yields 3 AZs where maxAzs: 2 previously gave 2 → 6 subnets instead of 4. I confirmed NAT gateways stay at 1 (the construct default), so there's no NAT/EIP cost blowup — just more subnets (free) and a wider ENI spread. Benign, but it's a silent topology change for accounts that deploy fine today, and it's not called out. Either cap the auto-pin at 2 (matching the old default and the MIN_AGENTCORE_AZS HA floor) or note in the docs that auto-pin uses all supported zones.
3. The CI-deployed stack still isn't auto-fixed — only local cdk deploy is (doc clarity)
Verified the deploy path: build.yml synthesizes cdk.out with persist-credentials: false and no aws-region, so the artifact is env-agnostic → resolveAgentCoreAzs returns undefined → the CI-deployed stack falls back to Fn::GetAZs/maxAzs, unpinned. deploy.yml then deploys that artifact (--app cdk/cdk.out). The DEPLOYMENT_GUIDE does say env-agnostic synth needs the override, so this is documented, not a bug — but the framing ("Default behavior (auto-pin)… No action needed") reads as if the common case is covered, when the CI/CD-deployed stack (the one most teams run) still needs the manual override. Worth making that contrast sharper so a team relying on deploy.yml doesn't assume they're protected.
Minor
- Override isn't cross-checked against
AGENTCORE_SUPPORTED_AZ_IDS— a typo'd but well-formed override (["us-east-1x","us-east-1y"]) is passed straight toec2.Vpc, deferring the failure to deploy time. Defensible (the override is a deliberate escape hatch, and names→IDs are account-specific so a name-level check can't validate zone-ID support), but a soft synth warning when an override's resolved IDs aren't in the supported set would catch fat-fingers earlier. Optional. defaultDescribeAzs(lines 153-169) is the one uncovered block — fine, since it's the injected live-call boundary and tests substitute it.
Verification
Ran at a7f04df: 33/33 tests, tsc clean. Reproduced: -c string form throws vs cdk.context.json array works (#1); auto-pin returns 3 zones / 6 subnets / 1 NAT (#2); env-agnostic synth returns undefined and build.yml synths credential-less (#3); well-formed-but-unsupported override passes unchecked (minor). Selection preserves the account's zone order (deterministic given a stable DescribeAZ response).
Recommendation: fix #1 before merge (small change, unblocks the documented recovery path); #2/#3 are doc/clarity items; the rest optional. Approach and structure are sound.
…samples#353) Follow-up to the auto-pin review on aws-samples#358. - resolveAgentCoreAzOverride now JSON-parses a string context value, so the documented '-c agentcore:availabilityZones=[...]' recovery path works identically to the cdk.context.json array form (CDK delivers -c values as raw strings). A non-JSON string is left as-is and still fails the array check with the same clear, key-named error, so true typos error out. - Auto-pin now pins the first MIN_AGENTCORE_AZS (2) supported zones instead of every match, matching AgentVpc's default maxAzs so enabling auto-pin no longer silently widens a working 2-AZ account to all supported zones (e.g. 3 AZs / 6 subnets). - DEPLOYMENT_GUIDE: sharpened the contrast that auto-pin is local-deploy-only and the CI/CD (deploy.yml) artifact is env-agnostic and must set the override; added an explicit -c example now that it parses. - Tests: added JSON-string override (success) and JSON-non-array (throws) cases; updated the auto-pin test to assert the 2-zone cap.
|
Thanks for the thorough re-review and for actually running it — pushed #1 (should-fix) — #2 — auto-pin no longer widens topology. Capped auto-pin to the first #3 — sharpened the CI/CD doc contrast. Reworked the deployment-guide section: the "auto-pin" heading now reads "local deploy only," and there's a dedicated "The CI/CD-deployed stack is NOT auto-pinned" paragraph spelling out that Minors:
Verification on |
isadeks
left a comment
There was a problem hiding this comment.
Re-review of 2891801 — all findings addressed, approving
Checked out and ran it. All three findings from the last review are fixed and verified empirically (not just from the reply):
#1 — -c JSON-string override now works. resolveAgentCoreAzOverride JSON.parses a string context value before the array check. Verified at both layers:
- Unit:
-cstring form →["us-east-1b","us-east-1c"]; array form unchanged; and the error paths all still fire — a bare-string typo, a JSON-but-non-array ('"us-east-1b"'), and a single-element array below the HA floor all throw the clear key-named error. So the recovery path works and typos still error. - Integration: the exact
resolveAgentCoreAzscallmain.tsmakes, fed the raw string CDK delivers for-c, now returns the array instead of throwing at synth entry (the crash from my last review is gone).
#2 — auto-pin no longer widens topology. names.slice(0, MIN_AGENTCORE_AZS) caps it. Verified: 3 supported zones → pins exactly 2 (["us-east-1a","us-east-1b"]), matching AgentVpc's default maxAzs. An account that deploys fine today stays at 2 AZs / 4 subnets.
#3 — CI/CD contrast sharpened. The doc now heads the default path "auto-pin — local deploy only" and adds a dedicated "The CI/CD-deployed stack is NOT auto-pinned" paragraph spelling out that build.yml synths credential-less and deploy.yml ships that artifact, so pipeline teams must set the override. Also documents the "first two" cap and adds the -c example. Reads clearly now.
Minors — both correctly dispositioned:
- Override-vs-supported-ID cross-check left out: agreed and well-reasoned — the override exists precisely for env-agnostic/pipeline deploys where there's no bound account to resolve zone IDs, so a live cross-check would break the offline use case, and a name-level check can't validate zone-ID support anyway. Deploy-time remains the failure point for a well-formed-but-wrong override, which is acceptable for an escape hatch.
defaultDescribeAzscoverage: fine, it's the injected live-call boundary.
Verification on 2891801: 35 AZ-module + agent-vpc tests green, tsc --noEmit clean. The rework from a manual pin into auto-select-with-override is a solid outcome for #353 and matches AWS's zone-ID model.
LGTM — approving. (Optional follow-up, non-blocking: resolveBedrockModelIds shares the same -c-string-vs-array structural limitation; worth the same JSON.parse treatment if a -c form ever gets documented for it.)
Summary
Adds an
availabilityZonesprop toAgentVpcand wires a CDK context key (agentcore:availabilityZones) so affected accounts can pin the VPC subnets to AZ names that map to AgentCore-supported physical zone IDs.Problem
AgentCore only supports a subset of physical AZs per region (for
us-east-1:use1-az1,use1-az2,use1-az4). AZ names are aliased per-account, so the defaultmaxAzs: 2selection can land in an unsupported zone, causing theAWS::BedrockAgentCore::Runtimeresource to fail withNotStabilizedand rolling back the entire stack.Changes
cdk/src/constructs/agent-vpc.ts— Added optionalavailabilityZonesprop that takes precedence overmaxAzswhen provided.cdk/src/stacks/agent.ts— Readsagentcore:availabilityZonesfrom CDK context and passes toAgentVpc.cdk/test/constructs/agent-vpc.test.ts— Added tests for the new prop (explicit AZs override maxAzs, 3-zone case).Usage for affected accounts
Or in
cdk.context.json:{ "agentcore:availabilityZones": ["us-east-1b", "us-east-1c"] }Testing
Closes #353