Add EKS Node Diagnostics MCP server - #41
Conversation
Port the sample-eks-node-diagnostics-mcp project into mcp/aws-eks-node-diagnostics-mcp/, including CDK infrastructure, Lambda sources, diagnostic runbooks, CNI semantics, the packaged skill, and tests. The skill archive (skills/eks-node-diagnostics.zip) is force-added because the repo's root .gitignore (*.zip) would otherwise exclude it.
Drop skills/eks-node-diagnostics.zip and its README section per review feedback; the skill is not required for the MCP server to function.
…proval Remove the restricted tcpdump_capture/tcpdump_analyze tools and gate the two mutating tools (collect, batch_collect) behind an out-of-band human approval flow instead: - Approval Lambda + Function URL with capability-URL tokens (SHA-256 stored server-side, single-use, DynamoDB TTL expiry) - SNS topic notifies approvers with approve/deny links - batch_collect defaults to dry-run - Stricter EKS instance validation: user-settable kubernetes.io/cluster/* tag rejected unless ALLOW_SELF_MANAGED_NODES=true - ReDoS-safe search regex validation and log-key path checks - Update README, architecture docs, and runbooks accordingly - Add pytest coverage for the approval flow
Replace the custom DynamoDB + Lambda Function URL approval flow with SSM Automation wrapper documents whose first step is the native aws:approve action. Approvers are fixed IAM principals set at deploy time (APPROVAL_APPROVER_ARNS) and approve/deny in the Systems Manager console; decisions are IAM-authenticated and CloudTrail-audited. The agent polls status() and collection proceeds automatically on approval, so no approvalId re-call is needed.
…proval # Conflicts: # mcp/aws-eks-node-diagnostics-mcp/README.md # mcp/aws-eks-node-diagnostics-mcp/bin/app.ts # mcp/aws-eks-node-diagnostics-mcp/deploy.sh # mcp/aws-eks-node-diagnostics-mcp/src/lambda/ssm-automation-enhanced.py # mcp/aws-eks-node-diagnostics-mcp/src/ssm-automation-gateway-construct-v2.ts # mcp/aws-eks-node-diagnostics-mcp/src/ssm-automation-gateway-stack-v2.ts # mcp/aws-eks-node-diagnostics-mcp/tests/construct-iam.property.test.ts # mcp/aws-eks-node-diagnostics-mcp/tests/construct-kms-s3.property.test.ts # mcp/aws-eks-node-diagnostics-mcp/tests/test_collection_approval.py
There was a problem hiding this comment.
Nitpicking, not a blocker: this JSDoc block (lines 2267–2278) describes the old DynamoDB/Function-URL approval mechanism (approvalId, token, DynamoDB record, approvalViaPublicUrl) that this exact PR removes. Looks like it got orphaned above getUnzipLambdaCode() during the refactor to native aws:approve — the function body below it is correct and unrelated to approval logic, just the comment is stale. Suggest deleting the second JSDoc block so it doesn't mislead the next reader into thinking there's a DynamoDB-based approval path elsewhere in the file.
Address PR review comments 2 and 3: - Idempotent replay of collect() with a token whose execution is an approval wrapper now runs the same augment_wrapper_status enrichment as status(), so agents see humanApproval/consoleUrl instead of a bare InProgress while paused at aws:approve. The replay response also reports the execution's actual region. - Add a synth-time CDK warning when requireCollectionApproval is on and allowedRegions spans multiple regions, since the approval wrapper documents exist only in the stack's home region. - Add regression tests for both replay paths.
|
Addressed the review feedback in 44c60b8: Comment 2 (idempotent replay loses approval context) — fixed. Comment 3 (single-region approval gate) — warning added. Single-region approval-gated collection is intentional for now: the wrapper documents are regional SSM documents owned by the stack, and embedding deployment-owned approvers/role/topic per region means the right fix is a stack per region rather than cross-region document sharing. Per the suggestion, synthesis now emits a Comment 4 (IAM document scoping) — agreed, deferring as a follow-up. Two corrections worth noting for whoever picks it up: (1) the scoped document list must also include the tcpdump approval wrapper, otherwise approval-gated captures break when the restricted tools are enabled; (2) the Validation: 46 Python tests pass (including the 2 new regression tests) and the TypeScript build is clean. The pre-existing jest property-test failures (test-file parse errors and the missing |
Address the MCP security review (Way In / Way Out threat model): - M3: tcpdump_capture is now both opt-in (ENABLED_RESTRICTED_TOOLS) and gated by its own native SSM aws:approve wrapper document, so a poisoned agent cannot self-approve via the confirmCapture parameter. - E1: kubernetes.io/cluster/* tags are no longer trusted alone. Only EKS-managed eks:cluster-name/eks:nodegroup-name tags are accepted, verified against the EKS API; self-managed nodes require explicit ALLOW_SELF_MANAGED_NODES opt-in with EKS API cross-check. - E2: Lambda-level ALLOWED_CLUSTER_NAMES allowlist enforced on instance validation, cluster_health, and batch_collect. - E4: validate_log_key enforces bundle-shaped keys, blocks traversal, and scopes reads to the instance under investigation. - E5: catastrophic regex rejection plus SIGALRM wall-clock timeouts around search matching. - Pod-scoped capture resolves PID/network namespace immediately before nsenter and rejects host-netns or changed namespaces. - Runbooks updated for the approval-gated capture flow; tcpdump approval regression tests added.
…eks-mcp-collection-approval
Close the two residual gaps from the MCP security review: - E2: an empty ALLOWED_CLUSTER_NAMES no longer permits all clusters by default. The Lambda now requires the deploy-time any-cluster acknowledgment (allowAnyClusterName -> ALLOW_ANY_CLUSTER_NAME) to operate without an allowlist, matching the CDK synth-time contract. - E4: read() and artifact() now require instanceId, so every log read and presigned URL is scoped to the instance under investigation. Previously the cross-instance check only applied when the caller chose to pass instanceId. Tool schemas updated to mark instanceId required; regression tests added for both behaviors.
Description
Adds the AWS EKS Node Diagnostics MCP server (
mcp/aws-eks-node-diagnostics-mcp): a CDK-deployed MCP server that lets the DevOps Agent collect and analyze EKS worker node logs via SSM Automation (the AWS-managedAWSSupport-CollectEKSInstanceLogsdocument), with 19 tools across four tiers (core operations, log analysis, cluster-level diagnostics, and 41 structured SOP runbooks).Key design points:
collectandbatch_collect, run through wrapper SSM Automation documents whose first step is the nativeaws:approveaction. The execution pauses in SSM until a designated approver (IAM principals fixed at deploy time viaAPPROVAL_APPROVER_ARNS) approves in the Systems Manager console. Decisions are IAM-authenticated and CloudTrail-audited; the MCP Lambda has nossm:SendAutomationSignalpermission, so an agent cannot approve its own request. The agent just pollsstatus()and collection proceeds automatically on approval.Type of change
Testing
npm run build(tsc) — passesnpm test— 12/12 jest tests passed (fast-check property tests for construct IAM and KMS/S3 policies), 44/44 pytest tests passed (approval workflow, region validation, tool validation wiring)deploy.shand exercised the approval flow end to end:collectreturnspending_approvalwith an SSM console deep link, approval in the console resumes collection, andstatus()reports the approval state and collection progressLicense confirmation