Enforce AWF domain policy on Claude hosted web search and fetch - #8929
Conversation
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Domain hierarchy handling, schema/runtime consistency, and request-array scalability need correction before approval.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 3
Open (3)
What changed in this PR
Adds Claude hosted web policy enforcement to the trusted API-proxy sidecar, closing provider-hosted egress gaps invisible to Squid.
Changes:
- Adds config schema, validation, normalization, and sidecar serialization.
- Enforces allowlists, blocklists, disabling, and usage limits.
- Adds documentation and comprehensive host/sidecar tests.
| File | Description |
|---|---|
src/types/api-proxy-diagnostics-options.ts |
Adds runtime policy option. |
src/services/api-proxy-env-config.ts |
Serializes normalized policy. |
src/config-mapper.ts |
Maps file configuration. |
src/config-file.ts |
Types the config structure. |
src/commands/build-config.ts |
Builds the runtime option. |
src/claude-hosted-web-policy.ts |
Validates and normalizes policies. |
src/claude-hosted-web-policy.test.ts |
Tests host-side configuration. |
src/awf-config-schema.json |
Adds runtime schema definitions. |
docs/awf-config.schema.json |
Adds canonical public schema. |
docs/awf-config-spec.md |
Documents the normative contract. |
docs/api-proxy-sidecar.md |
Documents sidecar enforcement. |
containers/api-proxy/providers/anthropic.js |
Installs the enforcement transform. |
containers/api-proxy/Dockerfile |
Includes the new module. |
containers/api-proxy/claude-hosted-web.test.js |
Tests policy behavior. |
containers/api-proxy/claude-hosted-web.server.test.js |
Tests request-pipeline enforcement. |
containers/api-proxy/claude-hosted-web.js |
Implements sidecar enforcement. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
✅ Copilot review passed with no inline comments. @copilot Add the |
Documentation PreviewDocumentation has been built for this PR. To view locally:
Built from commit 7674882 |
Updated the schema for provider allowlist and blocked domains. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Refactor domain filtering logic to handle subdomain matching. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅ Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
[!TIP] tools:
github:
mode: gh-proxySee GitHub Tools for more information on To allow these domains, add them to the network:
allowed:
- defaults
- "api.github.com"
- "example.com"See Network Configuration for more information.
|
|
Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed. Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"
- "github.com"See Network Configuration for more information.
|
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤
|
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅ Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "o205451.ingest.us.sentry.io"See Network Configuration for more information.
|
|
✅ Build Test Suite completed successfully! Warning Firewall blocked 8 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.nuget.org"
- "bun.sh"
- "dc.services.visualstudio.com"
- "deno.land"
- "dl.deno.land"
- "github.com"
- "releaseassets.githubusercontent.com"
- "repo.maven.apache.org"See Network Configuration for more information.
|
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓
|
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 Warning Firewall blocked 13 domainsThe following domains were blocked by the firewall during workflow execution:
[!TIP] tools:
github:
mode: gh-proxySee GitHub Tools for more information on To allow these domains, add them to the network:
allowed:
- defaults
- "ab.chatgpt.com"
- "accounts.google.com"
- "android.clients.google.com"
- "api.github.com"
- "clients2.google.com"
- "collector.github.com"
- "contentautofill.googleapis.com"
- "github.com"
- "github.githubassets.com"
- "msfeed25.pkgs.visualstudio.com"
- "update.googleapis.com"
- "www.google.com"
- "www.gstatic.com"See Network Configuration for more information.
|
|
✅ Smoke Claude passed Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
|
❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...
|
|
❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...
|
|
✅ Security Guard completed successfully! Security review complete: PR #8929 introduces Claude hosted web policy enforcement in the API proxy sidecar. No security weakening detected. The implementation uses fail-closed design, robust input validation (label-by-label DNS checking, strict version matching), and correct transform ordering. This is a security-strengthening change that closes a semantic gap where Anthropic-hosted tools could bypass configured domain policies.
|
Smoke Test: Cloud Hypervisor + Copilot
Overall: PASS (4/4) Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"
- "github.com"See Network Configuration for more information.
|
Smoke Test: Claude Engine Validation
Overall result: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
|
EGRESS_RESULT allow=pass deny=pass ✅ Allowed domain (api.github.com) reachable — Overall status: PASS cc Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
[!TIP] tools:
github:
mode: gh-proxySee GitHub Tools for more information on To allow these domains, add them to the network:
allowed:
- defaults
- "api.github.com"
- "example.com"See Network Configuration for more information.
|
github-actions
Bot
added
smoke-copilot-network-isolation
|
Smoke Test: Copilot Engine — PR #8929 "Enforce AWF domain policy on Claude hosted web search and fetch"
Overall: PASS cc
|
Smoke Test: Copilot BYOK (Direct) Mode — ✅ PASS
Running in direct BYOK mode (
|
Smoke Test: API Proxy OpenTelemetry Tracing — Results
Overall: ✅ Pass — All implemented functionality validated; Scenario 5's lack of exported spans is expected given no OTLP collector endpoint was configured for this run, not a regression. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "o205451.ingest.us.sentry.io"See Network Configuration for more information.
|
|
Smoke Test: Services Connectivity
Overall: PASS
|
Smoke Test: Gemini Engine Validation
Overall Status: PASS Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "github.com"
- "play.googleapis.com"See Network Configuration for more information.
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — PASS All 18 test projects across all 8 ecosystems (Bun, C++, Deno, .NET, Go, Java, Node.js, Rust) built and tested successfully through the AWF firewall network. Note: Java Maven builds required overriding Warning Firewall blocked 8 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.nuget.org"
- "bun.sh"
- "dc.services.visualstudio.com"
- "deno.land"
- "dl.deno.land"
- "github.com"
- "releaseassets.githubusercontent.com"
- "repo.maven.apache.org"See Network Configuration for more information.
|
Chroot Version Comparison
Result: ❌ Not all versions matched — Node.js version differs between host and chroot environment (
|
|
Merged PRs: "Document explicit upstream proxy port fix in runner-doctor B2 catalog"; "Register nvx as an opt-in preview runtime" Warning Firewall blocked 13 domainsThe following domains were blocked by the firewall during workflow execution:
[!TIP] tools:
github:
mode: gh-proxySee GitHub Tools for more information on To allow these domains, add them to the network:
allowed:
- defaults
- "ab.chatgpt.com"
- "accounts.google.com"
- "android.clients.google.com"
- "api.github.com"
- "clients2.google.com"
- "collector.github.com"
- "contentautofill.googleapis.com"
- "github.com"
- "github.githubassets.com"
- "msfeed25.pkgs.visualstudio.com"
- "update.googleapis.com"
- "www.google.com"
- "www.gstatic.com"See Network Configuration for more information.
|
Anthropic's hosted
web_search_*/web_fetch_*server tools execute on Anthropic infrastructure, so Squid only ever seesapi.anthropic.com— the searched or fetched destination is invisible to the domain ACL and to access logs, and the retrieved content returns inside an already-allowed API response. This adds an AWF-owned domain policy that the trusted api-proxy sidecar injects and enforces on every matching hosted tool before the request is dispatched upstream.Claude-only by design; no Codex/OpenAI enforcement here.
Config contract
apiProxy.hostedWeb.claudeindocs/awf-config.schema.json(canonical) and the regeneratedsrc/awf-config-schema.json. Schema itself enforces thatenabled: truerequires exactly one ofallowedDomains/blockedDomains, and thatenabled: falsecarries neither — there is no implicit unrestricted success state.AwfFileConfig→config-mapper→build-config→ api-proxy service options.src/claude-hosted-web-policy.tsnormalizes and validates domains (lowercase DNS hostnames only; no scheme, path, port, credentials, IP, CIDR, wildcard, or localhost; dedup after normalization) and throws with a source-labelled message, so file and--config -(JSON or YAML) both fail before any container starts.src/services/api-proxy-env-config.tsserializes the normalized policy intoAWF_CLAUDE_HOSTED_WEB_POLICY. It is derived solely from validated config and is never merged from agent-controlled env.Sidecar enforcement
containers/api-proxy/claude-hosted-web.js— validatedweb_(search|fetch)_YYYYMMDDmatcher (with month/day range checks) rather than an exact-name list. Aweb_search_*/web_fetch_*candidate that isn't a valid version is rejected, not forwarded unprotected.min(configured, requested)formax_uses); they cannot broaden, replace, or remove it. Cross-mode filters, requests carrying both filters, empty intersections, and malformed domains/tools/max_usesare rejected with stable error codes before upstream dispatch. Errors carry no prompts, queries, URLs, or bodies.providers/anthropic.jsparses the policy at adapter construction — an invalid serialized policy fails sidecar startup, not the first request — and composes the transform last, so model rewriting and prompt-cache transforms cannot re-expand or drop it. Added to theDockerfileCOPY list.Docs
docs/awf-config-spec.md§9.8: normative section with config-source and request-level precedence tables, stdin examples, error table, and an explicit warning that omitting the object preserves pass-through and does not constrain Claude-hosted egress.docs/api-proxy-sidecar.md: why Squid cannot observe provider-hosted destinations, the sidecar trust boundary, and the new env-var row.Tests
42 host-side (schema valid/invalid matrix, schema-copy equality, JSON/YAML × file/stdin parity, mapping, normalization, env serialization, and that internal sensitive domains never reach the sidecar policy) and 71 container-side covering the full precedence and failure matrix, including future date-versioned tool names, multiple tools per request, and end-to-end assertions of the exact upstream JSON body observed by a mock Anthropic endpoint.
One note for reviewers: the issue refers to
network.sensitiveAllowDomains, which doesn't exist in this repo — the equivalent is the internalsensitiveAllowedDomains(spec §9.7), and that's what the non-disclosure test asserts against.