Skip to content

feat(auth): add JWT verification and API Gateway authorization - #8469

Open
bfreiberg wants to merge 3 commits into
aws-powertools:developfrom
bfreiberg:feat/auth-rfc-8466
Open

bfreiberg wants to merge 3 commits into
aws-powertools:developfrom
bfreiberg:feat/auth-rfc-8466

Conversation

@bfreiberg

@bfreiberg bfreiberg commented Sep 16, 2026

Copy link
Copy Markdown

Applications that need a custom JWT verifier currently assemble signing-key refresh, token-profile validation, and Lambda authorization behavior themselves. This adds an optional Auth utility with a shared verifier for direct use, Event Handler middleware, and API Gateway authorizers.

This PR covers inbound JWT verification. OAuth client credentials, its documentation, tests, and outbound example have been removed for a separate follow-up; that implementation is preserved on bfreiberg:follow-up/oauth-client-8466.

Issue number: #8466 — inbound JWT portion of the RFC.

Summary

Changes

  • Verify asymmetric signatures, exact issuer, resource audience, and required expiration. Add claim-presence requirements and exact expected_claims / expected_headers values to enforce provider-specific token purpose after baseline verification.
  • Support static JWKS, OIDC discovery, coordinated refresh, maximum key age, unknown-key cooldown, and failure backoff. Include resource-bound Cognito access tokens and explicitly configured issuer groups.
  • Protect Event Handler routes and produce REST/HTTP API authorizer responses. Keep scope checks, exception-safe claims cleanup, opt-in scalar context, and IAM policies restricted to the current request.
  • Expose eight fixed AuthFailureReason string-enum values and retryability. Middleware callbacks receive them through AuthErrorContext; authorizers have an observation callback for rejected tokens and unavailable keys. Default responses remain generic, and the utility performs no automatic logging.
  • Keep construction free of network I/O. Document the first-invocation latency versus cold-start failure tradeoff of explicit prefetch, and the different outage responses from middleware and API Gateway authorizers.
  • Retain the declared urllib3 dependency in the Powertools Layer and end-to-end Layer builder. Document cryptography's architecture requirements.
  • Move SAM infrastructure to examples/auth/templates/sam.yaml and build authorizers with the Auth extra separately from base-only backend artifacts. Disable authorizer-result caching in both Gateway examples.
  • Include functional/TLS tests, API documentation, a testing helper, and an MCP SDK adapter that records sanitized JWKS availability failures.

User experience

from aws_lambda_powertools.utilities.auth import JWTVerifier

verifier = JWTVerifier(
    issuer="https://idp.example.com/",
    audience="https://orders.example.com",
    algorithms=["RS256"],
    required_claims=["sub"],
    expected_claims={"token_use": "access"},  # Adapt to the provider's profile.
)

@app.get("/orders", middlewares=[verifier.require(scopes=["orders:read"])])
def orders():
    return {"subject": app.context["claims"]["sub"]}

Applications can also call verify() directly or return authorize() from a Lambda authorizer. Error callbacks let the Lambda owner record fixed reasons and retryability without logging credentials. Invalid credentials still deny access; unavailable JWKS still fails an authorizer invocation even when a callback is configured.

Validation

Check Result
Full non-performance regression suite, excluding repository AWS end-to-end tests 2,866 passed; 4 existing skips; 96.81% local package coverage
Auth suite 296 passed on Python 3.10, 3.12, and 3.14; 284 functional tests, 12 TLS cases, and 99.48% local Auth coverage
Existing performance suite 10 passed
Dependency isolation Auth-extra: 284 passed; base-only: 969 passed, 1 existing skip
Static analysis Ruff formatting/lint, mypy, ty, Bandit baseline, and Xenon passed
Documentation and packaging MkDocs build, Markdownlint, cfn-lint, lock consistency, wheel/sdist builds, diff checks, and Gitleaks passed
Lambda Layer compatibility 90 deployed checks across Python 3.10–3.14, x86_64 and arm64, in us-east-1
Revised SAM example 8 deployed valid/invalid token checks plus 2 provider-outage checks against REST and HTTP APIs
MCP example SDK 2.2.0 initialization, verified-claim mapping, token-purpose/signature denial, and sanitized JWKS outage logging passed

The Layer matrix used urllib3 2.8.0 from the Layer with runtime boto3/botocore 1.42.97. Checks confirmed dependency constraints, module locations, a real SDK HTTPS request, remote discovery/JWKS verification, invalid-token rejection, and authorizer error visibility. The SAM checks used separate authorizer/backend artifacts on Python 3.12 x86_64; both APIs returned HTTP 500 during the controlled JWKS outage. Test infrastructure and signing material were removed afterward.

Functional tests use real signatures and in-memory providers; TLS tests use a loopback server with both closing and persistent connections. Deployment checks use a controlled HTTPS provider. They complement the repository tests and do not claim live Cognito/Keycloak interoperability coverage. Coverage thresholds and exclusions are unchanged.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

@bfreiberg
bfreiberg requested a review from a team as a code owner September 16, 2026 19:16
@bfreiberg
bfreiberg requested a review from svozza September 16, 2026 19:17
@boring-cyborg

boring-cyborg Bot commented Sep 16, 2026

Copy link
Copy Markdown

Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

@boring-cyborg boring-cyborg Bot added dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation tests labels Sep 16, 2026
@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Sep 16, 2026
if not isinstance(token, str) or not token:
raise InvalidTokenError()
try:
header = jwt.get_unverified_header(token)
try:
# This payload selects a configured verifier. No unverified claim
# is returned to callers or used to discover another provider.
payload = jwt.decode(token, options={"verify_signature": False})
@bfreiberg

Copy link
Copy Markdown
Author

I reviewed the two SonarCloud findings in verifier.py. They appear to flag intentional parsing before verification:

  • Line 257 — get_unverified_header(): reads the header to select a permitted algorithm and signing key. JWTVerifier.verify() then verifies the signature and validates the claims before returning them.
  • Line 308 — verify_signature=False: reads iss solely to select an explicitly configured verifier. Unknown issuers are rejected without network requests. The selected verifier performs full verification; the unverified payload is never
    returned to callers.

I reran the verifier and profile tests: 63 passed, including rejection of tampered signatures, tokens signed with another issuer’s key, and unknown issuers.

Could you review these as potential false positives in SonarCloud? The issuer-routing code already documents this behavior; I can add a similar explanation beside the header parsing.

@codecov

codecov Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.50000% with 54 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.46%. Comparing base (ceeb0c1) to head (56a6604).

Files with missing lines Patch % Lines
aws_lambda_powertools/utilities/auth/oauth2.py 90.00% 8 Missing and 8 partials ⚠️
aws_lambda_powertools/utilities/auth/verifier.py 87.20% 9 Missing and 7 partials ⚠️
...ws_lambda_powertools/utilities/auth/_authorizer.py 90.47% 3 Missing and 3 partials ⚠️
...lambda_powertools/utilities/auth/_authorization.py 92.72% 2 Missing and 2 partials ⚠️
aws_lambda_powertools/utilities/auth/_http.py 91.30% 2 Missing and 2 partials ⚠️
...ws_lambda_powertools/utilities/auth/_validation.py 87.87% 4 Missing ⚠️
aws_lambda_powertools/utilities/auth/__init__.py 76.92% 2 Missing and 1 partial ⚠️
aws_lambda_powertools/utilities/auth/_jwks.py 99.07% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #8469      +/-   ##
===========================================
- Coverage    96.66%   96.46%   -0.20%     
===========================================
  Files          296      310      +14     
  Lines        14911    15631     +720     
  Branches      1268     1372     +104     
===========================================
+ Hits         14413    15079     +666     
- Misses         363      394      +31     
- Partials       135      158      +23     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work here and for updating the RFC. I went through the new version again, and the changes around JWKS freshness, Cognito access tokens, expiration, API Gateway caching, async usage, and PyJWT address my earlier concerns.

I would like to keep this PR focused on inbound JWT verification. The OAuth client makes the change much larger and I still want to review that API separately. Please move OAuth2Client, its tests, documentation, and outbound example to a follow-up PR.

The JWT part is close. The inline comments cover the remaining points around error visibility, the generic token profile, the Layer dependency, and the Lambda examples.

from aws_lambda_powertools.utilities.auth.oauth2 import OAuth2Client as OAuth2Client
from aws_lambda_powertools.utilities.auth.verifier import JWTVerifier as JWTVerifier

__all__ = ["JWTVerifier", "OAuth2Client"]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the scope above, please remove OAuth2Client from this PR together with its implementation, tests, documentation, and outbound example. We can review the complete OAuth experience in a separate PR.

Comment thread aws_lambda_powertools/utilities/auth/exceptions.py
Comment thread aws_lambda_powertools/utilities/auth/_middleware.py
enforce_scopes(claims, self._scopes)
if self._authorize is not None and self._authorize(claims) is not True:
raise ForbiddenError()
except AuthError as error:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fail-closed behavior is correct: the protected handler does not run. The missing part is visibility for the Lambda owner.

Please map the safe reason into AuthErrorContext so customers can log or emit metrics from on_error. I would not add automatic logs for invalid tokens because that can create noise and log flooding.

if require_principal:
_validate_principal(candidate)
return candidate
except (InvalidTokenError, ForbiddenError):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This behavior is correct: invalid credentials return Deny, while JWKSFetchError escapes and fails the authorizer invocation.

The authorizer should have the same safe failure visibility as the middleware. Could we provide a callback for recording the reason, or document a supported wrapper pattern? Otherwise rejected tokens disappear into Deny, and JWKS failures are only visible as uncaught Lambda errors.

Comment thread docs/utilities/auth.md Outdated
Expired keys are never used after a failed refresh. Unknown keys during a cooldown are rejected, so a newly published key may take time to become usable.
Choose freshness and cooldown settings together with your provider's key rotation policy.

`prefetch()` fetches absent or expired keys during initialization. Later rotation, expiration, and outages can still cause network I/O.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain the Lambda tradeoff here. Calling prefetch() at module level moves the first JWKS fetch into INIT, but an IdP outage can then fail the cold start. Without prefetch, construction performs no network I/O and the first invocation pays the fetch latency.

This should remain an explicit option, not the default recommendation.

Comment thread docs/utilities/auth.md
IAM allows require a nonempty string `sub` as principal and cover only the supplied request ARN.
Wildcard, missing, or malformed ARNs raise `ValueError`; the helper cannot construct a request-specific IAM policy without a valid ARN.
Other routes need their own decision.
Invalid tokens and insufficient scopes produce a Deny or `isAuthorized=False`; unavailable signing keys raise `JWKSFetchError`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we describe what the API caller actually sees? Middleware converts unavailable keys to 503. A Lambda authorizer instead fails its invocation, and API Gateway normally returns a 5xx.

That difference matters when customers configure retries, alarms, and dashboards.

Comment thread docs/utilities/auth.md Outdated
async def verify_token(self, token: str) -> AccessToken | None:
try:
claims = await asyncio.to_thread(verifier.verify, token)
except (InvalidTokenError, JWKSFetchError):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This converts unavailable JWKS infrastructure into the same result as an invalid token. It is fail-closed, but during an IdP outage the client may incorrectly try to authenticate again.

If the MCP SDK cannot represent availability separately here, please show how the Lambda owner can record JWKSFetchError before returning None.

Comment thread examples/auth/templates/sam.yaml
Comment thread examples/auth/template.yaml Outdated
Globals:
Function:
Runtime: python3.12
CodeUri: src/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The authorizers and backend functions share the same source and requirements. This packages PyJWT and cryptography into backend functions that do not use Auth.

Can we separate the authorizer and backend artifacts so the additional package size stays with the functions that need it?

@bfreiberg bfreiberg changed the title feat(auth): add JWT verification and OAuth client credentials feat(auth): add JWT verification and API Gateway authorization Sep 18, 2026
@powertools-for-aws-oss-automation

Copy link
Copy Markdown

Not all issues are linked correctly.

Please link each issue to the PR either manually or using a closing keyword in the format fixes #<issue-number> format.

If mentioning more than one issue, separate them with commas: i.e. fixes #<issue-number-1>, closes #<issue-number-2>.

Add JWT verification, coordinated JWKS caching, API Gateway authorization,
and OAuth client credentials with optional dependencies, documentation,
examples, and tests.

Include exception-safe claims cleanup, sanitized provider errors, and lazy
imports for OAuth-only clients and static-key verification.
Exercise malformed inputs, shared failures, waiter deadlines, and persistent HTTPS connections. Collect fresh-process import coverage through coverage.py's subprocess patch for pytest-cov 7.
Defer OAuth client credentials to a follow-up. Add fixed failure reasons, authorizer diagnostics, and signed claim/header profile constraints. Retain urllib3 in Layers and separate the SAM authorizer and backend artifacts, with expanded tests and documentation.
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
D Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants