Skip to content

Plugin Directory: Validate the security scan callback contract at the route - #787

Closed
obenland wants to merge 3 commits into
WordPress:trunkfrom
obenland:feature/gandalf-callback-contract
Closed

Plugin Directory: Validate the security scan callback contract at the route#787
obenland wants to merge 3 commits into
WordPress:trunkfrom
obenland:feature/gandalf-callback-contract

Conversation

@obenland

Copy link
Copy Markdown
Member

Summary

The Gandalf callback route currently validates nothing: the handler trusts every field of the body, the plugin is resolved via $request['plugin_slug'] — where JSON body params outrank the URL segment, so a body field named plugin_slug overrides the routed identity — and an unresolvable slug reaches the handler as a null plugin. This PR moves contract enforcement to the route, ahead of #777, whose scan policy acts on these fields (max_risk_score deciding whether a release is blocked deserves a validated type).

What it does

  • The route args carry the callback body schema: types, the status/subject_type enums, required identity fields. Gandalf_Scan::validate_callback_data() adds the per-status required sets (a completed callback must carry its verdict fields; a failed one its error).
  • The plugin is resolved from the URL segment via get_url_params(), a payload asserting a different plugin than it was routed to is rejected and recorded via record_invalid_callback(), and an unknown slug is rejected at URL-arg validation.

Deliberately lenient

The contract is strict only about what the directory acts on. Everything descriptive is open, so Gandalf can evolve without voiding deliveries:

  • No unknown-field rejection at any level — new top-level fields, new finding fields, new investigation fields all flow through.
  • No enums on display-only fields — a new severity or investigation status/result is accepted as a plain string.
  • No upper bound on scores — a recalibrated scale doesn't 400; the threshold comparison is the policy layer's job.
  • No string length caps — an over-long model-generated title must not void a verdict.

Testing

tests/Gandalf_Scan_Endpoint_Test.php dispatches a production-shaped callback body (three findings, multi-paragraph explanations, code snippets, skipped investigations) through rest_do_request(), covering the layers unit tests bypass: bearer authentication (missing and wrong token), routing, schema rejections (mistyped and missing fields), unknown-plugin rejection, slug reconciliation, and — the leniency regression test — contract additions at every level plus a novel severity and an out-of-range score flowing through to a 200.

Notes

Split out of #777, which will be rebased on top: its scan policy reads callback fields on the promise that the route validated them.

🤖 Generated with Claude Code

Copilot AI lite review requested due to automatic review settings August 11, 2026 20:54
@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props obenland, lucasbustamante.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR tightens the Plugin Directory’s Gandalf security-scan callback endpoint by enforcing a request contract at the REST route layer (schema + cross-field validation), preventing plugin identity confusion between URL and body payload, and adding end-to-end REST tests to exercise authentication, routing, validation, and acceptance/leniency behavior.

Changes:

  • Adds a detailed REST arg schema for the Gandalf callback payload and a validate_callback_data() cross-field validator.
  • Resolves the plugin strictly from the URL segment and rejects callbacks whose payload asserts a different plugin slug.
  • Introduces a comprehensive REST-dispatch test suite for valid callbacks, auth failures, schema failures, slug reconciliation, and leniency toward unknown fields.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File Description
wordpress.org/public_html/wp-content/plugins/plugin-directory/api/routes/class-gandalf-scan.php Adds route-level schema + cross-field validation and tightens plugin identity resolution to the URL param.
wordpress.org/public_html/wp-content/plugins/plugin-directory/tests/Gandalf_Scan_Endpoint_Test.php New end-to-end REST tests for callback auth, routing, validation, identity checks, and leniency regression.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@obenland
obenland force-pushed the feature/gandalf-callback-contract branch from d74c0ce to 293b9c5 Compare August 11, 2026 21:15
@obenland
obenland requested a lite review from Copilot August 11, 2026 21:16

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

wordpress.org/public_html/wp-content/plugins/plugin-directory/api/routes/class-gandalf-scan.php:255

  • This WP_Error message is currently a raw string (not translatable), while other endpoint error messages use __() with the wporg-plugins textdomain. Consider localizing it for consistency.
		// The payload must assert the same plugin the callback was routed to.
		if ( ! $error && ( $data['slug'] ?? '' ) !== $plugin->post_name ) {
			$error = new WP_Error( 'invalid_gandalf_scan', 'Security scan callback slug does not match the plugin.', [ 'status' => WP_Http::BAD_REQUEST ] );
		}

wordpress.org/public_html/wp-content/plugins/plugin-directory/api/routes/class-gandalf-scan.php:214

  • The missing-field error message here is not run through i18n, but other REST errors in this handler use __() with the wporg-plugins textdomain. Please make this string translatable as well (and keep the sprintf() placeholder).

This issue also appears on line 252 of the same file.

			if ( ! isset( $data[ $field ] ) ) {
				return new WP_Error(
					'invalid_gandalf_scan_callback',
					sprintf( 'Invalid security scan callback: missing %s.', $field ),
					[ 'status' => WP_Http::BAD_REQUEST ]
				);

@obenland
obenland force-pushed the feature/gandalf-callback-contract branch from 293b9c5 to 77707a0 Compare August 11, 2026 21:44
@obenland
obenland requested a review from Luc45 August 11, 2026 21:47
@obenland

Copy link
Copy Markdown
Member Author

@Luc45 Could you take a look at this when you get a chance?

The intention is to be a bit more specific about what the endpoint expects, so that consumers down the stack can rely on certain information to be available, without constraining future changes to the payload too much.
As in, you should always be able to add stuff without breaking the callback endpoint.

obenland and others added 3 commits August 11, 2026 16:54
… route.

The route args now carry the callback body schema, so a malformed
delivery is rejected before the handler trusts its fields, the plugin is
resolved from the URL segment rather than get_param()'s body-first
precedence, a payload asserting a different plugin than it was routed to
is rejected and recorded, and an unresolvable slug no longer reaches the
handler as a null plugin.

The contract is strict only about what the directory acts on — status
branching, identity, and field types. It is deliberately lenient about
the descriptive payload: no length caps, no unknown-field rejection at
any level, no enums on display-only fields, and no upper bound on
scores, so the scanner can evolve without voiding deliveries.

Tests dispatch a production-shaped callback through the REST server,
covering authentication, routing, schema rejections, contract additions
flowing through, and the slug reconciliation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…k contract.

Findings now require only risk_score — the one field the directory
indexes unguarded — and the investigation sub-object is fully optional:
descriptive fields are read defensively by every consumer, and requiring
them contradicted the contract's leniency goal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The scanner builds callback bodies through strict schemas on its own
side, so a completed verdict missing its envelope is not a constructible
delivery — the same reasoning that keeps the contract free of
unknown-field rejection. The args still validate types and identity, and
the handlers read verdict fields defensively.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@obenland
obenland force-pushed the feature/gandalf-callback-contract branch from 59bf8d9 to 73daf6c Compare August 11, 2026 21:54
@Luc45

Luc45 commented Aug 11, 2026

Copy link
Copy Markdown

I agree with validating at the boundary so that downstream consumers can rely on it without having to check for isset, adding nullables, fallbacks, etc.

I can add automated tests on Gandalf to make sure I never break the contract expected by this callback endpoint. To keep it up-to-date, I'd fetch the actual file wordpress.org/public_html/wp-content/plugins/plugin-directory/api/routes/class-gandalf-scan.php from trunk as part of the test.

And I ack that this is a guarantee of presence and format of expected fields, not a strict limitation for new fields.

I'll approve it as soon as I verify Gandalf sends the callback as expected.

@Luc45 Luc45 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran this PR locally and pointed Gandalf locally against it, the callbacks were accepted, both for success and failed scans, and also asserted that a mistyped max_risk_score was rejected.

@bazza bazza closed this in 3f2e7fe Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants