The same minimal server as above declares only tools. Against it, server mode reports 6 passed, 24 failed. Nearly all of the 24 are not conformance failures:
| Cause |
Scenarios |
Capability not declared; -32601 Method not found is the correct answer |
prompts-* (5), resources-* (6), logging-set-level, tools-call-with-logging, completion-complete |
Fixture tool not present (test_image_content, test_tool_with_progress, …) |
tools-call-image, -audio, -embedded-resource, -mixed-content, -with-progress, -sampling, -elicitation, elicitation-sep1034-defaults, elicitation-sep1330-enums |
| Genuine |
dns-rebinding-protection (served a foreign Host/Origin with 200) |
For comparison, @modelcontextprotocol/server-everything 2026.8.31 scores 13 passed, 19 failed on 0.1.16.
This is expected for a suite built to test SDKs against the everything-server. The result is still that the suite cannot be pointed at a production server to answer "is this server conformant?", which is the question most operators and registries are asking. A correct server scores 6/30, and its only real defect is buried among 23 false ones.
Suggestions, cheapest first:
- Gate on declared capabilities. Read
capabilities from initialize and report prompts, resources, logging and completions scenarios as not applicable when the capability is absent. Answering -32601 for an undeclared capability is what the spec says to do.
- Mark fixture-dependent scenarios (those that call
test_* tools or test:// resources) as needing the fixture, and report them as not applicable when the tool is missing from tools/list, rather than failing them.
- A capability-agnostic suite (e.g.
--suite production) holding only the scenarios that apply to any server: initialize, ping, tools/list shape, unknown-method and invalid-params handling, session handling, protocol-version header validation, and DNS-rebinding protection. That would let registries and operators cite the official suite for a real server.
Happy to contribute scenarios for (3) if that direction is welcome.
The same minimal server as above declares only
tools. Against it,servermode reports 6 passed, 24 failed. Nearly all of the 24 are not conformance failures:-32601 Method not foundis the correct answerprompts-*(5),resources-*(6),logging-set-level,tools-call-with-logging,completion-completetest_image_content,test_tool_with_progress, …)tools-call-image,-audio,-embedded-resource,-mixed-content,-with-progress,-sampling,-elicitation,elicitation-sep1034-defaults,elicitation-sep1330-enumsdns-rebinding-protection(served a foreign Host/Origin with 200)For comparison,
@modelcontextprotocol/server-everything2026.8.31 scores 13 passed, 19 failed on 0.1.16.This is expected for a suite built to test SDKs against the everything-server. The result is still that the suite cannot be pointed at a production server to answer "is this server conformant?", which is the question most operators and registries are asking. A correct server scores 6/30, and its only real defect is buried among 23 false ones.
Suggestions, cheapest first:
capabilitiesfrominitializeand report prompts, resources, logging and completions scenarios as not applicable when the capability is absent. Answering-32601for an undeclared capability is what the spec says to do.test_*tools ortest://resources) as needing the fixture, and report them as not applicable when the tool is missing fromtools/list, rather than failing them.--suite production) holding only the scenarios that apply to any server: initialize, ping,tools/listshape, unknown-method and invalid-params handling, session handling, protocol-version header validation, and DNS-rebinding protection. That would let registries and operators cite the official suite for a real server.Happy to contribute scenarios for (3) if that direction is welcome.