Skip to content
Merged
24 changes: 12 additions & 12 deletions pkgs/mcpx/docs/spec/conformance-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Generated by `internal/conformance` (`MCPX_UPDATE_MATRIX=1 go test ./internal/conformance -run TestMatrix`); do not edit.

Generated from inputs `fb2ae39dd9de0779` (sha256 of docs/spec/requirements.md, internal/conformance/overrides/*.json, internal/conformance/coverage*.go). `TestMatrix` regenerates this file and fails when it differs, so it describes the tree it is committed in.
Generated from inputs `46df49c7a0e8cc95` (sha256 of docs/spec/requirements.md, internal/conformance/overrides/*.json, internal/conformance/coverage*.go). `TestMatrix` regenerates this file and fails when it differs, so it describes the tree it is committed in.

Every normative requirement of every revision (catalogue: `docs/spec/requirements.md`), for mcpx as a server and as a client. A requirement is *tested* when a named test verifies it in that revision, *n/a* when it does not bind mcpx in that role, and a *gap* when it binds mcpx and mcpx does not meet it yet, with the issue that tracks it. Run exactly the covering tests with `go run ./internal/conformance/cmd/covering`.

Expand Down Expand Up @@ -46,27 +46,27 @@ Every normative requirement of every revision (catalogue: `docs/spec/requirement
| 2025-06-18 | client | SHOULD NOT | 2 | 1 | 1 | 0 | 3 |
| 2025-06-18 | client | MAY | 15 | 13 | 2 | 0 | 20 |
| 2025-06-18 | client | other | 10 | 8 | 2 | 0 | 8 |
| 2025-11-25 | server | MUST | 115 | 94 | 21 | 0 | 65 |
| 2025-11-25 | server | MUST | 115 | 96 | 19 | 0 | 65 |
| 2025-11-25 | server | MUST NOT | 22 | 18 | 4 | 0 | 13 |
| 2025-11-25 | server | SHOULD | 78 | 61 | 17 | 0 | 81 |
| 2025-11-25 | server | SHOULD NOT | 11 | 9 | 2 | 0 | 3 |
| 2025-11-25 | server | MAY | 27 | 25 | 2 | 0 | 28 |
| 2025-11-25 | server | other | 19 | 19 | 0 | 0 | 2 |
| 2025-11-25 | client | MUST | 76 | 53 | 23 | 0 | 104 |
| 2025-11-25 | client | MUST | 76 | 54 | 22 | 0 | 104 |
| 2025-11-25 | client | MUST NOT | 16 | 14 | 2 | 0 | 19 |
| 2025-11-25 | client | SHOULD | 67 | 45 | 22 | 0 | 92 |
| 2025-11-25 | client | SHOULD NOT | 5 | 4 | 1 | 0 | 9 |
| 2025-11-25 | client | MAY | 20 | 17 | 3 | 0 | 35 |
| 2025-11-25 | client | other | 13 | 11 | 2 | 0 | 8 |
| 2026-07-28 | server | MUST | 128 | 105 | 23 | 0 | 96 |
| 2026-07-28 | server | MUST | 128 | 107 | 21 | 0 | 96 |
| 2026-07-28 | server | MUST NOT | 44 | 41 | 3 | 0 | 20 |
| 2026-07-28 | server | SHOULD | 74 | 59 | 15 | 0 | 86 |
| 2026-07-28 | server | SHOULD | 74 | 60 | 14 | 0 | 86 |
| 2026-07-28 | server | SHOULD NOT | 11 | 9 | 2 | 0 | 9 |
| 2026-07-28 | server | MAY | 31 | 29 | 2 | 0 | 30 |
| 2026-07-28 | server | other | 23 | 22 | 1 | 0 | 4 |
| 2026-07-28 | client | MUST | 91 | 61 | 30 | 0 | 133 |
| 2026-07-28 | client | MUST | 91 | 62 | 29 | 0 | 133 |
| 2026-07-28 | client | MUST NOT | 26 | 22 | 4 | 0 | 38 |
| 2026-07-28 | client | SHOULD | 75 | 51 | 24 | 0 | 85 |
| 2026-07-28 | client | SHOULD | 75 | 52 | 23 | 0 | 85 |
| 2026-07-28 | client | SHOULD NOT | 13 | 11 | 2 | 0 | 7 |
| 2026-07-28 | client | MAY | 15 | 11 | 4 | 0 | 46 |
| 2026-07-28 | client | other | 13 | 11 | 2 | 0 | 14 |
Expand Down Expand Up @@ -1377,8 +1377,8 @@ Every normative requirement of every revision (catalogue: `docs/spec/requirement
|---|---|---|---|---|
| [json-schema-default-2020-12](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#schema-dialect) | (definitional) | both | tested: `conformance.TestMessagesServer/2025-11-25/json-schema/json-schema-default-2020-12` | n/a: catalogue marks it n/a |
| [json-schema-may-declare-dialect](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#schema-dialect) | MAY | both | tested: `conformance.TestMessagesServer/2025-11-25/json-schema/json-schema-may-declare-dialect` | tested: `conformance.TestMessagesClient/2025-11-25/json-schema/json-schema-may-declare-dialect` |
| [json-schema-support-2020-12](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#implementation-requirements) | MUST | both | **gap** #254: mcpx never validates arguments against a tool's inputSchema; schemas pass through | n/a: passes through |
| [json-schema-validate-per-dialect-and-error-on-unsupported](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#implementation-requirements) | MUST | both | **gap** #254: no validator in the binary, so an unsupported $schema is forwarded, not refused | **gap** #254: no validator in the binary, so an unsupported $schema is accepted |
| [json-schema-support-2020-12](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#implementation-requirements) | MUST | both | tested: `mcpclient.TestCallToolRejectsInvalidArgumentsBeforeTheTransport@all` | n/a: passes through |
| [json-schema-validate-per-dialect-and-error-on-unsupported](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#implementation-requirements) | MUST | both | tested: `mcpclient.TestListToolsExcludesUncompilableSchemasWithWarning@all`<br>`schemaval.TestDeclaredDialectsCompile@all`<br>`schemaval.TestUnsupportedDialect@all` | tested: `mcpclient.TestListToolsExcludesUncompilableSchemasWithWarning@all`<br>`schemaval.TestDeclaredDialectsCompile@all`<br>`schemaval.TestUnsupportedDialect@all` |
| [json-schema-document-dialects](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#implementation-requirements) | SHOULD | both | **gap** #254: no document states which dialects mcpx supports | **gap** #254: no document states which dialects mcpx supports |
| [json-schema-recommend-2020-12](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#schema-dialect) | RECOMMENDED | both | tested: `conformance.TestMessagesServer/2025-11-25/json-schema/json-schema-recommend-2020-12` | n/a: catalogue marks it n/a |
| [json-schema-schemas-valid](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#schema-validation) | MUST | both | tested: `conformance.TestMessagesServer/2025-11-25/json-schema/json-schema-schemas-valid` | n/a: catalogue marks it n/a |
Expand Down Expand Up @@ -2065,14 +2065,14 @@ Every normative requirement of every revision (catalogue: `docs/spec/requirement
|---|---|---|---|---|
| [json-schema-default-2020-12](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#schema-dialect) | (definitional) | both | tested: `conformance.TestMessagesServer/2026-07-28/json-schema/json-schema-default-2020-12` | n/a: catalogue marks it n/a |
| [json-schema-may-declare-dialect](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#schema-dialect) | MAY | both | tested: `conformance.TestMessagesServer/2026-07-28/json-schema/json-schema-may-declare-dialect` | tested: `conformance.TestMessagesClient/2026-07-28/json-schema/json-schema-may-declare-dialect` |
| [json-schema-support-2020-12](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#implementation-requirements) | MUST | both | **gap** #254: mcpx never validates arguments against a tool's inputSchema; schemas pass through | n/a: passes through |
| [json-schema-validate-per-dialect-and-error-on-unsupported](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#implementation-requirements) | MUST | both | **gap** #254: no validator in the binary, so an unsupported $schema is forwarded, not refused | **gap** #254: no validator in the binary, so an unsupported $schema is accepted |
| [json-schema-support-2020-12](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#implementation-requirements) | MUST | both | tested: `mcpclient.TestCallToolRejectsInvalidArgumentsBeforeTheTransport@all` | n/a: passes through |
| [json-schema-validate-per-dialect-and-error-on-unsupported](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#implementation-requirements) | MUST | both | tested: `mcpclient.TestListToolsExcludesUncompilableSchemasWithWarning@all`<br>`schemaval.TestDeclaredDialectsCompile@all`<br>`schemaval.TestUnsupportedDialect@all` | tested: `mcpclient.TestListToolsExcludesUncompilableSchemasWithWarning@all`<br>`schemaval.TestDeclaredDialectsCompile@all`<br>`schemaval.TestUnsupportedDialect@all` |
| [json-schema-document-dialects](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#implementation-requirements) | SHOULD | both | **gap** #254: no document states which dialects mcpx supports | **gap** #254: no document states which dialects mcpx supports |
| [json-schema-recommend-2020-12](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#schema-dialect) | RECOMMENDED | both | tested: `conformance.TestMessagesServer/2026-07-28/json-schema/json-schema-recommend-2020-12` | n/a: catalogue marks it n/a |
| [json-schema-schemas-valid](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#schema-validation) | MUST | both | tested: `conformance.TestMessagesServer/2026-07-28/json-schema/json-schema-schemas-valid` | n/a: catalogue marks it n/a |
| [json-schema-no-network-ref](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#ref-resolution) | MUST NOT | both | tested: `conformance.TestMessagesServer/2026-07-28/json-schema/json-schema-no-network-ref` | tested: `conformance.TestMessagesClient/2026-07-28/json-schema/json-schema-no-network-ref` |
| [json-schema-remote-ref-opt-in-off-by-default](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#ref-resolution) | MAY / MUST / SHOULD | both | n/a: catalogue marks it n/a | n/a: catalogue marks it n/a |
| [json-schema-reject-unresolved-ref](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#ref-resolution) | SHOULD | both | **gap** #254: an unresolvable $ref is rendered permissively (codegen) rather than rejected | **gap** #254: an unresolvable $ref is rendered permissively (codegen) rather than rejected |
| [json-schema-reject-unresolved-ref](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#ref-resolution) | SHOULD | both | tested: `mcpclient.TestListToolsExcludesUncompilableSchemasWithWarning@all`<br>`schemaval.TestRemoteRefMakesNoRequests@all`<br>`schemaval.TestUnresolvedInternalRef@all` | tested: `mcpclient.TestListToolsExcludesUncompilableSchemasWithWarning@all`<br>`schemaval.TestRemoteRefMakesNoRequests@all`<br>`schemaval.TestUnresolvedInternalRef@all` |
| [json-schema-bound-composition-cost](https://modelcontextprotocol.io/specification/2026-07-28/basic/index#composition-keyword-resource-use) | SHOULD | both | n/a: no validator | n/a: catalogue marks it n/a |
| [tools-schema-default-2020-12](https://modelcontextprotocol.io/specification/2026-07-28/server/tools#tool) | (definition) | both | tested: `conformance.TestMessagesServer/2026-07-28/json-schema/tools-schema-default-2020-12` | tested: `conformance.TestMessagesClient/2026-07-28/json-schema/tools-schema-default-2020-12` |

Expand Down
15 changes: 5 additions & 10 deletions pkgs/mcpx/internal/conformance/coverage_messages.go
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,11 @@ func init() {
"tools-schema-default-2020-12",
"json-schema-no-network-ref",
),
S("json-schema-support-2020-12", "mcpclient.TestCallToolRejectsInvalidArgumentsBeforeTheTransport@all"),
S("json-schema-validate-per-dialect-and-error-on-unsupported", "mcpclient.TestListToolsExcludesUncompilableSchemasWithWarning@all", "schemaval.TestUnsupportedDialect@all", "schemaval.TestDeclaredDialectsCompile@all"),
C("json-schema-validate-per-dialect-and-error-on-unsupported", "mcpclient.TestListToolsExcludesUncompilableSchemasWithWarning@all", "schemaval.TestUnsupportedDialect@all", "schemaval.TestDeclaredDialectsCompile@all"),
S("json-schema-reject-unresolved-ref", "mcpclient.TestListToolsExcludesUncompilableSchemasWithWarning@all", "schemaval.TestUnresolvedInternalRef@all", "schemaval.TestRemoteRefMakesNoRequests@all"),
C("json-schema-reject-unresolved-ref", "mcpclient.TestListToolsExcludesUncompilableSchemasWithWarning@all", "schemaval.TestUnresolvedInternalRef@all", "schemaval.TestRemoteRefMakesNoRequests@all"),
)
registerGaps(
Gap{ID: "errors-local-errors-not-mistaken-for-peer", Side: Server, Issue: "#206",
Expand All @@ -117,16 +122,6 @@ func init() {
Why: "mcpx answers methods whatever was negotiated (accept liberally, revisions.go); the schema guidance asks for -32601"},
Gap{ID: "mrtr-missing-capability-error-32021", Side: Server, Issue: "#257",
Why: "through the gateway (mcpx_call) a call that needs input from a client that declared none goes to the broker instead of failing with -32021; the specification contradicts itself here (docs/spec/revision-conflicts.md). In pass-through mode the client's declaration is relayed and the upstream's -32021 returned (mcpserver.TestAPassThroughUpstreamsMissingCapabilityIsRelayed)"},
Gap{ID: "json-schema-support-2020-12", Side: Server, Issue: "#254",
Why: "mcpx never validates arguments against a tool's inputSchema; schemas pass through"},
Gap{ID: "json-schema-validate-per-dialect-and-error-on-unsupported", Side: Server, Issue: "#254",
Why: "no validator in the binary, so an unsupported $schema is forwarded, not refused"},
Gap{ID: "json-schema-validate-per-dialect-and-error-on-unsupported", Side: Client, Issue: "#254",
Why: "no validator in the binary, so an unsupported $schema is accepted"},
Gap{ID: "json-schema-reject-unresolved-ref", Side: Server, Issue: "#254",
Why: "an unresolvable $ref is rendered permissively (codegen) rather than rejected"},
Gap{ID: "json-schema-reject-unresolved-ref", Side: Client, Issue: "#254",
Why: "an unresolvable $ref is rendered permissively (codegen) rather than rejected"},
Gap{ID: "json-schema-document-dialects", Side: Server, Issue: "#254",
Why: "no document states which dialects mcpx supports"},
Gap{ID: "json-schema-document-dialects", Side: Client, Issue: "#254",
Expand Down
78 changes: 75 additions & 3 deletions pkgs/mcpx/internal/conformance/officialclient/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -241,21 +241,93 @@ func callsFor(ctx context.Context, m *mcpx, scenario string) ([]toolCall, error)
return nil, fmt.Errorf("search: %v\n%s", err, out)
}
var hits []struct {
Namespace string `json:"namespace"`
Tool string `json:"tool"`
Namespace string `json:"namespace"`
Tool string `json:"tool"`
InputSchema map[string]any `json:"inputSchema"`
}
if err := json.Unmarshal(out, &hits); err != nil {
return nil, fmt.Errorf("search output: %v\n%s", err, out)
}
var calls []toolCall
for _, h := range hits {
if h.Namespace == *namespace {
calls = append(calls, toolCall{Name: h.Tool})
calls = append(calls, toolCall{Name: h.Tool, Arguments: minimalArgs(h.InputSchema)})
}
}
return calls, nil
}

// minimalArgs builds the smallest argument object a tool's inputSchema
// accepts: one minimal value per required property, nothing else. This
// fallback used to call every advertised tool with no arguments, which
// was fine while mcpx forwarded arguments unvalidated; since #1483 the
// client enforces inputSchema at CallTool, so a driver exercising a
// tool it knows nothing about has to ask the schema what a call looks
// like -- the same thing a schema-aware host does.
func minimalArgs(schema map[string]any) map[string]any {
args := map[string]any{}
props, _ := schema["properties"].(map[string]any)
for _, name := range requiredNames(schema) {
sub, _ := props[name].(map[string]any)
args[name] = minimalValue(sub)
}
return args
}

func requiredNames(schema map[string]any) []string {
raw, _ := schema["required"].([]any)
names := make([]string, 0, len(raw))
for _, r := range raw {
if s, ok := r.(string); ok {
names = append(names, s)
}
}
return names
}

func minimalValue(schema map[string]any) any {
if schema == nil {
return ""
}
if v, ok := schema["default"]; ok {
return v
}
if v, ok := schema["const"]; ok {
return v
}
if e, ok := schema["enum"].([]any); ok && len(e) > 0 {
return e[0]
}
typ := schema["type"]
if list, ok := typ.([]any); ok {
typ = nil
for _, item := range list {
if s, _ := item.(string); s != "" && s != "null" {
typ = s
break
}
}
}
switch typ {
case "string":
return ""
case "integer", "number":
return 0
case "boolean":
return false
case "array":
return []any{}
case "object":
return minimalArgs(schema)
case "null":
return nil
}
if _, ok := schema["properties"]; ok {
return minimalArgs(schema)
}
return ""
}

// schemaEcho round-trips the focal tool's inputSchema -- as mcpx holds it,
// which is what the scenario is checking mcpx preserved -- back through the
// scenario's echo tool.
Expand Down
26 changes: 26 additions & 0 deletions pkgs/mcpx/internal/daemon/callerr.go
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,17 @@ func (r *Registry) explainCall(ctx context.Context, server, tool string, args an
return err
}
code, message, ok := upstreamFault(err)
if !ok {
// A client-side validation refusal (Phase 2 of #1481) never
// reached the server, but the failure is still the arguments'
// fault: the client detected the same -32602 the server would
// have answered with, and the error carries that code (the
// gateway relays it by the same interface). Explain it with
// its own message.
if c, m, coded := clientFault(err); coded {
code, message, ok = c, m, true
}
}
if !ok {
// A timeout, a server that would not start, a pipe that broke before
// the answer: nothing came back, so there is nothing to explain, and
Expand Down Expand Up @@ -199,6 +210,21 @@ func upstreamFault(err error) (code int, message string, ok bool) {
return 0, "", false
}

// clientFault is a failure mcpx itself detected before the call reached
// a server, when the error carries the JSON-RPC code it stands for.
// Only Invalid Params is read this way: it is the one code that says the
// arguments were at fault, which is the one thing a schema can explain.
// The check is an interface assertion (the gateway's precedent in
// internal/mcpserver), not a concrete type, so a validation error from
// any layer that carries its code is explained the same way.
func clientFault(err error) (code int, message string, ok bool) {
var coded interface{ RPCCode() int }
if errors.As(err, &coded) && coded.RPCCode() == diagnose.InvalidParams {
return coded.RPCCode(), err.Error(), true
}
return 0, "", false
}

// upstreamData is the data member of the error upstreamFault found.
func upstreamData(err error) json.RawMessage {
for e := err; e != nil; e = errors.Unwrap(e) {
Expand Down
10 changes: 7 additions & 3 deletions pkgs/mcpx/internal/e2e/callerr_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -126,10 +126,14 @@ func TestARefusedCallCarriesTheSchemaChangeOverV1(t *testing.T) {
if !hasRepoChange(doc.Diagnostics) {
t.Errorf("/v1/call should name the tool, the field and what changed:\n%s", b)
}
// Added to, never replaced.
if !strings.HasPrefix(doc.Error, "mcp error -32602: Invalid params\n") ||
// Added to, never replaced. Phase 2 of #1481: the refusal is now
// detected by mcpx's own argument validation before the call
// reaches the server, so the headline is the validator's -32602
// naming the tool and the path-qualified reason, not the server's
// answer — still first and whole, the diagnostic after it.
if !strings.HasPrefix(doc.Error, "Invalid params: tool \"create_issue\": ") ||
!strings.Contains(doc.Error, "repo is required") {
t.Errorf("the error text should be the server's, then the rendered diagnostic:\n%s", doc.Error)
t.Errorf("the error text should be the refusal's, then the rendered diagnostic:\n%s", doc.Error)
}

// The same call collected later, as a task.
Expand Down
7 changes: 5 additions & 2 deletions pkgs/mcpx/internal/e2e/ceiling_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -270,8 +270,11 @@ func TestRepairCannotRaiseAutonomyAboveTheCeiling(t *testing.T) {
if len(got.Diagnostics) != 0 {
t.Errorf("repair.autonomy run past autonomy.max off still attached diagnostics:\n%s", b)
}
if !strings.HasPrefix(got.Error, "mcp error -32602") {
t.Errorf("the server's own error should still come back:\n%s", b)
// Phase 2 of #1481: the refusal is detected by mcpx's own argument
// validation, so the error that comes back unexplained is the
// validator's -32602 headline, not a server answer.
if !strings.HasPrefix(got.Error, "Invalid params") {
t.Errorf("the refusal's own error should still come back:\n%s", b)
}
}

Expand Down
Loading
Loading