Monitor the MCP servers your agents depend onNot just an HTTP 200
An MCP server that answers HTTP 200 can still be broken: the handshake fails, a tool disappears, a schema drifts, or a tool call returns the wrong output. Yorker drives the full MCP session as a unit and emits the result as standard OTLP into your own backend.
monitors:
- name: Docs MCP Server
type: mcp
endpoint: https://mcp.example.com/mcp
frequency: 5m
locations: [us-east, eu-west]
timeoutMs: 30000
detectSchemaDrift: true
auth:
type: bearer
token: {{secrets.MCP_TOKEN}}
expectedTools:
- search_docs
- get_page
testCalls:
- toolName: search_docs
arguments:
query: rate limits
expectedOutputContains: "rate"
How it works
Initialize handshake
Yorker sends a JSON-RPC initialize, checks the response carries protocolVersion and capabilities, captures any Mcp-Session-Id, then sends notifications/initialized.
tools/list + expected tools
Discovers the tool catalog and asserts every tool in your expectedTools list is present. A missing tool fails the check.
Schema-drift detection
Each tool's input schema is hashed deterministically and diffed against the previous run. Added, removed, and modified tools are reported.
Test calls + OTLP emit
Optional tools/call runs assert on output. Status, discovered tools, drift, and per-phase timing are emitted as standard OTLP to your backend.
The whole session, not one request
Yorker speaks JSON-RPC 2.0 and the MCP lifecycle. It runs the exact sequence a real client runs and validates each phase, so a failure points at the actual problem instead of a generic timeout.
initializeAsserts a JSON-RPC 2.0 success carrying protocolVersion and capabilities. A handshake that fails or omits these fields is caught here.notifications/initializedSent as a real notification (no id), with a non-2xx response treated as a phase failure.tools/listValidates result.tools is an array, filters out nameless entries, and records the discovered tool catalog on every run.tools/callOptional per-tool calls with arguments, validated against an expected output substring when configured.
# 1. initialize → result.protocolVersion + capabilities
→ initialize { protocolVersion, clientInfo }
← result { protocolVersion, capabilities }
Mcp-Session-Id: sess_abc123
# 2. notifications/initialized (no id, 2xx expected)
→ notifications/initialized
# 3. tools/list → result.tools[]
→ tools/list
← result.tools [search_docs, get_page]
# 4. tools/call → assert output
→ tools/call search_docs { query }
← result contains "rate" ✓summarizeaddedsearch_docsmodifiedlegacy_lookupremovedCatch schema drift before your agents do
A tool whose input schema changes underneath you is a silent break: the server stays up, the HTTP status stays 200, but the arguments your agents send no longer match. Yorker hashes each tool's input schema with a deterministic, key-sorted representation, so equivalent schemas always hash the same.
On the next run it diffs against the previous hashes and reports each tool as added, removed, or modified. The first run records an empty result and seeds the baseline, so you can tell "no drift this run" apart from "drift detection off."
Turn a tool into a contract test
List the tools your agents require in expectedTools; any missing tool fails the check. Add testCalls to actually exercise a tool and assert on its output.
A missing expected substring fails as an assertion failure; a JSON-RPC error from the tool fails as an error. Auth is injected per request — basic, bearer, or api-key — with tokens pulled from team secrets, never written into your config.
auth:
type: api-key
header: X-API-Key
value: {{secrets.MCP_API_KEY}}
expectedTools:
- search_docs
- get_page
- summarize
testCalls:
- toolName: search_docs
arguments:
query: rate limits
expectedOutputContains: "rate"
# missing tool, drift, or wrong output → check fails# synthetics.check.run (mcp)
synthetics.check.type mcp
synthetics.check.name "Docs MCP Server"
synthetics.location.name us-east
synthetics.response_time_ms 214
# per-phase timing (responseBody)
phaseTiming.initializeMs 61
phaseTiming.toolsListMs 48
phaseTiming.testCallsMs 105
# discovered catalog + drift
mcpToolsDiscovered [search_docs, get_page]
mcpSchemaDrift [{ summarize: added }]
# W3C trace propagation to the MCP server
traceparent 00-4bf92f35...-a3ce929d-01MCP health as standard OTLP
MCP checks run on the same ephemeral, tenant-isolated runners as your HTTP and browser checks, across 14 hosted locations or a private location behind your firewall. Every run emits standard OTLP into your own OTel backend — no proprietary format, no separate dashboard.
Each result carries status, total response time, the discovered tool catalog, the computed schema drift, and per-phase timing for the initialize, tools/list, and test-call phases. A W3C traceparent is propagated to the MCP server so the check links into your distributed traces.
Why an HTTP check isn't enough
An HTTP 200 says the process is listening. It does not say the initialize handshake succeeds, the tools your agents depend on are present, the input schemas are unchanged, or a tool call returns the right output.
Yorker validates the full MCP session as a unit — handshake, tools/list, schema drift, and tool-call output — so a failure points at the real break instead of a generic timeout.
MCP checks emit standard OTLP into your own backend, alongside your HTTP, browser, and infrastructure telemetry. Same runners, same locations, same data model.
Frequently asked
What does an MCP check actually validate?
Yorker drives the full Streamable HTTP MCP session as a single unit: it sends an initialize request, validates that the response carries protocolVersion and capabilities, sends the notifications/initialized notification, then calls tools/list. From there it checks that the tools you expect are present, hashes each tool's input schema to detect drift against the previous run, and optionally runs tools/call against specific tools and asserts on the output. Session ids returned in the Mcp-Session-Id header are carried on every subsequent request, exactly as a real client would.
How is this different from an HTTP check against the MCP endpoint?
An HTTP 200 only tells you the process is listening. It does not tell you whether the initialize handshake succeeds, whether the tools your agents depend on are still in the catalog, whether a tool's input schema changed underneath you, or whether a tool call returns the right output. Yorker speaks JSON-RPC 2.0 and the MCP lifecycle, so it catches a broken handshake, a missing tool, a schema change, and a wrong tool result — none of which move the HTTP status code.
How does schema-drift detection work?
When detectSchemaDrift is enabled, the executor computes a SHA-256 hash of each tool's inputSchema using a deterministic, key-sorted JSON representation, so equivalent schemas always hash identically regardless of key order. On the next run it diffs the current hashes against the previous run's hashes and reports each tool as added, removed, or modified. The first run has no baseline to compare against, so it records an empty drift result and seeds the baseline for the next run.
Can I assert that a tool call returns the right answer?
Yes. Each entry in testCalls names a tool, optional arguments, and an optional expectedOutputContains string. Yorker issues a tools/call, and if expectedOutputContains is set it serializes the JSON-RPC result and asserts the substring is present. A missing substring fails the check as an assertion failure; a JSON-RPC error returned by the tool fails it as an error. This turns a tool you depend on into a contract test that runs on a schedule.
What telemetry does an MCP check emit?
MCP checks run on the same runners as HTTP and browser checks and emit standard OTLP into your own OTel backend. Each run carries status, total response time, the list of discovered tools, the per-tool schema hashes, the computed drift, and per-phase timing for the initialize, tools/list, and test-call phases. A W3C traceparent is propagated to the MCP server so the check links into your distributed traces. No proprietary format, no separate silo.
Put your first check live in one command.
Free tier includes 10,000 HTTP + MCP checks and 1,500 browser checks per month. No credit card required.
npx @yorker/cli init