Monitor the MCP servers your agents depend onNot just an HTTP 200
An MCP server that answers HTTP 200 can still be broken, or changed underneath you: the handshake fails, a tool disappears, a tool's schema, description or permissions change after you approved it, or a tool call returns the wrong output. Yorker drives the full MCP session as a unit and emits the result as standard OTLP into your own backend.
monitors:
- name: Docs MCP Server
type: mcp
endpoint: https://mcp.example.com/mcp
frequency: 5m
locations: [us-east, eu-west]
timeoutMs: 30000
detectSchemaDrift: true
rejectUnauthenticated: true
auth:
type: bearer
token: {{secrets.MCP_TOKEN}}
expectedTools:
- search_docs
- get_page
testCalls:
- toolName: search_docs
arguments:
query: rate limits
expectedOutputContains: "rate"
How it works
Initialize handshake
Yorker sends a JSON-RPC initialize, checks the response carries protocolVersion and capabilities, captures any Mcp-Session-Id, then sends notifications/initialized.
tools/list + expected tools
Discovers the tool catalog and asserts every tool in your expectedTools list is present. A missing tool fails the check.
Tool and server drift
Each tool's input schema, description and annotations are fingerprinted and diffed against the previous run, along with the server's reported identity. Changes and permission escalations are reported.
Test calls + OTLP emit
Optional tools/call runs assert on output, and an optional credential-less initialize checks that auth is enforced. Check results and drift events go to your backend as standard OTLP.
The whole session, not one request
Yorker speaks JSON-RPC 2.0 and the MCP lifecycle. It runs the exact sequence a real client runs and validates each phase, so a failure points at the actual problem instead of a generic timeout.
Yorker uses the Streamable HTTP transport and initializes with protocol version 2025-03-26. If your server responds with HTML or a non-JSON content type, the result says the Streamable HTTP transport may not be supported, rather than a generic timeout.
initializeAsserts a JSON-RPC 2.0 success carrying protocolVersion and capabilities. A handshake that fails or omits these fields is caught here.notifications/initializedSent as a real notification (no id), with a non-2xx response treated as a phase failure.tools/listValidates result.tools is an array, filters out nameless entries, and records the discovered tool catalog on every run.tools/callOptional per-tool calls with arguments, validated against an expected output substring when configured.
# 1. initialize → result.protocolVersion + capabilities
→ initialize { protocolVersion, clientInfo }
← result { protocolVersion, capabilities }
Mcp-Session-Id: sess_abc123
# 2. notifications/initialized (no id, 2xx expected)
→ notifications/initialized
# 3. tools/list → result.tools[]
→ tools/list
← result.tools [search_docs, get_page]
# 4. tools/call → assert output
→ tools/call search_docs { query }
← result contains "rate" ✓summarizeaddedsearch_docsmodifiedupdate_pagemodifiedlegacy_lookupremovedserver version1.4.2 → 1.5.0Know when a tool changes after you approved it
A tool that changes underneath you is a silent break, and sometimes a security problem. The server stays up and the HTTP status stays 200, but the arguments your agents send no longer match, or the tool's description now tells the model to do something it didn't before. Changing a tool after clients have approved it is how published MCP attacks such as tool poisoning and rug pulls work.
Yorker fingerprints each tool's input schema, description and annotations separately and diffs them against the previous run. You see which part changed, the description before and after, and any annotation hint that moved to the less safe side, such as a tool that stops being read-only. Server name, version, protocol version and capabilities are tracked too, and the Tools tab lists recent changes from the last 30 days.
Drift doesn't fail the check. It's recorded on the run, and an alert rule with the mcp_schema_drift condition notifies you, filtered by change type: for example, page on escalated and new tools, and opt in to server version changes separately.
Turn a tool into a contract test
List the tools your agents require in expectedTools; any missing tool fails the check. Add testCalls to actually exercise a tool and assert on its output.
A missing expected substring fails as an assertion failure; a JSON-RPC error from the tool fails as an error. Auth is injected per request (basic, bearer, or api-key) with tokens pulled from team secrets, never written into your config. Add rejectUnauthenticated and each run that completes the session then opens one with no credentials, failing the check unless the server answers 401 or 403.
auth:
type: api-key
header: X-API-Key
value: {{secrets.MCP_API_KEY}}
rejectUnauthenticated: true
expectedTools:
- search_docs
- get_page
- summarize
testCalls:
- toolName: search_docs
arguments:
query: rate limits
expectedOutputContains: "rate"
# missing tool, wrong output or open auth → check fails
# tool drift → recorded; alert via an mcp_schema_drift rule# emitted as OTLP to your backend
synthetics.check.type mcp
synthetics.check.name "Docs MCP Server"
synthetics.location.name us-east
synthetics.response_time_ms 214
# recorded in Yorker (Tools tab, API)
mcpToolsDiscovered [search_docs, get_page]
mcpSchemaDrift [{ summarize: added }]
# synthetics.mcp.schema_drift event (OTLP, on drift)
added_tools [summarize]
escalated_tools [update_page]
# W3C trace propagation to the MCP server
traceparent 00-4bf92f35...-a3ce929d-01MCP health as standard OTLP
MCP checks run on the same ephemeral, tenant-isolated runners as your HTTP and browser checks, across 14 hosted locations or a private location behind your firewall. Every run emits standard OTLP into your own OTel backend — no proprietary format, no separate dashboard.
Every run emits status, response time, location and any error as standard OTLP. The discovered tool catalog and drift detail are recorded in Yorker, on the Tools tab and through the API. A W3C traceparent is propagated to the MCP server so the check links into your distributed traces.
When a run detects drift and an OTLP endpoint is configured, Yorker also forwards a synthetics.mcp.schema_drift event to your backend with per-change-type counts, the affected tool names, and which tools changed description, escalated permissions, or which server fields changed. A security team collecting telemetry can pick the change up alongside its other signals. The event fires only when drift is present.
Why an HTTP check isn't enough
An HTTP 200 says the process is listening. It does not say the initialize handshake succeeds, the tools your agents depend on are present, the input schemas are unchanged, or a tool call returns the right output.
Yorker validates the full MCP session as a unit (handshake, tools/list and tool-call output) so a failure points at the real break instead of a generic timeout, and it flags tools whose schema, description or permissions change after you approved them.
MCP checks emit standard OTLP into your own backend, alongside your HTTP, browser, and infrastructure telemetry. Same runners, same locations, same data model.
Frequently asked
What does an MCP check actually validate?
Yorker drives the full Streamable HTTP MCP session as a single unit: it sends an initialize request, validates that the response carries protocolVersion and capabilities, sends the notifications/initialized notification, then calls tools/list. From there it checks that the tools you expect are present, fingerprints each tool's input schema, description and annotations to detect changes against the previous run, and optionally runs tools/call against specific tools and asserts on the output. It can also check that the server rejects a session opened with no credentials. Session ids returned in the Mcp-Session-Id header are carried on every subsequent request, exactly as a real client would.
How is this different from an HTTP check against the MCP endpoint?
An HTTP 200 only tells you the process is listening. It does not tell you whether the initialize handshake succeeds, whether the tools your agents depend on are still in the catalog, whether a tool's input schema changed underneath you, or whether a tool call returns the right output. Yorker speaks JSON-RPC 2.0 and the MCP lifecycle, so it catches a broken handshake, a missing tool, a schema change, and a wrong tool result — none of which move the HTTP status code.
How does drift detection work?
When detectSchemaDrift is enabled, each run computes separate SHA-256 hashes of every tool's inputSchema, description and annotations, using a deterministic, key-sorted JSON representation so equivalent definitions always hash identically. It diffs them against the previous run and reports each tool as added, removed or modified, with the parts that changed. It also records the server's reported name, version, protocol version and capabilities and reports when those change. The first run has no baseline to compare against, so it seeds one and reports no drift. Drift doesn't fail the check: it's recorded on the run, and you get notified by adding an alert rule with the mcp_schema_drift condition. Server changes are opt-in on that rule.
Can Yorker catch a tool that changed after I approved it?
Yes. Published MCP attacks, often called tool poisoning or rug pulls, work by changing a tool after a client has approved it, frequently by hiding instructions in the tool's description, which the model reads but most users never see. Yorker flags any change to a tool's description, input schema or annotations on the next run, shows the description before and after in the Tools tab, and lists recent changes from the last 30 days. It reports that the definition changed; it doesn't judge whether the new text is malicious, so treat a description change on a server you depend on as something to review.
What counts as a permission escalation?
MCP tools can declare hints such as readOnlyHint and destructiveHint. Yorker compares them run to run and flags any hint that moves to the less safe side, for example readOnlyHint going from true to false, or an explicit readOnlyHint: true being removed, since clients then assume the tool may modify state. You can alert on escalations alone with changeTypes: [escalated].
Can I check that my MCP server requires authentication?
Yes. With rejectUnauthenticated: true and auth configured, each run that completes the authenticated session then sends an initialize request with no credentials, and fails the check unless the server answers HTTP 401 or 403. An MCP endpoint that accepts anonymous sessions is open to anyone who finds the URL, and this turns that into something checked on every healthy run.
Can I assert that a tool call returns the right answer?
Yes. Each entry in testCalls names a tool, optional arguments, and an optional expectedOutputContains string. Yorker issues a tools/call, and if expectedOutputContains is set it serializes the JSON-RPC result and asserts the substring is present. A missing substring fails the check as an assertion failure; a JSON-RPC error returned by the tool fails it as an error. This turns a tool you depend on into a contract test that runs on a schedule.
What telemetry does an MCP check emit?
MCP checks run on the same runners as HTTP and browser checks and emit standard OTLP into your own OTel backend: status, response time, location and any error for every run, and a synthetics.mcp.schema_drift event with counts and the affected tool names (including description changes, escalations and server changes) whenever drift is detected. The full tool catalog, per-tool hashes and drift detail are recorded in Yorker, on the Tools tab and through the API. A W3C traceparent is propagated to the MCP server so the check links into your distributed traces.
Do MCP checks cost more than HTTP checks?
No. A full MCP session runs several JSON-RPC requests in sequence (handshake, tool catalog, drift detection, and any tool calls or access checks you configure) but bills as a single run at the HTTP rate: 10,000 HTTP + MCP runs a month on the free tier, unlimited on the $29.99 plan.
Put your first check live in one command.
Free tier includes 10,000 HTTP + MCP checks and 1,500 browser checks per month. No credit card required.
npx @yorker/cli init