Monitor MCP Servers

An MCP server can return HTTP 200 while being unusable to every agent that depends on it. The process is listening, but the initialize handshake fails, a tool your agents rely on has vanished from the catalog, or a tool's input schema changed shape underneath a caller that worked yesterday. None of those move an HTTP status code.

MCP monitors exercise the server the way a real client does. Every run executes the full JSON-RPC session over Streamable HTTP:

1. Sends initialize and asserts the response carries protocolVersion and capabilities, capturing any Mcp-Session-Id for the rest of the session.
2. Sends the notifications/initialized notification.
3. Calls tools/list and verifies every tool in expectedTools is present (if configured).
4. Runs each testCalls entry: invokes the tool and checks the result contains expectedOutputContains (if provided).
5. Fingerprints each tool's input schema, description and annotations, records the server's reported identity, and reports what changed since the previous run (when detectSchemaDrift is enabled).
6. Optionally sends a second initialize with no credentials and fails unless the server rejects it (when rejectUnauthenticated is enabled).

A failure names the phase that broke (handshake, missing tool, wrong output, unauthenticated access), not a generic timeout.

Before you start

• Your server must speak the Streamable HTTP transport. This is the standard remote transport; a typical endpoint looks like https://example.com/mcp. Servers that only run over stdio are local processes and cannot be monitored externally.
• If the server requires auth, have the credential ready. Yorker supports basic, bearer, and api-key auth, with secrets pulled from your team's secret store so tokens never live in config files.

Create the monitor (Web UI)

1. Open the dashboard, click Create Monitor, and select the MCP tab.
2. Enter the Streamable HTTP endpoint URL.
3. Optionally list the tool names your agents require. A missing tool fails the check.
4. Optionally add test calls: a tool name, arguments, and a substring the result must contain.
5. Choose locations and frequency, then click Create. The first run starts immediately.

Create the monitor (YAML)

monitors:
  - name: Docs MCP Server
    type: mcp
    endpoint: https://mcp.example.com/mcp
    frequency: 5m
    locations:
      - loc_us_east
    auth:
      type: bearer
      token: "{{secrets.MCP_TOKEN}}"
    expectedTools:
      - search_docs
      - fetch_page
    testCalls:
      - toolName: search_docs
        arguments:
          query: "pricing"
        expectedOutputContains: "Plans"
    detectSchemaDrift: true
    rejectUnauthenticated: true

Deploy with yorker deploy. The full field reference is in Configuration. Note that MCP monitors cannot be executed locally with yorker test; deploy them and watch results with yorker results tail or the dashboard.

What schema drift catches

When detectSchemaDrift is enabled (the default), each run fingerprints every tool and compares it with the previous run. Hashes use a deterministic, key-sorted JSON representation, so equivalent definitions always hash identically regardless of key order. The first run has no baseline, so it seeds one and reports no drift.

Description changes matter because the description is the text a model reads to decide what a tool does and when to call it. Instructions hidden in a description, sometimes called tool poisoning, change agent behaviour without touching the input schema. The Tools tab shows the description before and after the change.

Escalations use the MCP spec defaults for hints a server leaves unset, so removing an explicit readOnlyHint: true counts as an escalation, because clients then assume the tool may modify state. The hints compared are readOnlyHint, destructiveHint, idempotentHint and openWorldHint.

The Tools tab lists every change from the last 30 days, not just the latest run, because a change is only reported on the run where it first appears.

Require authentication

An MCP endpoint that accepts sessions without credentials is open to anyone who finds the URL. With rejectUnauthenticated: true (and auth configured), each run also sends an initialize request with no credentials. The check fails if the server answers anything other than HTTP 401 or 403, and the failure message says which status came back. The API rejects the option without auth, since the monitor itself would be unauthenticated.

A server that answers a credential-less request with HTTP 200 and a JSON-RPC error body, instead of a 401 or 403 status, fails this check. That's deliberate: the status code is what proxies, gateways and scanners see.

Description, annotation and server-identity tracking, and rejectUnauthenticated, need a current runner. Hosted locations update automatically. Private location agents running an older image skip the probe and compare input schemas only, so update the agent image before relying on them.

Alerting and telemetry

MCP monitors plug into the same alert rules as HTTP and browser monitors: consecutive failures, multi-location correlation, and notification channels are configured identically. See Set Up Alerts.

Drift doesn't fail a run; alert on it with the mcp_schema_drift condition. Filter by changeTypes (added, removed, modified, escalated, server) and toolNames. server is opt-in, so existing rules don't start firing on every server release:

alerts:
  - conditions:
      - type: mcp_schema_drift
        changeTypes: [added, escalated]
        severity: critical
    channels:
      - "@security-slack"
  - conditions:
      - type: mcp_schema_drift
        changeTypes: [server]
        severity: info
    channels:
      - "@ops-slack"

Results emit as standard OTLP to your configured backend, with per-phase timing for the initialize, tools/list, and test-call phases, the discovered tool catalog, and the computed drift. When a run detects drift and your team has an OTLP endpoint configured, Yorker forwards a synthetics.mcp.schema_drift event carrying per-change-type counts (synthetics.mcp.drift.added_count, removed_count, modified_count, total_count) and the affected tool names (capped at 50 per list), plus synthetics.mcp.drift.description_changed_tools, synthetics.mcp.drift.escalated_count, synthetics.mcp.drift.escalated_tools and synthetics.mcp.drift.server_changed_fields. The event fires only when tool or server drift is present.

Billing

A full MCP session (handshake, tool catalog, schema hashing, and any tool calls you configure) bills as one run at the HTTP rate: 10,000 HTTP + MCP runs a month on the free tier, unlimited on the paid plan.

Troubleshooting

• "Streamable HTTP transport may not be supported": the endpoint answered with HTML or a non-JSON content type. Check that the URL is the MCP endpoint itself (commonly /mcp), not a documentation page or an SSE-only path.
• Handshake fails with a valid endpoint: the response must be a JSON-RPC success carrying protocolVersion and capabilities. Yorker sends Accept: application/json on every request; a server that insists on text/event-stream in the Accept header before answering will fail at this phase. A server that rejects the initialize payload fails with the JSON-RPC error surfaced in the result.
• HTTP 401/403: the server requires auth. Add an auth block; for a bearer-token server, use type: bearer with the token stored as a team secret.
• Expected tool missing intermittently: some servers assemble their catalog from downstream services. Pair expectedTools with a multi-location alert rule so a single flaky assembly does not page you.