Create a Monitor

Yorker supports three monitor types:

• HTTP: sends an HTTP request and asserts on the response (status code, body, headers, timing, SSL expiry, OpenAPI conformance).
• Browser: runs a real Chromium session, either against a URL (automatic Core Web Vitals + screenshots) or scripted with Playwright TypeScript.
• MCP: exercises a Model Context Protocol server over Streamable HTTP, validates the advertised tool list, and (optionally) calls tools with expected output.

There are three ways to create monitors: the Web UI, the CLI (monitoring as code), and natural-language generation.

Web UI

To create a monitor through the dashboard:

1. Navigate to the dashboard and click Create Monitor.
2. Select the monitor type (HTTP, Browser, or MCP).
3. For HTTP monitors, fill in the URL, method, headers, authentication, and assertions.
4. For browser monitors, choose URL mode (just a URL) or Scripted mode and write a Playwright TypeScript script in the built-in editor.
5. For MCP monitors, enter the endpoint and optionally list expected tools and test calls.
6. Choose check locations and frequency.
7. Click Create.

CLI / Monitoring as Code

To manage monitors as code, define them in yorker.config.yaml and deploy with yorker deploy.

HTTP monitor

project: my-app
monitors:
  - name: API Health
    type: http
    url: https://api.example.com/health
    method: GET
    frequency: 1m
    locations:
      - loc_us_east
      - loc_eu_central
    timeoutMs: 10000
    followRedirects: true
    headers:
      Accept: application/json
    labels:
      - env:production
      - service:api
    assertions:
      - type: status_code
        operator: equals
        value: 200
      - type: response_time
        max: 2000
      - type: body_contains
        value: '"status":"ok"'
      - type: body_json_path
        path: $.version
        operator: exists
      - type: header_value
        header: content-type
        operator: contains
        value: application/json
      - type: ssl_expiry
        daysBeforeExpiry: 14

Agent personas

To check that the same URL responds to AI crawlers and agents, add an agentPersonas block to an HTTP monitor. Each run can repeat the request as the selected personas and report any that are blocked, challenged, or served different content.

    agentPersonas:
      personas:
        - claude-user
        - chatgpt-user
      every: 6

See Monitor AI Agent Access for verdicts, sampling, cost and alerting.

Agent readiness

To monitor what AI agents look for on the site itself (robots.txt rules, llms.txt, a Markdown version of the page, discovery files), add an agentReadiness block to an HTTP monitor.

    agentReadiness:
      every: 12
      require:
        - llms_txt

See Check Agent Readiness for the probes, the score and alerting.

HTTP authentication

Three auth types are supported. Add an auth block to any HTTP monitor:

# Basic auth
auth:
  type: basic
  username: "{{secrets.API_USER}}"
  password: "{{secrets.API_PASS}}"

# Bearer token
auth:
  type: bearer
  token: "{{secrets.AUTH_TOKEN}}"

# API key header (defaults to X-API-Key)
auth:
  type: api-key
  header: X-API-Key
  value: "{{secrets.API_KEY}}"

Assertion types

See the Assertions reference for full details.

Browser monitor: URL mode

The simplest browser monitor: Yorker navigates to the URL, captures Core Web Vitals, takes screenshots, and runs assertions. No script to write.

URL-mode browser monitors are currently created via the Web UI or the REST API. yorker deploy is scripted-only for browser monitors. Create URL-mode monitors from the dashboard, or via POST /api/checks with browserConfig.browserMode: "url". See the REST API reference.

Browser monitor: scripted mode

Scripted browser monitors run full Playwright TypeScript scripts. Reference the script file from your config:

monitors:
  - name: Checkout Flow
    type: browser
    script: ./monitors/checkout.ts
    frequency: 5m
    locations:
      - loc_us_east
    viewport:
      width: 1280
      height: 720
    screenshotMode: every_step
    videoEnabled: false

The script file (./monitors/checkout.ts) can take either of two forms:

• A script body. Write the body of your check directly, referencing page (a Page) and context (a BrowserContext) as if they were already in scope. The Yorker runner wraps the script in an async function and injects both. Do not add other import or require statements; a script body has no module scope.
• A standard @playwright/test spec. A file that imports test and expect from @playwright/test and wraps the journey in test("name", async ({ page }) => { ... }) runs as it is, so an existing end-to-end test can double as a monitor.

In both forms, use // @step: Name comments to mark steps. Yorker captures a screenshot at each step (when screenshotMode: every_step) and surfaces step timing in the filmstrip view.

// @step: Go to shop
await page.goto("https://shop.example.com");

// @step: Add to cart
await page.click("text=Add to Cart");

// @step: Checkout
await page.click("text=Checkout");
await page.waitForSelector(".order-confirmation");

Browser configuration options (scripted mode, YAML)

MCP monitor

MCP monitors check the health of a Model Context Protocol server over Streamable HTTP. On each run, Yorker executes the full JSON-RPC session lifecycle the way a real MCP client does:

1. Sends initialize and asserts the response carries protocolVersion and capabilities, capturing any Mcp-Session-Id for the rest of the session.
2. Sends the notifications/initialized notification.
3. Calls tools/list and verifies every tool in expectedTools is present (if configured).
4. Runs each testCalls entry: invokes the tool and checks the result contains expectedOutputContains (if provided).
5. Optionally detects drift: fingerprints each tool's input schema, description and annotations and the server's reported identity, and reports what changed since the previous run, including annotation hints that became less safe.
6. Optionally checks that the server rejects an initialize request sent with no credentials.

A server that returns HTTP 200 but cannot complete the handshake fails the check with the phase that broke, not a generic error. Yorker speaks the Streamable HTTP transport; if the server responds with HTML or a non-JSON content type, the result says the Streamable HTTP transport may not be supported.

monitors:
  - name: Docs MCP Server
    type: mcp
    endpoint: https://mcp.example.com/mcp
    frequency: 5m
    timeoutMs: 30000
    locations:
      - loc_us_east
    auth:
      type: bearer
      token: "{{secrets.MCP_TOKEN}}"
    expectedTools:
      - search_docs
      - fetch_page
    testCalls:
      - toolName: search_docs
        arguments:
          query: "pricing"
        expectedOutputContains: "Plans"
    detectSchemaDrift: true

MCP configuration options

Note: MCP monitors cannot be executed locally with yorker test; deploy them and watch results via yorker results tail or the dashboard.

Labels

Attach labels to any monitor. Labels serve two purposes: filtering and grouping in the dashboard, and emission as OTel resource attributes so you can slice telemetry by label in your observability backend.

monitors:
  - name: Payments API
    type: http
    url: https://api.example.com/payments
    labels:
      - env:production      # key:value label
      - service:payments
      - critical            # boolean label (becomes yorker.label.critical="true")

Labels follow the convention [a-zA-Z0-9][a-zA-Z0-9_.:-]*, max 128 characters. Plain labels (no colon) emit as yorker.label.<name>="true". Key-value labels emit as yorker.label.<key>="<value>".

Omitting the labels field leaves labels unmanaged by config; the CLI preserves whatever labels exist on the remote. Setting labels: [] explicitly clears all labels.

Defaults and groups

To avoid repeating configuration across monitors, use defaults and groups.

Defaults apply to all monitors unless overridden:

defaults:
  frequency: 5m
  locations:
    - loc_us_east
    - loc_eu_central
  http:
    timeoutMs: 15000
    followRedirects: true
    assertions:
      - type: status_code
        value: 200
  browser:
    viewport:
      width: 1280
      height: 720
    screenshotMode: every_step

Groups apply shared settings to a subset of monitors:

groups:
  - name: API Endpoints
    frequency: 1m
    locations:
      - loc_us_east
      - loc_us_west
      - loc_eu_central
    monitors:
      - name: Users API
        type: http
        url: https://api.example.com/users
      - name: Orders API
        type: http
        url: https://api.example.com/orders

The cascade order is: defaults -> group -> monitor. Each level overrides the previous.

Per-monitor assertions replace defaults entirely (they do not merge). To clear inherited assertions, set assertions: [] on the monitor.

Frequency format

Frequencies use a shorthand: 30s (seconds), 5m (minutes), 1h (hours). Valid range: 10 seconds to 24 hours.

CLI imperative commands

For quick one-off monitors, use the monitors commands instead of a config file:

Create a monitor

yorker monitors create --name "API Health" --type http --url https://api.example.com/health --frequency 1m

List monitors

yorker monitors list
yorker monitors list --type http --status enabled

View monitor details

yorker monitors get "API Health"

Edit, pause, and resume

yorker monitors edit "API Health" --frequency 30s --add-location loc_eu_central
yorker monitors pause "API Health"
yorker monitors resume "API Health"

Delete a monitor

yorker monitors delete "Old Endpoint" --yes

See the CLI reference for the full list of monitor commands and flags.

Natural language

To create a monitor using natural language, use either the Web UI or the API.

Web UI: Click Describe in plain English on the create monitor page and type a description like "Monitor our checkout flow every 2 minutes from US and EU, alert if it takes longer than 3 seconds." Yorker generates a Playwright script you can edit before saving.

API: Send a POST request to /api/checks/generate with a description. The endpoint returns a generated Playwright script you can review, optionally refine, and save as a browser monitor:

curl -X POST https://yorkermonitoring.com/api/checks/generate \
  -H "Authorization: Bearer $YORKER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "description": "Navigate to https://shop.example.com, add the first product to cart, and verify the cart shows it.",
    "targetUrl": "https://shop.example.com"
  }'

You can also pass previousScript and refinement to iteratively improve an existing script. See the REST API reference.

Generate HTTP checks from an OpenAPI spec

/api/checks/generate also accepts a spec field. Pass an existing spec ID, an OpenAPI URL Yorker should fetch, or the name of a spec already on your team. Yorker creates one HTTP check per operation, skips operations that already have a check, and returns the full list. This is the API equivalent of the spec import flow:

curl -X POST https://yorkermonitoring.com/api/checks/generate \
  -H "Authorization: Bearer $YORKER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "spec": { "source": "url", "specUrl": "https://api.example.com/openapi.json" },
    "locations": ["us-east", "eu-west"],
    "frequencySeconds": 300
  }'

The spec field is a discriminated union: pass {"source": "id", "specId": "spec_..."} to use an existing spec, {"source": "url", "specUrl": "https://..."} to fetch and (if needed) import a new one, or {"source": "name", "specName": "..."} to look up by name. If the spec yields more than 50 new operations the first call returns 409 with requiresConfirmation: true; re-submit with "confirm": true to proceed. See the Generate Checks From OpenAPI Spec reference for the full request/response shape and error codes.