API Specs

If you publish an OpenAPI spec, Yorker can generate one HTTP monitor per operation in a single call and add an openapi_conformance assertion to each one. Every check then validates the live response against the spec on every run, catching drift the moment it ships rather than the moment a customer hits it.

This guide covers the full workflow: registering a spec, generating monitors from it, keeping the spec in sync, and updating or deleting it. For the underlying endpoint contracts, see REST API > API Specs.

When to use this

Spec-driven monitor generation pays off when:

• The API has more than a handful of operations and you do not want to hand-author one HTTP check per route.
• Response shapes are part of the contract and you want a regression alert when they diverge.
• The API definition is the source of truth and you want monitors to follow it instead of drifting independently.

If you only need a couple of status_code: 200 checks against known URLs, the Create a Monitor flow is faster.

Two spec workflows

Both produce the same outcome (one HTTP monitor per operation, all carrying the openapi_conformance assertion). The split is whether you keep a spec entity in Yorker that you can re-sync later, or treat the spec as a one-shot input to the generator.

Both workflows share the same generation pipeline, so idempotency rules, the confirmation gate, the skip-reason vocabulary, and the per-check shape are identical. The response JSON is not byte-for-byte identical: POST /api/checks/generate adds a top-level mode: "spec" discriminator and a spec.newlyCreated flag (since it can also create the spec entity), neither of which the per-spec endpoint emits.

Workflow 1: Persistent spec entity

Register the spec

Two creation modes are available: upload the raw bytes, or point Yorker at a URL it can fetch on demand. The choice affects how you sync later.

Upload mode

curl -X POST https://yorkermonitoring.com/api/specs \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Petstore API",
    "sourceType": "upload",
    "content": "{\"openapi\":\"3.0.0\",\"info\":{\"title\":\"Petstore\",\"version\":\"1.0.0\"},\"servers\":[{\"url\":\"https://petstore3.swagger.io/api/v3\"}],\"paths\":{\"/pet/{petId}\":{\"get\":{\"operationId\":\"getPetById\",\"parameters\":[{\"name\":\"petId\",\"in\":\"path\",\"required\":true,\"schema\":{\"type\":\"integer\"}}],\"responses\":{\"200\":{\"description\":\"OK\"},\"404\":{\"description\":\"Not found\"}}}}}}"
  }'

content accepts JSON or YAML up to 4 MB. The response is the spec summary including the assigned id (spec_xxx) and the SHA-256 contentHash. The hash is computed over the dereferenced, sorted-keys JSON of the parsed spec, so a re-formatted YAML version of the same spec hashes identically to the JSON version.

To re-sync after an upload-mode spec changes, send PUT /api/specs/:id with a new content string. Yorker re-parses, re-hashes, and updates lastSyncedAt.

URL mode

curl -X POST https://yorkermonitoring.com/api/specs \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Petstore API",
    "sourceType": "url",
    "sourceUrl": "https://petstore3.swagger.io/api/v3/openapi.json"
  }'

URL mode is the right path when the spec is already published somewhere reachable on the public internet. Yorker fetches and parses on registration, then on every POST /api/specs/:id/sync call. The fetcher is SSRF-guarded: it rejects non-HTTP(S) schemes, URLs with embedded credentials, and any URL whose hostname resolves (or is) a private/reserved IP address (RFC1918, loopback, link-local, IMDS, etc.). Plain http:// is allowed but only for publicly routable hosts; HTTP redirects are refused entirely (so an attacker-controlled redirect chain cannot pivot to a private IP). Responses are capped at 10 MB.

The Yorker dashboard exposes the same endpoint via the API Specs panel on the Monitors page. If you prefer a UI, register the spec there and use the API only for the check-generation step.

Generate checks from the registered spec

curl -X POST https://yorkermonitoring.com/api/specs/spec_abc123/generate-checks \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "locations": ["loc_us_east", "loc_eu_central"],
    "frequencySeconds": 300,
    "validateHeaders": false,
    "confirm": false
  }'

The pipeline walks every operation in the spec, derives the canonical URL from the spec's servers block + the operation's path template, and inserts one HTTP check per eligible operation with the openapi_conformance assertion attached.

A successful first run returns 201 with the catalog of created checks:

{
  "spec": { "id": "spec_abc123", "name": "Petstore API" },
  "created": [
    { "id": "chk_xyz", "name": "Petstore API: GET /pet/{petId}", "operationKey": "GET /pet/{petId}" }
  ],
  "skipped": [],
  "summary": {
    "operationsInSpec": 1,
    "eligible": 1,
    "created": 1,
    "skipped": 0,
    "labelAttachmentFailures": 0
  }
}

Each generated check name is "{spec name}: {operation summary}" when the operation has a summary, falling back to "{spec name}: {METHOD} {pathTemplate}". Names longer than 255 characters are truncated with a trailing ....

Idempotency and re-runs

The generator is idempotent on (specId, method, pathTemplate): a re-run after a deploy that added two new operations creates only those two and skips the others as already_exists. Re-running an unchanged spec returns 200 (not 201) with created: [] and every operation in the skipped array.

The full skip-reason vocabulary:

Confirmation gate

When a spec yields more than 50 new operations in a single call, the route returns 409 with a confirmation envelope:

{
  "error": "This spec would create 87 new monitors. Confirm to create all of them.",
  "requiresConfirmation": true,
  "operationCount": 87,
  "threshold": 50
}

Re-submit the same body with confirm: true to proceed. Existing operations counted as already_exists do not contribute to the threshold, so re-running an 87-operation spec after the first batch landed does not require re-confirmation.

Sync the spec

URL-mode specs re-fetch on demand:

curl -X POST https://yorkermonitoring.com/api/specs/spec_abc123/sync \
  -H "Authorization: Bearer sk_..."

The response includes a changed: boolean indicating whether the new contentHash differs from the stored one. Yorker only writes a new contentJson when the hash changes (avoiding write amplification on periodic syncs).

When the hash changes, runners invalidate their cached copy on the next poll. The control plane injects the current specContentHash field onto each openapi_conformance assertion at runner-poll time; the runner uses that value to decide whether to re-fetch the spec, so you do not need to re-deploy or re-generate checks.

Sync only works on URL-mode specs. Calling sync on an upload-mode spec returns 400. Update an upload-mode spec by PUT-ing new content to /api/specs/:id.

Update spec metadata or content

# Rename the spec
curl -X PUT https://yorkermonitoring.com/api/specs/spec_abc123 \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{ "name": "Petstore v2" }'

# Replace the content (also switches the spec to upload mode)
curl -X PUT https://yorkermonitoring.com/api/specs/spec_abc123 \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{ "content": "{ ... new spec body ... }" }'

# Switch from upload mode to URL mode
curl -X PUT https://yorkermonitoring.com/api/specs/spec_abc123 \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{ "sourceUrl": "https://api.example.com/openapi.json" }'

content and sourceUrl are mutually exclusive in a single request. Yorker normalizes the sourceType on the row to match whichever one you sent.

Delete the spec

curl -X DELETE https://yorkermonitoring.com/api/specs/spec_abc123 \
  -H "Authorization: Bearer sk_..."

Returns 204 No Content on success. Generated monitors are not deleted or auto-detached: their openapi_conformance assertion still references the (now-missing) spec ID. The runner's in-memory spec cache is best-effort (5-minute TTL, 50-entry LRU cap, and specs over 2 MB are not cached at all), so the assertion may keep validating against a cached spec for up to 5 minutes after deletion before the next miss. Specs that were never cached (size > 2 MB) or that have already been LRU-evicted will fail on the very next run, when the runner's getSpec call returns 404. Either way, other assertions on the same check (status code, response time, body) keep evaluating normally throughout. Re-creating a spec with the same ID is not supported (IDs are unique), so the cleanest path after a delete is to create a new spec and either re-generate checks against it or update the existing checks' assertions to point at the new ID. Wiring up automatic detach is tracked in issue #583.

Workflow 2: One-shot generation via /api/checks/generate

When the integration already knows the spec source and does not want to manage the spec entity in two API calls, POST /api/checks/generate accepts a spec reference inline.

# By URL: fetches, dedupes against existing specs by content hash, generates
curl -X POST https://yorkermonitoring.com/api/checks/generate \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "spec": {
      "source": "url",
      "specUrl": "https://petstore3.swagger.io/api/v3/openapi.json",
      "name": "Petstore API"
    },
    "locations": ["loc_us_east", "loc_eu_central"],
    "frequencySeconds": 300,
    "validateHeaders": false,
    "confirm": false
  }'

# By existing spec ID
curl -X POST https://yorkermonitoring.com/api/checks/generate \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "spec": { "source": "id", "specId": "spec_abc123" },
    "locations": ["loc_us_east"]
  }'

# By spec name (looked up via the team-unique (teamId, name) index)
curl -X POST https://yorkermonitoring.com/api/checks/generate \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "spec": { "source": "name", "specName": "Petstore API" },
    "locations": ["loc_us_east"]
  }'

URL mode dedupes by content hash: if a spec with the same hash already exists for the team, Yorker reuses the existing row instead of creating a duplicate. The response wraps the same payload as the per-spec endpoint with a mode discriminator and a newlyCreated flag so callers can tell whether the spec entity was just inserted:

{
  "mode": "spec",
  "spec": { "id": "spec_abc123", "name": "Petstore API", "newlyCreated": true },
  "created": [ ... ],
  "skipped": [ ... ],
  "summary": { ... }
}

newlyCreated: false means the URL resolved to an already-registered spec on this team. mode: "spec" is always present on this endpoint when the body contains a spec field; without it, the same endpoint runs the Playwright script-generation flow and returns mode: "playwright" instead.

When URL mode creates a new spec, the name field on the request is the display name. If you omit it, Yorker derives it from the spec's info.title, falling back to api-{hostname} derived from the URL, and finally to the literal api-spec if neither is available. Only the omit-name path applies a numeric collision suffix (base, base-2, base-3, ...) so repeat URL-mode runs are idempotent. If you pass name explicitly and it collides with a different-hash spec already on the team, the route returns 409 instead.

How openapi_conformance validates

Each generated check carries the assertion:

{
  "type": "openapi_conformance",
  "specId": "spec_abc123",
  "operationPath": "GET /pet/{petId}",
  "validateHeaders": false
}

The generator always writes operationPath so the runner does not have to re-derive it from the check URL on every execution. Hand-authored assertions can omit operationPath and the runner will auto-detect by matching the check URL's (method, path) against the spec's path templates.

On every run, the runner:

1. Fetches the spec by ID from its in-memory cache (refreshed when the control plane signals a specContentHash change).
2. Resolves the operation: the pinned operationPath for generated checks, or auto-detection from the check URL for hand-authored assertions that omitted it.
3. Validates the response status code against the operation's declared responses block.
4. Validates the response body shape against the matching response schema.
5. If validateHeaders: true, also validates response header presence and values.

A mismatch in any of those steps fails the assertion and produces a failed check run. See Assertions > openapi_conformance for the full assertion field reference.

CLI workflow

There is no yorker specs command today. Use curl (or your preferred HTTP client) for spec registration and generation, and the API Specs panel on the Monitors dashboard page for visual management. Generated checks appear in yorker monitors list and can be edited, paused, or deleted with the standard yorker monitors edit, yorker monitors pause, and yorker monitors delete commands.

Errors

Related

• REST API > API Specs: full endpoint contracts.
• Assertions > openapi_conformance: the assertion field reference.
• Create a Monitor: for hand-authored HTTP checks when spec-driven generation is overkill.