---
title: 'Five shifts reshaping synthetic monitoring'
description: 'Synthetic monitoring''s four eras got it to OTel-native, portable telemetry. What''s happening now is different: who writes the check, who writes the code, where testing ends, who the user is, and who reads the output are all changing at once.'
date: '2026-09-22'
author: 'Drew Post'
tags: ['synthetic-monitoring', 'perspective', 'ai-infrastructure', 'opentelemetry', 'ai-sre']
canonical_url: 'https://yorkermonitoring.com/blog/five-shifts-reshaping-synthetic-monitoring'
---

![Long-exposure light trails from traffic moving through a city at night.](/blog/five-shifts-reshaping-synthetic-monitoring/00-hero.jpg)

In May I wrote about [synthetic monitoring's four eras](/blog/synthetic-monitoring-is-changing): uptime pings, scripted journeys, one signal locked inside a platform, and now backend-portable OTLP with no vendor tax. That post was about where the telemetry goes. In the four months since, something underneath it has moved faster than I expected: who sits on either end of that telemetry has changed, on both sides, for reasons that have nothing to do with OTel.

Five things are different, and I see them in the check configs and CI pipelines I look at every week rather than in some future-of-testing forecast. I build Yorker, so weigh what follows accordingly, but the shifts hold whether or not you ever use it.

![A then-versus-now comparison across the five shifts: checks moved from hand-written to generated, the code under test moved from human-written to model-written, e2e tests and monitors merged into one Playwright script, users widened to include agents, and the reader of the output widened to AI-SRE tools and deploy gates, while the underlying question stayed the same.](/blog/five-shifts-reshaping-synthetic-monitoring/01-five-shifts-then-now.svg)

## Shift one: the author changed

Most of the synthetic checks I see set up now aren't typed by hand. They're generated from a plain-English description of a flow, from an OpenAPI spec, or by a coding agent editing a YAML config next to the code it just wrote. I covered the mechanics of those paths in [five ways to create a synthetic test](/blog/five-ways-to-create-a-synthetic-test), so I won't repeat them here.

None of that is bad. Generation is faster and it lowers the bar to having any coverage at all, which used to be the real failure mode: nobody wrote the check. But once the check is generated, typing speed stops being the bottleneck and judgment becomes the scarce skill. Does the generated assertion test the thing that matters, or the thing that was easiest to assert on? Did a spec-derived check inherit a contract that's already stale? The work moves from writing checks to reviewing them, the same way code review outlasted code writing. Teams that treat generation as "now I don't have to think about it" end up with wide, shallow coverage that looks reassuring on a dashboard and catches almost nothing.

## Shift two: the code changed

The thing being checked changed too. I wrote about this at length in [synthetic monitoring matters more when agents write the code](/blog/synthetic-monitoring-when-agents-write-the-code): when the same model writes an implementation and its test in one pass, the test becomes a restatement of whatever the model already believed. Green CI still means something. It means less than it used to.

What I didn't dwell on there is the effect on things that aren't correctness bugs at all. A coding agent optimizes for what it can verify in its own loop: the build passes, the test it wrote goes green, the page renders. It gets no feedback on whether the page renders *fast*, because nothing in a typical agent harness measures Largest Contentful Paint. I went into this in [your coding agent doesn't care about LCP](/blog/your-coding-agent-doesnt-care-about-lcp). Performance regressions are a natural casualty of generated frontends because performance was never in the loss function, and agents writing UI code badly has little to do with it.

That pattern generalizes beyond performance. An oracle that sits outside the process that wrote the code, with no access to its reasoning and no stake in its self-consistency, catches a whole category of regression that an agent checking its own work structurally can't: config that's correct in the diff and wrong in the deployed environment, a CDN cache-control header that flipped, Web Vitals nobody asked the agent to optimize for. The more of your codebase a model writes, the more this outside-in read matters.

## Shift three: the boundary with testing is dissolving

For most of the last decade, end-to-end tests and synthetic monitors did overlapping jobs with different tools. QA ran a Selenium or Cypress suite against staging before release; a separate monitoring product, with its own recorder or scripting format, watched production. Two configs and two runtimes meant two places to keep a login flow up to date.

That split made sense when monitoring vendors ran proprietary scripting languages. It makes much less sense once both sides are vanilla Playwright. I laid out that convergence in [end-to-end tests and synthetic monitors are converging](/blog/end-to-end-tests-and-synthetic-monitors-are-converging): the journey you test in CI against a preview deploy is the journey you want watched in production, written once. It doesn't collapse into one thing. Side effects, destructive paths, cost, flake tolerance and vantage point still have to differ, and the monitor needs its own guardrails. But the steps and selectors can be shared.

In practice, stop treating "ship the feature" and "add the monitor" as separate backlog items. If the checkout flow got an e2e test in this PR, the production monitor for that flow is the same journey, ported into the monitor and deployed with the code, instead of a follow-up ticket that quietly never happens. I wrote about making that literal in [ship the monitor with the pull request](/blog/ship-the-monitor-with-the-pull-request): the monitor config lives in the same diff as the feature, is reviewed by the same person, and merges in the same commit.

## Shift four: the user changed

More and more of the traffic hitting your API has no browser or human behind it. It's an agent, calling your endpoints directly or through an MCP server and making decisions based on whatever comes back. I wrote about what that does to the definition of "working" in [when your user is an agent](/blog/when-your-user-is-an-agent).

An HTTP 200 was always a weak signal for a human user. For an agent it's weaker still, because agents tolerate small contract drift far less than a person clicking around a slightly broken UI. A renamed field, a schema that gained a required parameter, a tool that silently vanished from a catalog: a human notices something's off and works around it. An agent following a fixed schema often just breaks, or calls the wrong tool with the wrong arguments and tells nobody.

So protocol-level checking matters now in a way it didn't three years ago. For an HTTP API, that means asserting against the actual contract instead of a hand-picked set of fields someone thought to check. For an MCP server, it means running the real session on a schedule (handshake, tool discovery, tool calls) with drift detection on what agents actually depend on: the tool schemas. The deep dive on why an uptime check misses all of this is [MCP server monitoring: why an HTTP 200 isn't enough](/blog/mcp-server-monitoring). If agents are a real fraction of your traffic, the agent-facing surface deserves the same outside-in scrutiny as the one your humans use.

## Shift five: the consumer changed

The last shift is on the output side. For twenty years the only thing reading synthetic monitoring output was a person at a dashboard, so every product in the category optimized for that: a status page, a chart, a red dot. Now the reader is increasingly a machine: an agentic RCA tool building a causal model of an incident, or a CI pipeline deciding whether a deploy is safe to keep rolling.

I covered the first case in [the missing input to your AI-SRE tool](/blog/the-missing-input-to-your-ai-sre-tool). A green dot has no attribution, a bare latency number has no join key, and a raw network waterfall is a context-window problem for the model reading it. A causal engine needs the analysis done before the signal reaches it. The second case, the gate instead of the dashboard, is in [synthetic checks as the deploy verification gate](/blog/synthetic-checks-as-the-deploy-verification-gate): the question a deploy pipeline is really asking is whether the deployed thing works right now, and answering that is a synthetic check's job, since a unit test never touches the deployed thing.

Both cases need the same property: output that is structured and machine-consumable by default, instead of a dashboard that happens to export OTLP. That means a conclusion with a join key attached, portable into whatever backend you run, because the machine reading it downstream belongs to you and not to the monitoring vendor.

![The five shifts as one pipeline: a check written by a person, a description, a spec or a coding agent runs against a deployed system built by people and models and used by people and agents, then emits enriched OTLP into your own backend, where a human dashboard, an AI-SRE tool or a deploy gate reads it.](/blog/five-shifts-reshaping-synthetic-monitoring/02-shifts-pipeline.svg)

## What hasn't changed

Under all of this, the question at the bottom of the stack is the one it's always been: can a real user, starting from where they actually are, complete the journey? That means landing on your site, logging in, searching, and checking out, over a real network, through the real CDN and the real auth proxy, right now. A function returning the right value doesn't answer it, and neither does a passing unit test. Every shift here changes who asks that question and who answers it. The question still has to be asked from outside the system, on a schedule, against the deployed thing, and no amount of AI on either end of the pipeline makes that optional.

## Where we're placing our bets

Here's what that means for what Yorker does today, shift by shift.

For the author changing, there are three creation paths. Describe a flow in plain English and get back a real Playwright script, with a `previousScript` and a `refinement` field so you iterate on what was generated instead of starting over. Point at an OpenAPI spec and it generates one HTTP check per operation, each carrying an `openapi_conformance` assertion tied back to the spec. Or write `yorker.config.yaml` by hand, or let your coding agent write it, and review the change the way you'd review any infrastructure diff:

```bash
$ yorker diff
Yorker deploy plan for "storefront"

  Checks:
    ~ UPDATE  browser  "checkout-flow"
        ~ frequencySeconds  300 → 120
    + CREATE  http  "payments-api-contract"  (300s, 2 locations)
    + CREATE  mcp   "docs-mcp-server"  (300s, 1 locations)

  Summary: 2 to create, 1 to update

$ yorker deploy
  Deploy complete: 2 created, 1 updated
```

For the user changing, MCP is a first-class, generally available check type. It runs the real session (the `initialize` handshake, `tools/list`, `tools/call`) and hashes each tool's input schema run over run, so drift shows up within minutes instead of when an agent starts misbehaving in production.

For the consumer changing, every check emits standard OTLP into your own backend. Browser checks carry third-party domain attribution, the run's completion event carries anomaly scores against per-monitor baselines, and a W3C `traceparent` is injected into every request so the check and the backend trace it touches share a join key. A causal engine or a deploy gate reading your backend gets a conclusion it can act on.

![A HyperDX log-event detail for a synthetics.check.completed event with the anomaly deviation, the anomalous flag and SLO budget context attached to the event itself.](/blog/synthetic-monitoring-is-changing/08-clickstack-anomaly-attrs-on-event.png)

I'll be equally direct about what isn't built yet. There are no self-healing selectors, no session-replay-to-monitor, no visual regression, no WebKit engine, no network-condition simulation, and no multi-step API test chaining beyond one check per OpenAPI operation. Web Vitals capture covers LCP, FCP, CLS, and TTFB, but not INP, and there's no built-in fixed-threshold LCP assertion type; if you want to fail a check on LCP, you assert it yourself inside your Playwright script. If any of those are why you're evaluating a tool, say so before you sign up.

![Yorker's status against the five shifts: nine shipped capabilities drawn solid in teal, from plain-English generation and OpenAPI checks to MCP drift detection, OTLP into your backend, attribution, anomaly scores, traceparent and four Web Vitals, beside eight not-yet items drawn dashed, including self-healing selectors, visual regression, WebKit, INP and a built-in LCP threshold assertion.](/blog/five-shifts-reshaping-synthetic-monitoring/03-built-vs-not-yet.svg)

The bet underneath all five shifts is the same: as more of the writing and the reading around your systems gets automated, the thing that stays valuable is an outside-in check that nobody inside the loop can quietly make pass.

[Start free, no credit card required →](/sign-up)