The Two Weeks My Web Monitor Said Everything Was Fine — Field Notes on Catching Silent Misses

Anchor the Target — Measure Whether Extraction Is Alive

To measure whether extraction is alive, I defined an "anchor" for each target. An anchor is a landmark that must exist if the page was read correctly. For a pricing page: "at least three numbers carrying a currency symbol," "at least two things that look like plan names." Expectations rooted in content, not in structure.

If the anchors are not satisfied, the crawl is treated as an extraction failure even on an HTTP 200. It never advances to the diff.

// anchor-check.mjs — verify extraction is alive using content, not structure
// Goal: surface "empty successes" caused by selector misses as faults
 
const ANCHORS = {
  'saas-pricing': [
    { name: 'price-tokens', test: (t) => (t.match(/[$¥€]\s?\d[\d,]*/g) || []).length >= 3 },
    { name: 'plan-headings', test: (t) => (t.match(/\b(Free|Pro|Team|Business|Enterprise)\b/g) || []).length >= 2 },
    { name: 'min-length',    test: (t) => t.replace(/\s+/g, '').length >= 400 },
  ],
};
 
export function checkAnchors(siteId, text) {
  const rules = ANCHORS[siteId] || [];
  const results = rules.map((r) => ({ name: r.name, passed: r.test(text) }));
  const failed = results.filter((r) => !r.passed).map((r) => r.name);
  return {
    // extraction is "alive" only when every anchor passes
    alive: failed.length === 0,
    failed,
    coverage: rules.length ? (rules.length - failed.length) / rules.length : 0,
  };
}

The important choice here is writing anchors in terms of content (regexes, text length) rather than structure (CSS selectors). Structure changes whenever the competitor feels like it. But the property that "a pricing page carries numbers with currency symbols" is far more stable. Anchoring the liveness check to something the target cannot control is what makes faults stand up and announce themselves.

On the crawl side, this check sits just before the diff.

// crawl-guard.mjs — a crawl that fails its anchors never reaches the diff
import { checkAnchors } from './anchor-check.mjs';
 
export function ingest(siteId, text, { onExtractionFailure, onAlive }) {
  const anchor = checkAnchors(siteId, text);
  if (!anchor.alive) {
    // treat this as "the monitor is blind," not "no changes"
    onExtractionFailure({ siteId, failed: anchor.failed, coverage: anchor.coverage });
    return { proceeded: false };
  }
  onAlive({ siteId, coverage: anchor.coverage });
  return { proceeded: true };
}

That one layer turned a silence like my two weeks into an unmistakable anomaly — "extraction failed 14 times in a row" — floating at the very top of the log.

Treat "Too Quiet" as an Anomaly

Anchors alone did not let me relax. Extraction can be alive and the comparison logic can still drop something. So I decided to watch from the other side too: to treat "unchanged for far too long" as a weak alert in its own right.

Real competitor sites move, to varying degrees. Copy tweaks, updated dates, campaigns coming and going. A page that has not changed a single character in weeks is more likely a page you can no longer see than a page that truly stood still. In monitoring, perfect silence is often not health but paralysis.

// quietness-watch.mjs — flag a long streak of zero changes as "too quiet"
export const QUIET_THRESHOLD_DAYS = {
  'saas-pricing': 30,     // pricing moves slowly, so allow a long stretch
  'competitor-blog': 5,   // a blog should move often, so keep it short
};
 
export function evaluateQuietness(siteId, changeHistory) {
  // changeHistory: [{ date, changed }] in ascending date order
  let streak = 0;
  for (let i = changeHistory.length - 1; i >= 0; i--) {
    if (changeHistory[i].changed) break;
    streak++;
  }
  const threshold = QUIET_THRESHOLD_DAYS[siteId] ?? 14;
  return {
    quietStreak: streak,
    suspicious: streak >= threshold,
    // "suspicious" is not a verdict of failure; it just sends a human to look
    action: streak >= threshold ? 'manual-spotcheck' : 'none',
  };
}

This alert does not declare anything broken. It only says "this is unnaturally quiet — please check it with human eyes once." In practice it fires maybe once a month, but when it does, there is usually either extraction decay or a miss on my side behind it. A weak signal close to a false positive costs almost nothing at low frequency, and in exchange it reliably lowers the risk of silence.

Catch Selector Drift with a Structure Fingerprint

Anchors are strong when extraction dies completely, but blunt when only part of it slips — two of three prices read, one dropped. To catch that half-degraded state, I started taking a fingerprint of the page structure itself on every crawl and tracking how it moves.

A structure fingerprint is a short summary of the "shape" of the elements wrapping the prices: the set of class names, element depth, sibling counts — the structural features the extraction stands on. It reacts when the footing changes, not the values.

// structure-fingerprint.mjs — track whether the extraction's footing shifted
// Run on the target page via Claude in Chrome's javascript_tool
export function fingerprintScript() {
  return `JSON.stringify((() => {
    const nodes = Array.from(document.querySelectorAll('[class*="price"], [class*="plan"], [data-price]'));
    return {
      count: nodes.length,
      classes: [...new Set(nodes.flatMap(n => Array.from(n.classList)))].sort(),
      depths: nodes.map(n => { let d = 0, e = n; while (e.parentElement) { d++; e = e.parentElement; } return d; }),
      hasDataPrice: nodes.some(n => n.hasAttribute('data-price')),
    };
  })())`;
}
 
// compare last vs current fingerprint and quantify the footing shift
export function compareFingerprint(prev, curr) {
  if (!prev) return { drift: 0, notes: ['baseline'] };
  const notes = [];
  if (prev.count !== curr.count) notes.push(`node count ${prev.count} -> ${curr.count}`);
 
  const prevClasses = new Set(prev.classes);
  const currClasses = new Set(curr.classes);
  const removed = [...prevClasses].filter((c) => !currClasses.has(c));
  const added = [...currClasses].filter((c) => !prevClasses.has(c));
  if (removed.length) notes.push(`removed classes: ${removed.join(', ')}`);
  if (added.length) notes.push(`added classes: ${added.join(', ')}`);
 
  // drift: symmetric difference of class sets normalized by the union (0=stable, 1=fully swapped)
  const union = new Set([...prevClasses, ...currClasses]).size || 1;
  const drift = (removed.length + added.length) / union;
  return { drift: Number(drift.toFixed(2)), notes };
}

In practice I set a threshold on this drift. Above 0.3, it raises a medium-priority notice that "the selectors may need review." Keeping a value change (a price went up) and a footing change (class names changed) as separate signals means I never have to wonder each morning whether the price moved or the monitor slipped. The former is a business finding; the latter is monitor maintenance. Mix them, and both get buried.

Quantify Coverage — A Health Dashboard for the Monitor

Once these signals exist, you can roll the monitor's own health into a single view. What I check every morning is not the competitor changes it detected, but first: "is the monitor actually seeing anything today?"

// health-dashboard.mjs — aggregate the monitor's own health
export function buildHealth(siteIds, loadDaily) {
  // loadDaily(siteId) -> { alive, coverage, quietStreak, drift, suspicious }
  const rows = siteIds.map((id) => {
    const d = loadDaily(id);
    const status = !d.alive ? '🔴 extraction-fail'
      : d.drift > 0.3 ? '🟡 drift'
      : d.suspicious ? '🟡 too-quiet'
      : '🟢 healthy';
    return { id, ...d, status };
  });
 
  const healthy = rows.filter((r) => r.status.startsWith('🟢')).length;
  const ratio = Number((healthy / rows.length * 100).toFixed(0));
 
  let md = `# Monitor Health — ${new Date().toISOString().slice(0, 10)}\n\n`;
  md += `Healthy ratio: ${ratio}% (${healthy}/${rows.length})\n\n`;
  md += `| Site | Status | Coverage | Quiet days | Drift |\n|---|---|---|---|---|\n`;
  for (const r of rows) {
    md += `| ${r.id} | ${r.status} | ${(r.coverage * 100).toFixed(0)}% | ${r.quietStreak} | ${r.drift} |\n`;
  }
  return { markdown: md, healthyRatio: ratio };
}

Having a single "healthy ratio" changed how I relate to the monitor. I used to care only whether the competitor moved today. Now I check first how many of my sites the monitor could actually see. On a day when the ratio drops, I do not trust the competitor report no matter how calm it looks — the calm might just be closed eyes.

Turning it into a number also catches gradual decay. One month the ratio slid from 100% to 90% to 80% over a graph, and I realized only afterward that several competitors had migrated to a similar framework around the same time. A change invisible in any single crawl showed up plainly in the trend.

Fold It into the Scheduled Task

Finally, here is how I run all of this inside the scheduled task. Order matters: health assessment goes before diff detection.

The scheduled task prompt writes that order out as plain steps. Replace the placeholder values with your own targets.

## Task: competitor monitoring (health-first)

Step 1: Crawl each URL in the web-monitor target list. For each,
  capture body text via get_page_text and a structure fingerprint
  via javascript_tool, and save as today's snapshot.

Step 2: Run anchor-check per site. Any site that fails its anchors
  is recorded as an extraction failure in the health log, and its
  diff detection is skipped (do NOT write "no changes").

Step 3: Compare fingerprints against yesterday and compute drift.
  Sites above 0.3 go on the notice list as "needs maintenance."

Step 4: For sites that passed anchors only, run the diff and append
  any changes to the report.

Step 5: Evaluate the zero-change streak; sites over threshold are
  recorded as "manual spot-check requested."

Step 6: Aggregate the healthy ratio across sites and update
  health-YYYY-MM-DD.md. If the ratio dropped from yesterday, add a
  warning banner to the top of the competitor report.

One implementation note: I keep the health log and the competitor report in separate files, always. Mixed into one file, "the monitor is broken" and "the competitor moved" blur together visually, and you end up missing both. Faults are for maintenance; movements are for the business. Split the outputs, and a few minutes each morning is enough to respond to both correctly.

Wrapping Up

What those two silent weeks taught me is that the dangerous thing in monitoring is not the loud error but the unreactive one dressed as normal. Errors you notice. But when a miss hides behind a calm line reading "no changes," the only thing that exposes it is instrumentation you built into the monitor itself.

First, separate fetch success from extraction success. Then measure extraction liveness with anchors, and get ahead of decay with quiet streaks and selector drift. Finally, confirm with a single healthy-ratio number whether the monitor has its eyes open today. None of it is flashy, but this quiet layer is the foundation for whether you can trust the competitor report at all.

If you have a monitor running quietly right now, add just one anchor check to tomorrow's crawl. Even counting how often it fails will start to reveal what you were not seeing. I hope these notes help in your own operations, and thank you for reading.