Skip to content

E2E flake: broken-links 'check meta tag images and resources' fails on external OG-image fetch (statusCode undefined → dead-IPv6/external-host class) #323

Description

@TortoiseWolfe

Symptom

[chromium-gen] › tests/e2e/tests/broken-links.spec.ts:279:7 › Broken Links Detection › check meta tag images and resources flaked on PR #322's E2E run (passed only on retry). The flaky honesty gate (#300, annotate mode) caught it:

1) [chromium-gen] › broken-links.spec.ts:279:7 › check meta tag images and resources
   Error: expect(received).toEqual(expected)
   + Object {
   +   "error": "Open Graph image not found",
   +   "sourceUrl": "http://localhost:3000/",
   +   "statusCode": undefined,
   +   "targetUrl": "https://tortoisewolfe.github.io/ScriptHammer/opengraph-image.png",
   + }
   at broken-links.spec.ts:366 (expect(brokenResources).toEqual([]))

Root cause (class, not one-off)

statusCode: undefined means the fetch failed to connect — it is NOT a 404. The targetUrl is the external, absolute production URL on GitHub Pages (https://tortoisewolfe.github.io/ScriptHammer/opengraph-image.png), not a local resource. This is the dead-IPv6 / external-host-unreachable class (HTTP 000 + dns=0.000000s) documented in project memory: *.github.io leads with an AAAA record, this machine's/runner's IPv6 is intermittently dead, and the same URL "breaks" with nothing changed. It passed clean on retry.

So the test conflates two very different outcomes:

  • a genuinely broken resource (real 404 / wrong URL) — a real bug, and
  • a transient connection failure to an external host (statusCode: undefined) — environmental noise.

The OG <meta property="og:image"> points at the absolute production URL by design (crawlers need an absolute URL), so the test reaches out to the live internet on every run.

Candidate fixes (pick one at implementation)

  1. Treat statusCode: undefined (connection failure) as NOT-broken for external/absolute hosts — only fail on a real error status (404/410/5xx). A connection failure to an external host is not a broken-link signal the app is responsible for.
  2. Scope the crawl to same-origin resources (localhost) and skip absolute external URLs (OG image, any CDN), which the app cannot control and which are covered by deploy-time checks instead.
  3. Retry the individual external fetch with a short backoff inside the test before declaring it broken (narrower than Playwright's whole-test retries).

Recommend (1)+(2): don't fail an app E2E on an external host's transient reachability.

Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions