# ADR-0034 — The proxy answers unknown paths with a true 404 (/docs/maintain/adr/ADR-0034-the-proxy-answers-unknown-paths-with-a-true-404)

- **Status:** Approved
- **Date:** 2026-11-03
- **Deciders:** [@puncoz](https://github.com/puncoz)
- **Approved by:** [@puncoz](https://github.com/puncoz)
- **Approved on:** 2026-11-03
- **Supersedes:** [ADR-0015](./ADR-0015-every-entry-lives-at-the-root-in-one-shape-and-one-look.mdx)
  (in part: its implementation's "the proxy acts only on catalogued paths")
- **Superseded by:** —
- **Design documents:** [`../app-flow/unknown-paths.mdx`](../app-flow/unknown-paths.mdx),
  [`../api/content-routes.mdx`](../api/content-routes.mdx),
  [`../architecture/system.mdx`](../architecture/system.mdx)

## Context [#context]

- Every path that is not a site route reaches `src/app/(site)/(docs)/[...slug]/page.tsx`, which
  calls `notFound()` when it has no such page.
- That call comes too late for a path `generateStaticParams` did not list. Measured on
  `bunx next build`, for a path not requested before:

  | Request | Status | HTML |
  |---|---|---|
  | First `/no-such-page` | 200 | Blank: `<html id="__next_error__">`, no page content |
  | The same path again | 404 | Blank, the same shell |
  | First `/x/y/z`, `/docs/nothing-here` | 200 | Blank, the same shell |

  The route's prerendered fallback shell is empty, and the page learns the path is unknown only
  when it resumes. Scripts then build the 404 in the browser, with a `noindex` tag.
- So a reader without scripts sees a blank page, which contradicts the 404's documented state
  (*Back to the system* still works). Monitoring counts these as successes, some crawlers report
  soft 404s, and in Chrome the first hit's tab showed no title although the title is in the HTML.
- The installed Next.js documents this. A streamed page keeps its 200, and for a 404 "you can
  check the slug in Proxy, then rewrite it to a not-found route or return a 404 response"
  (`file-conventions/loading.md`, *Status codes*).
- `dynamicParams = false`, which would refuse unlisted paths, is not available with Cache
  Components (ADR-0016; `route-segment-config/dynamicParams.md`).
- The proxy has no `matcher`, so it runs on every request, including `_next` and files. It already
  answers line redirects and Markdown, but only for catalogued slugs, and it knows only `meta.json`
  files: `src/lib/catalog-routes.ts` keeps compiled pages out of its bundle.
- `fumadocs-mdx/macro` has an `async` mode (`docs: { async: true }`): frontmatter is imported at
  once and the compiled body only when asked for (`transform-*.js`, `only: "frontmatter"`).

## Decision [#decision]

1. **The proxy answers every unknown path with a true 404, before any page renders.** The status
   is 404 from the first request and the response is the whole out-of-orbit page, the existing
   `src/app/not-found.tsx`, in the HTML. The address does not change, and the page still shows no
   part of it.
2. **What it judges.** Only paths `[...slug]` would otherwise render. These pass through untouched:
   `/`, `/_next/…`, the site's own routes (`api`, `diagrams`, `llms-full.txt`, `llms.mdx`,
   `llms.txt`, `og`, `static.json`), `icon.svg` and any file in `public/`.
3. **What it knows.** Every page's URL, and each `/<slug>/use` and line root that redirects to a
   first page (ADR-0015 decision 1). Comparison is exact, so `/DOCS` is unknown.
4. **The existing order stays.** Line redirects and Markdown come first, so a retired line still
   redirects and `/<unknown>.md` still gets its own 404 from the Markdown route.
5. **The list comes from the content, and the build checks it.** `catalog-routes.ts` lists the
   pages' paths through an `async` collection, so no compiled page runs in the proxy, and works out
   each URL with the same line segments as `source.ts`. **The build fails if this list and
   `source.getPages()` differ**, because a page wrongly refused would be a real page gone.
6. **`notFound()` in the page stays**, as the safety net for anything that reaches it.

## Options considered [#options-considered]

| Option | Pros | Cons | Verdict |
|---|---|---|---|
| Check every path in the proxy | True 404 with full HTML everywhere; what Next.js documents | The proxy must know the pages; a wrong list would hide a real page, so it needs a build check | Chosen (owner) |
| Check only the first segment | Uses data the proxy already has | `/docs/nothing-here` stays a blank 200 | Rejected (owner) |
| Leave it | No work; `noindex` already keeps these out of search | Blank without scripts, 200 in monitoring | Rejected (owner) |
| `dynamicParams = false` | One line, no list | Not available with Cache Components | Not possible |
| Generate the list in a script, or in `next.config.mjs` | No macro involved | A new build step, and it can go stale in development | Rejected |
| Read `content/` from the proxy at request time | Always current | Slow, and the content need not be on the server | Rejected |
| Fetch from a route in the proxy | Reuses `source` | A request per request; Next.js advises against it | Rejected |

## Consequences [#consequences]

**Positive**

- Unknown paths are true 404s for readers, tools and monitoring, and readable without scripts.
- The 404's documented degraded state becomes true.

**Negative / accepted trade-offs**

- The proxy gains a second view of the pages. The build check is what makes that safe.
- It runs one set lookup on each request it judges.
- The `async` collection adds lazily loaded chunks to the proxy's bundle. They are not run, but
  their size is unmeasured until this is built.
- A new top-level route must be added to the proxy's pass-through list, or it would answer 404. A
  test compares that list with `src/app/`.

## Content invariants [#content-invariants]

§6.5 is kept: Markdown routes are not changed. No content is added or exposed. The URL list holds
only pages that are already published.

## Security considerations [#security-considerations]

Low. The request's path is only compared with a fixed set, and it is never echoed, so the page
still shows no part of it. Nothing leaves the system and no secret is touched. The proxy's bundle
holds the same public URLs as `llms.txt`. An unknown path now gets a cached static page, which
costs less than a render. The threat model is checked on implementation.

## Implementation notes [#implementation-notes]

- [x] First, verified with a spike on `bunx next build`: a proxy rewrite to `/_not-found` with
  status 404 serves the whole page on the first request, for document and `_rsc` requests alike.
  The ADR's other route was not needed.
- [x] Measured the proxy's traced chunks: 512 KB before; 2.9 MB with a plain `async` collection,
  because it compiles a lazy chunk with the text of every page, which the proxy never loads. That
  was out of proportion, so the collection in `catalog-routes.ts` compiles each body empty, with a
  one-line remark plugin of its own. The result is 585 KB, with no page text in any proxy chunk,
  and the site's own collection keeps its bodies. The chunk loaded at start grows by about 20 KB,
  the frontmatter. A request took 1.5ms at the median through the proxy.
- [x] `src/lib/known-paths.ts`: the site routes, `knownPaths` and `knownPathProblems`, without the
  macro so they can be tested. `src/lib/catalog-routes.ts`: the collection, `isKnownPath` and
  `assertKnownPaths`. `src/proxy.ts`: the check, after the existing steps.
- [x] `lineSlugs` in `src/lib/catalog-plugin.ts`, used by `source.ts` and `known-paths.ts`, so both
  work out a line's URL the same way.
- [x] The build check of decision 5: `checkKnownPaths` in `src/lib/catalog.ts`, called from the
  page's `generateStaticParams`, comparing the list with `source.getPages()` and the catalogue's
  landings. Dropping one page from the list fails `bunx next build` and names that page.
- [x] Tests (ADR-0007), in `src/lib/known-paths.test.ts`: known pages, landings, line segments,
  unknown paths at each depth, the wrong case and trailing slashes; each pass-through; the
  pass-through list against `src/app/` and `public/`; the check failing on each kind of difference.
  Checked with `curl` against `bunx next build`:
  - the Context table's paths give 404 with the page in the HTML on the first request, and no part
    of the path in it;
  - the 93 pages in `llms.txt` and in `static.json` give 200;
  - the site's routes and files are unchanged, `/<entry>/<unknown>.md` still gets the Markdown
    route's 404, and `/docs/` still redirects to `/docs`.

  In Chrome: the tab's title on a first hit; the page with scripts off, in a frame sandboxed without
  `allow-scripts`, whole and with its links; a client-side push to an unknown path; seven real
  prefetches, all 200; *Back to the system* and an ordinary link staying client-side.

  In development, against `next dev`: a page added to `content/` answered 200 within four seconds,
  and 404 again within four seconds of being removed, so the proxy's list does not go stale there.
- [x] Found while checking:
  - A client-side link to an unknown path now loads the whole page rather than swapping it in,
    because Next.js does that for any RSC response that is not OK
    (`fetch-server-response.js`). The reader lands on the same page.
  - The 404 is served with `Cache-Control: s-maxage=31536000`, as a static page is. A shared cache
    that keeps it across a deploy would hide a page added later. No host is adopted yet; the ADR
    that adopts one must check this.
  - Decision 2 leaves everything below a site route alone, so a path such as `/api/nope` or
    `/diagrams/nope`, which only `[...slug]` matches, is still a blank 200. `/og/nope` is a 404 from
    its own route. ADR-0035 later removed `api` from the site routes, since no such route exists, so
    `/api/nope` is now a true 404; `/diagrams/nope` and `/llms.txt/x` are still blank 200s.
- [x] `bun run test`, `bun run types:check` and `bunx next build` pass; `bun run lint` is reported
  while it stays blocked.

## Docs to update on implementation [#docs-to-update-on-implementation]

- [x] [`../app-flow/unknown-paths.mdx`](../app-flow/unknown-paths.mdx): Current.
- [x] [`../api/content-routes.mdx`](../api/content-routes.mdx): the 404 response.
- [x] [`../architecture/system.mdx`](../architecture/system.mdx): the routes table and what the
  proxy acts on, with the diagram's Archify pair; `content/docs/maintain/security/threat-model.mdx`.
- [x] `AGENTS.md` §1, §5 and §10. `PROJECT_AGENTS.md`: no change; a consuming repository's pages
  are listed from the content.
- [x] ADR-0015's "Superseded by" line, and [`./index.mdx`](./index.mdx): index row and status.