Content negotiation

Serving the right format to the right audience — humans get HTML, agents get Markdown — from the same URL.

What it is

A 1996 HTTP idea, refreshed for the agent web. Same URL, different response shape based on what the client asks for. The server inspects the request and decides what to serve; callers don't need a second URL or a query parameter.

Two signals we negotiate on

text

1. User-Agent — matches known AI crawler patterns (GPTBot, ClaudeBot, etc.)
2. Accept: text/markdown — explicit content negotiation header set by AI tools

Either signal triggers the bot branch in the SDK middleware. Humans (browsers sending Accept: text/html) get the full page unchanged.

Example

bash

# Human request — gets the full HTML
curl https://buildonto.dev/why-onto
# Content-Type: text/html
# 600 KB

# AI request — gets clean Markdown from the same URL
curl -A 'GPTBot/1.0' https://buildonto.dev/why-onto
# Content-Type: text/markdown; charset=utf-8
# X-Onto-Matched: true
# 18 KB

Why it matters

Without negotiation, you have two bad options: serve everyone the HTML payload (AI agents pay 10–100× more tokens than needed) or maintain a parallel /ai/... URL tree (every page needs a duplicate; agents have to guess where to look; you fragment your analytics).

ℹ

Onto is content negotiation done at the edge. One URL, one set of links on the internet, one source of analytics. The format flips per-request based on who's asking. No URL fragmentation, no separate AI subdomain, no parallel content tree.