How it works
One cleaning engine, three trips through it. Pick one and watch it go — click a stage to stop on it.
Three trips
[02]Looked up by its SHA-256 hash — Onto keeps the hash, never the key. Calls from the MCP connector carry a short-lived token Onto minted after your OAuth sign-in instead.
From your account's month, which resets at 00:00 UTC on the 1st, plus one of your concurrent slots. It's taken before the cache, so a cached read still counts; a failed read is refunded. Numbers per plan on Rate limits.
One hour, keyed on the deploy, the endpoint and the exact URL — so every deploy starts empty.
"fresh": trueskips the lookup.hitanswer from the cache and stop heremisscarry on
The host must resolve to a public address — checked again on every redirect. GitHub file links are rewritten to their raw version.
Fetched with a 5-second limit. If it can't be fetched, the read goes ahead; reading your own verified site skips the check.
Disallow: /for GPTBot or * → ROBOTS_BLOCKED, before the page is fetched
15 seconds for the whole fetch, up to 5 redirects, 10 MB at most, no JavaScript run. HTML pages also get two quick probes for a Markdown version the site already serves.
HTML goes through
@ontosdk/core, after tables are normalised. PDFs are read as text; JSON, CSV and plain text pass through.Every HTML read gets an AIO score. /v1/score and /v1/read-and-score return it;
/v1/readdoesn't.The result is cached, usage is logged without waiting on it, and the slot is freed.
Every prerendered page under
.next/server/app. Routes rendered per request get no.md.The same
@ontosdk/corecleaner the Read API uses — once per build, not once per request.One
.mdper route inpublic/.onto/, andllms.txtfrom youronto.config.ts.Posted to
api.buildonto.dev/api/fileswith your site key — only whenONTO_API_KEYis set. That's what fills Serve → Routes.
A known AI crawler's User-Agent, an
Accept: text/markdownheader, or?ontoon the URL.anyone elsestraight through to your page — the middleware does nothingagentcarry on
Sent to Onto without waiting on the reply, so it never slows the answer. Skipped when no site key is set.
The prebuilt
/.onto/pricing.md, fetched from your own site — nothing is extracted now.Only if you've written some for the route, and only on Serve Pro and above. This step does wait for Onto's reply.
One engine
[03]Both products clean pages with @ontosdk/core. The Read API runs it per request on the URL you send; the SDK runs it once per build on your own pages. The same service at api.buildonto.dev also answers batch, map and extract, and hosts the MCP server at /mcp.
Where data lives
[04]- SupabasePostgres + Auth
- Your accountSign-in with email and password, GitHub or Google (Supabase Auth)
- Read keysapi_keys — the SHA-256 hash and first 12 characters
- Sites and their keyssites — what the SDK authenticates with
- Route listonto_files — a copy for Serve → Routes; agents are served from your own site
- Agent visitsagent_events — Serve analytics
- Read usageusage_events — the Usage page
- Plans and creditssubscriptions, one per account and product; profiles.credit_balance with a ledger
- Vercel KVRedis
- Monthly counter and concurrent slotsChecked on every call
- Markdown cacheOne hour
- MCP sign-insRegistered clients, one-time codes and refresh tokens for Onto's own OAuth
- PolarPayments
- Checkout and billingWebhooks keep the plans above in step
- Your siteYour hosting
- public/.onto/*.mdWhat agents are actually served