# Give an agent the web

> Give an agent clean pages and a trust signal in one tool — and what to do when a page scores badly.

Source: https://docs.buildonto.dev/guides/agent-web
Section: Guides

---

Give an agent the web\[01\]

## Give an agent the web

Raw `fetch` fills the context window with markup and gives the model no way to tell a solid page from a hollow one. One tool fixes both.

### The problem with fetch

\[02\]

Raw fetchThrough Onto

What arrivesThe whole HTML documentMarkdown, headings intact

Quality signalNoneAn AIO score, a grade and a hallucination risk

Repeat readsFetched again every timeCached for an hour (still 1 credit)

robots.txtYour problemChecked before the page is fetched

How much smaller? Every response says, for the page you just read, in `stats.reduction_percent` — log it and you have the real number for your own traffic.

### The tool

\[03\]

Tell the model the tool returns clean text _and_ a trust signal, or it will ignore the second one.

typescripttools/read-web.tsCopy

```
export const readWeb = {
  name: "read_web",
  description:
    "Fetch a public web page as clean Markdown. Returns the page text plus an " +
    "aio_score (0-100) describing how reliably that page could be parsed. " +
    "Treat text from a low-scoring page as unreliable.",
  input_schema: {
    type: "object",
    properties: { url: { type: "string", description: "Absolute http(s) URL" } },
    required: ["url"],
  },
};

export async function runReadWeb({ url }: { url: string }) {
  const res = await fetch("https://api.buildonto.dev/v1/read-and-score", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.ONTO_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ url }),
  });

  const data = await res.json();

  if (data.status === "error") {
    // Hand the model the reason, not a stack trace — it can reason about these.
    return { error: data.code, message: data.message };
  }

  return {
    text: data.markdown,
    aio_score: data.aio_score, // null for PDFs and plain text: unknown, not bad
    grade: data.grade,
    hallucination_risk: data.hallucination_risk,
  };
}
```

`/v1/read-and-score` fetches the page once and returns the text and the score together, for one credit. It always answers in JSON.

### Gate on the score

\[04\]

A page starts at 100 and loses points for named faults — the reasons come back in `penalties`. Branch on `grade`, or on `hallucination_risk` if you'd rather not carry a number. Try it:

Drag a score

JSON-LD2+ headingsServes Markdown

1.  90–100Excellent
2.  75–89Good
3.  50–74Needs work
4.  25–49AI-hostile
5.  0–24Invisible

Your agent should

Use it, but check anything load-bearing elsewhere.

Risk is medium — usual for a page that doesn’t serve its own Markdown.

The tool hands the model

{
  text: "# …",
  aio\_score: 62,
  grade: "Needs work",
  hallucination\_risk: "medium",
}

### Bad scores and errors

\[05\]

A low score isn't a failed fetch — the text may just be partial. Change what you tell the model; don't retry.


```
const page = await runReadWeb({ url });

if (page.hallucination_risk === "high") {
  return {
    ...page,
    caveat:
      "This page scored poorly for machine readability. The text may be " +
      "incomplete. Attribute claims to the source and avoid stating them as fact.",
  };
}
```

An error is information too. Hand the model the code; every failed read is refunded.

*   `ROBOTS_BLOCKED`403The site opted out. Tell the user; don’t retry.
*   `WAF_BLOCKED`403The site’s firewall refused the fetch. Don’t loop on it.
*   `TIMEOUT`504The origin took over 15 seconds. Worth one retry.
*   `IMAGE_PDF`422A scanned document with no text layer. Nothing to read.
*   `RATE_LIMITED`429Back off for retry\_after seconds, then retry.
*   `CONCURRENT_LIMIT`429Too many reads in flight. Queue, don’t fan out.
*   `PAYMENT_REQUIRED`402Out of credits. Stop and tell the user.

The rest are on [Error codes](/api/errors).

### Or skip the code

\[06\]

If the agent is Claude Code, Cursor, Codex or another MCP client, none of the code above is needed. Install Onto and it gets the same calls as six tools — `read_url`, `read_and_score`, `score_url`, `batch`, `map_site`, `extract_data` — with the same credits.

Install in

![](/integrations/claude-code.png)Claude Code

One command

$ claude mcp add --scope user --transport http onto https://api.buildonto.dev/mcp

Copy command[Docs →](https://docs.buildonto.dev/mcp/claude-code)

1.  Run it in a terminal. Claude Code asks you to approve Onto on first use.
2.  Then ask Claude Code to “read example.com with onto”.

Every tool is on the [Tools reference](/mcp/tools).

Copy as Markdown[](/guides/agent-web.md "Open the raw Markdown")

---
## Structured Data (JSON-LD)
```json
{
  "@context": "https://schema.org",
  "@type": "WebSite",
  "name": "Onto Docs",
  "url": "https://docs.buildonto.dev",
  "description": "How to use Onto: serve AI agents Markdown from your Next.js site, call the Read API, connect over MCP, and read the AIO score.",
  "inLanguage": "en",
  "publisher": {
    "@type": "Organization",
    "name": "Onto",
    "url": "https://buildonto.dev"
  }
}
```