# AI

> What AI crawlers may fetch from the site and what they actually get, whether the pages read well for an assistant to quote, and which questions the site's passages answer. The bot check runs from this computer.

Describes GLLmolfar 0.9.22, checked 2026-11-01. Canonical page: https://www.getlooploop.com/gllmolfar/docs/ai

- **Where:** AI › AI
- **Mode:** Simple and Pro
- **Built from:** The open crawl and a bot check of about 10 pages
- **Changes:** When you press Check again or open another crawl

## On the screen

At the top, the AI readiness score with its four parts. Below, two columns: on the left whether AI can get in and which questions the site answers; on the right whether AI can read and quote it, trust, llms.txt and Markdown for agents.

![The AI tab of GLLmolfar after a bot check of getlooploop.com](https://www.getlooploop.com/images/gllmolfar/docs/ai.webp)

Numbered parts of the screenshot:

1. **AI readiness.** A score from 0 to 100 with its band.
2. **Headline and check.** One sentence about the site, when the bots were checked, and **Check again**. The note under it says how the check fetches.
3. **Four parts.** Access, Text bots get, Meaning and Trust, each with a bar. Meaning and Trust are marked **estimate**.
4. **Can AI get in.** Every AI bot with its purpose, whether robots.txt lets it into the home page, and what the sample pages answered. A bot opens its pages.
5. **Can AI read and quote it.** Lines with the number of pages each touches. Measured lines come first; style checks are marked **heuristic**. A line opens its pages.
6. **What the pages answered.** Pages fetched as that bot, by result. Bots that only exist as a robots.txt token say so.
7. **Who says it.** Authors, dates, unsourced numbers, the organization and its contacts across the site.

## AI readiness

The app's own score, not a standard. Access (40%) and text bots get (30%) are measured: robots.txt, the sample pages fetched as each bot, text only after JavaScript and nosnippet. Meaning (15%) and trust (15%) are the app's estimate. The score never exceeds the lower of the two measured parts. A part not measured is left out and reads **not checked yet**.

| Score | Headline on screen |
| --- | --- |
| 80–100 | AI crawlers can get in and read this site well. |
| 55–79 | AI crawlers get in, but parts of the site are hard for them to read or quote. |
| 0–54 | AI crawlers are blocked or see little of this site. |
| Not checked | Check the site as AI bots to see how they get in. |

> **Held down by a measured part.** When access or text bots get is below the rest, a line says so: “The score is held at N: it cannot be higher than Access, because bots that cannot get in or get the text cannot use the site, whatever its style.”

## Can AI get in

**Check as AI bots** fetches about 10 pages, each as a browser and with the User-Agent of each AI bot robots.txt allows — from this computer, without JavaScript. A CDN that verifies a bot's IP address may answer the real bot differently.

| Column | What it shows |
| --- | --- |
| Bot | GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Google-Extended, Applebot-Extended, CCBot, Bytespider, meta-externalagent and Amazonbot, each with its owner. |
| Purpose | **search** bots build the indexes AI search answers from; **user** bots fetch a page when someone asks; **training** bots only collect text for future models. |
| Home page in robots.txt | **allowed** or **blocked**. A blocked training bot is grey and reads **your choice, does not affect AI search**. |
| What the pages answered | Counts by result: like a browser (60%+ of its text), under 60% of the browser's text, CAPTCHA or bot check, error or refused, blocked by robots.txt. |

A bot opens its pages: the address, what the page gave as a browser and as that bot (status and words), and the result. Under the table, **robots.txt, bot by folder** shows ✓ or ✕ for every bot in each folder; hovering gives the matching rule and its line.

> **Training bots and Google.** Closing a training bot is a legitimate choice and does not remove the site from AI search. Google's AI Overviews and AI Mode use Googlebot, not Google-Extended.

## Can AI read and quote it

| Line | Kind |
| --- | --- |
| AI sees under half the text (needs JavaScript) | Measured |
| nosnippet or max-snippet: 0 | Measured |
| noai or noimageai | heuristic |
| Mostly menus and footers (under 40% content) | heuristic |
| Sections that cannot be quoted alone | heuristic |
| Walls of text (3,500+ characters without a heading) | heuristic |
| Headings skip a level | heuristic |
| Long text without lists, tables or headings | heuristic |
| Has question headings (good) | A good sign, not a problem |

Lines marked “heuristic” are style checks with no confirmed effect on AI answers. They do not move the score.

## The other blocks

| Block | What it shows |
| --- | --- |
| Which questions the site answers | Paste the questions people ask, one per line, and press **Match questions**. The local embedding model matches each with every section of the site: **yes** from similarity 0.56, **weak** from 0.42, with the best section and its page. Thresholds set on one site: a guide, not a verdict. |
| Questions people search | With Search Console connected: queries phrased as questions, with impressions, clicks and position, and a button to check them against the passages. |
| Who says it, when, and with what sources | The share of long pages (600+ words, not the home page) that name an author, the share that show a date, and the share of pages with numbers without a source. Then the organization, sameAs profiles, About and contact pages, phones and emails; **pages state different values** when they disagree. Four lines open their pages: no author, no date, not updated for over 18 months, numbers and claims without a source. |
| llms.txt | Whether `llms.txt` and `llms-full.txt` exist, their size, links and sections, links that lead to errors and links not in this crawl. When missing, a note on what the file is and that no major AI search engine has said it reads it. |
| Markdown for agents | For each sample page: whether it answers `Accept: text/markdown` with Markdown, and whether a `.md` version exists, with the words in each. A warning when Markdown is served without `Vary: Accept`. |

## States and messages

- **No crawl open** — the app says: "Enter a site address above and press Start." The tab needs a crawl.
- **Not checked** — the app says: "AI bots have not been checked for this crawl." Access, llms.txt and Markdown say **Press Check as AI bots above.**
- **Checking** — the app says: "Reading robots.txt…" A bar and the step: as a browser, as each bot, looking for llms.txt, asking for Markdown.
- **robots.txt down** — the app says: "robots.txt did not answer (status N). Under RFC 9309 a crawler that cannot read robots.txt treats the whole site as closed, so AI bots may skip it entirely." Access counts as zero.
- **Old crawl** — the app says: "Crawl again to read authors, dates and sources (crawls before 0.7.5 do not have them)." The trust block stays empty.

> **Terms with a dotted underline.** Terms such as robots.txt, nosnippet and llms.txt carry a dotted underline. Hovering one shows a sentence on what it is and **More in the GetLoopLoop glossary**. [Settings › General](https://www.getlooploop.com/gllmolfar/docs/settings-general) switches it off.

> **Locked screens.** A screen with a lock in the panel is not open on this account; pressing it says what opens it.

## Related pages

- [Overview](https://www.getlooploop.com/gllmolfar/docs/overview.md): The Overview tab after a crawl: the health score, six counters, what to fix first, response codes, indexability, site depth, topics and earlier crawls.
- [URL audit](https://www.getlooploop.com/gllmolfar/docs/url-audit.md): Site › URL audit: one page checked without a crawl — status, speed, JavaScript, page map, failed checks, accessibility, robots.txt, headers and screenshots.
- [Search Console: Performance](https://www.getlooploop.com/gllmolfar/docs/search-console.md): GSC › Performance: Google clicks, impressions, CTR and position read-only, kept on this computer and shown with their date when you are offline or signed out.
- [Agent · test](https://www.getlooploop.com/gllmolfar/docs/agent-test.md): The Agent · test screen: your own models plan tasks from the fix list, rewrite pages, write social posts and review risky pages; GLLmolfar measures the results.
- [Reverse prompts](https://www.getlooploop.com/gllmolfar/docs/reverse-prompts.md): Site › Reverse prompts: GLLmolfar reads a page into blocks, your AI agent describes what each does, and every block gets a reusable prompt and an example.
- [MCP tools for agents](https://www.getlooploop.com/gllmolfar/docs/mcp-tools.md): The MCP server inside GLLmolfar: how to connect an AI agent, and every tool it can call to crawl, read fixes, audit pages, search by meaning and save reports.