llms.txt Generator

llms.txt is a plain-text file in the site's root directory that tells language models what matters most on the site. The format is described at llmstxt.org: the site name as a heading, a short description as a quote, and sections with lists of links.

How the file is built

Enter the site address. We look up the sitemap through the Sitemap entries in robots.txt or at /sitemap.xml, read nested and compressed sitemaps, and pick up to a thousand pages. For the first three hundred we read the page titles; for the rest we take the name from the URL, so the site does not get hundreds of requests in a row. Those pages go into an "Other pages" section marked as Optional. Pages are grouped by site section: everything under /blog/ lands in one section. The section name comes from its first page.

The file skips list and tag pages, search, cart, and login pages, files and images, and addresses blocked in robots.txt or by a noindex tag. The result shows how many addresses were left out and why.

Editing before download

Uncheck any page or section you do not want. If a section is optional, mark it "in Optional": by the format description, such links get skipped when a short overview of the site is needed. The site name and description are edited in the fields above the list, and the file rebuilds right away.

If there is no sitemap, or you only need selected pages, choose "From my own list" and paste the addresses one per line.

Checking an existing file

The "Check an existing file" mode reads /llms.txt on the site and compares it against the format description: a heading with the name comes first and appears only once, sections use second-level headings, and each section holds a list of links. It also checks whether the first forty links from the file open.