How to Create llms.txt File in WordPress with DefiniteSEO

The way people search the web is changing. Alongside traditional search engines, AI models like ChatGPT, Claude, and Perplexity are increasingly shaping how information is discovered. These models rely on crawlers to access content from websites. To guide how your website is treated by these AI crawlers, a new standard called llms.txt has emerged.

Think of it as the AI equivalent of robots.txt in a nascent stage. It provides instructions about which parts of your site can be accessed by large language models (LLMs).

If you are using WordPress, managing this file manually can be technical and time-consuming. That’s why the DefiniteSEO WordPress SEO Plugin includes full support for creating and managing an llms.txt file directly from your WordPress dashboard, no coding, no file uploads.

In this guide, you’ll learn what llms.txt is, why it matters, and exactly how to configure it using DefiniteSEO.

What is llms.txt?

This file is a recently introduced standard that helps website owners guide large language models (LLMs) such as ChatGPT, Claude, or Perplexity. Similar to the familiar robots.txt, it lives in the root of your website (for example, https://yourwebsite.com/llms.txt) and is written in plain text. What makes it different is its purpose: instead of simply blocking or allowing crawlers, it provides structured and curated information that tells AI systems which parts of your website are most important.

How It Differs from robots.txt and Sitemaps

While robots.txt instructs search engines on what they can or cannot crawl, and XML sitemaps provide a full index of your site’s URLs, the llms.txt file is designed specifically for AI crawlers. It focuses less on restrictions and more on presenting a clear picture of your content. By listing posts, pages, categories, tags, and additional resources in a well-organized way, it helps AI models quickly understand the structure and value of your site. In essence, robots.txt controls access, sitemaps show everything, and this new standard highlights what matters most for AI-driven experiences.

Why It Exists

Generative AI is transforming how people search for and consume information online. Users are no longer just typing queries into Google, they’re asking AI assistants to summarize, recommend, and compare. Without this file, AI crawlers may gather incomplete or irrelevant information from your site. By providing one, you can ensure that your best content, such as blog posts, product categories, or cornerstone pages, is included, while sensitive or less relevant areas like admin sections or checkout pages can be left out.

What It Looks Like

Its human-readable and it may contain sections for Posts, Pages, Categories, Tags, and Additional Content, each with direct links. For example, under “Posts” you might list your top articles, under “Pages” you could include your About and Contact pages, and under “Categories” you could feature key topics such as “Digital Marketing” or “SEO Tips.” This format gives AI crawlers a structured roadmap of your website, making it easier for them to represent your content accurately.

Why It Matters for Your Website

In short it may acts as a bridge between your website and AI systems. It does not replace files like robots.txt or sitemaps but works alongside them to serve a new purpose, guiding large language models. By maintaining one, you future-proof your website for AI-driven discovery, protect sensitive information, and ensure your brand is represented correctly in the growing world of generative search.

Why is it Important for Your WordPress Website?

For WordPress websites, this is particularly valuable because WP sites often have a lot of automatically generated content, such as tag archives, attachment pages, and custom post types. These pages are not always useful to highlight. And with this newly proposed standard you can focus crawlers on high-value areas like your main posts, cornerstone pages, and category hubs while filtering out unnecessary or sensitive sections.

In short, it gives you greater control over how AI models interpret your content. It helps present your site in the best possible way, increases the chance of being cited by AI tools, and ensures that your brand message is delivered accurately when users search through modern, AI-driven platforms.

Without this file, AI crawlers may collect your data in ways you didn’t intend, or worse, skip valuable pages you want them to see.

Step-by-Step Guide to Creating an llms.txt File with DefiniteSEO

Creating an llms.txt file with DefiniteSEO is a simple process that only takes a few minutes. The plugin provides clear options to enable the feature, decide what content to include, and instantly generate a ready-to-use file for your website. By following the steps below, you’ll be able to customize the file according to your needs and make it live without any manual coding or server work.

Step 1: Enable it

The first step is to turn on this feature inside the DefiniteSEO plugin. Since it is disabled by default, you’ll need to enable it from the settings before making any customizations. Once activated, the plugin unlocks all the options you need to configure your file.

1. Log in to your WordPress dashboard.
2. Go to DefiniteSEO → Preferences → llms.txt.
3. Click Enable.
4. Save your changes.

Step 2: Select what Content to include

DefiniteSEO gives you three different modes to decide what content appears in this AI instruction file. You can choose the Automatic mode, which includes all content types like posts, pages, categories, tags, and custom post types with adjustable limits; the Manual mode, which lets you enable or disable specific content types while still controlling how many items are shown; or the Custom mode, which allows you to hand-pick the most important pages and optionally include or exclude other content types. These flexible options ensure you have full control over how your website is presented to AI crawlers.

Option 1: Automatic Content Selection (Default)

When you first enable this feature in DefiniteSEO, the plugin automatically puts your website into Automatic Content Selection mode. This is the simplest option because it requires no extra setup, the plugin gathers and add all the common content types on your site, such as posts, pages, categories, tags, and even custom post types.

To prevent the file from becoming too large, DefiniteSEO applies default limits. For example, it will list the most recent 50 posts, 50 pages, 50 categories, and 50 tags. If you have a smaller site, this is usually enough to cover everything. For larger websites, you can adjust these numbers to display more or fewer items depending on your needs.

In addition, this mode gives you an Additional Content field, where you can manually add any extra URLs that are not already picked up by the plugin. This is especially useful for special landing pages, promotional content, or external resources you want AI crawlers to notice. Automatic mode is best for users who want a quick setup without having to make a lot of manual decisions.

• Includes all posts, pages, categories, tags, and custom post types.
• Each content type is limited to 50 items by default.
• You can adjust the limit (e.g., set posts = 100, categories = 30).
• Add extra links using the Additional Content field.
• Best choice for users who want a complete output without manual setup.

Option 2: Manual Content Selection

If you want more control over what gets included, the Manual Content Selection mode is the right choice. Unlike the automatic mode, where everything is included by default, this option lets you choose exactly which types of content should appear. For example, you may decide to include posts and pages but exclude categories, tags, or certain custom post types.

Along with enabling or disabling content types, you also have the ability to set content limits. This means you can specify how many posts, pages, categories, or tags should be listed in the file. For instance, you might want only your 20 most recent posts, your 10 most important pages, and no categories or tags at all.

Just like in automatic mode, you also get access to the Additional Content field. Here you can add specific URLs that don’t fall into the standard content types but are still valuable for AI crawlers to know about, such as campaign landing pages or resource links.

Manual mode is ideal for site owners who want a balance between automation and control. It saves you from having to hand-pick every page individually but still gives you the flexibility to exclude unnecessary sections and keep the file lean and focused.

• Lets you enable or disable specific content types such as posts, pages, categories, tags, or CPTs.
• Still allows you to set item limits per type.
• Supports Additional Content just like the automatic option.
• Good choice if you want to exclude certain content types.

Option 3: Custom Content Selection

The Custom Content mode is designed for maximum precision. Instead of relying on automatic detection or broad content type controls, this option allows you to hand-pick the exact pages you want to showcase in your output file. This is especially useful for highlighting cornerstone content like your About page, Contact page, Privacy Policy, Terms and Conditions, or any other high-value pages that represent your brand.

For other content types, such as posts, categories, tags, or custom post types, you won’t be able to choose individual items one by one. Instead, you can simply decide whether to enable or disable them. Once enabled, you can still apply content limits to control how many entries from that type should be included. For example, you may allow posts but restrict the list to your 10 most recent ones, while completely disabling categories and tags.

This mode also supports the Additional Content field, giving you the flexibility to add any extra URLs that aren’t part of the standard WordPress content structure. This can include custom landing pages, promotional URLs, or even external resources you want AI crawlers to pick up.

• Allows you to manually pick important pages (About, Contact, Privacy Policy, T&Cs, Shop, Blog, etc.).
• For other types like posts, categories, and tags, you can only enable or disable them.
• Still offers item limits and additional content.
• Perfect if you only want to highlight key pages for LLM crawlers.

Step 3: Save Your Settings

Once you’ve chosen your preferred mode and adjusted the options, click the Save Changes button to apply your settings. DefiniteSEO will then generate the file instantly in the background, so you don’t need to upload or edit anything manually on your server.

Step 4: Verify the Output

After saving your settings, the final step is to verify if this file has been generated correctly. To do this, open your browser and go to: https://yourwebsite.com/llms.txt

Here you should see a neatly structured text file with sections for posts, pages, categories, tags, and any additional content you added. Each entry will appear as a URL, with the title of your content type as the prefix and the excerpt as the suffix. For example, under “Posts” you might see your latest blog articles, under “Pages” your About or Contact page, and under “Categories” the main topics from your site.

It’s a good idea to review the file carefully to make sure it matches your intentions. If you notice too many entries or missing items, you can always go back into the DefiniteSEO settings, adjust the limits or selections, and save again. The plugin will automatically update the file in real time, so any changes are reflected immediately when you refresh your browser.

By verifying the output, you ensure that AI crawlers are seeing exactly what you want them to see, giving you confidence that your site is properly represented in AI-driven search and discovery tools.

The output will look like this:

## Post
- [Example Post](https://example.com/sample-post)

## Page
- [Contact Page](https://example.com/contact)

## Category
- [SEO](https://example.com/category/seo)

## Post_tag
- [Best Plugins](https://example.com/tag/best-plugins)

## Additional Content
- https://example.com/custom-page

FAQs

Q1. Is llms.txt required for every website?
No, it’s not mandatory. Your website will function normally without it, but as per this proposal adding it gives you more control over how large language models (LLMs) like ChatGPT or Perplexity understand and represent your content. Think of it as future-proofing your site for the growing role of AI search.

Q2. Does llms.txt replace robots.txt or XML sitemaps?
Not at all. Each file serves a different purpose. robots.txt controls crawler access, XML sitemaps provide a structured list of all URLs for search engines, and llms.txt is specifically designed to help AI crawlers understand and prioritize your content. It’s best to use all three together.

Q3. Will adding an llms.txt file improve my Google rankings?
Currently, Google does not use llms.txt for ranking or indexing. Its primary use is with AI crawlers, not traditional search engines. However, indirectly, it can increase your brand visibility in AI-driven tools and assistants, which may lead to more traffic.

Q4. What happens if my website grows beyond the default 50 items per content type?
If your site has more than 50 posts, pages, or taxonomies, only the set number will be displayed unless you increase the limit in DefiniteSEO settings. This prevents the file from becoming too large and helps keep it manageable. You can always adjust these limits to suit your site’s size.

Q5. Can I exclude sensitive pages from llms.txt?
Yes. DefiniteSEO allows you to hide specific types of content (such as tags, categories, or custom post types) and in Selected mode, you can hand-pick only the pages you want. This ensures private areas like checkout pages, thank-you pages, or admin-only content do not appear.

Q6. Will AI crawlers always respect my llms.txt file?
The llms.txt standard is still very new, and as of now, no major LLM provider has officially announced support for it. This means some AI crawlers may not recognize or follow the file right away. However, the format is gaining attention in the industry, and adoption could increase over time, similar to how robots.txt became widely respected by search engines. Adding an llms.txt file now helps you prepare in advance and makes your website ready for when broader support arrives.

Q7. Do I need technical knowledge to manage llms.txt?
Not at all. With the DefiniteSEO plugin, you don’t need to edit text files or access your server. Everything can be done through your WordPress dashboard with simple toggles, dropdowns, and fields.

Q8. Can I include external URLs in my llms.txt file?
Yes. Using the “Additional Content” field in DefiniteSEO, you can list external resources or custom links you want AI crawlers to notice. This can be helpful for pointing to external documentation, affiliate pages, or microsites related to your brand.

Q9. How often should I update my llms.txt file?
That depends on how frequently your site changes. For blogs or news sites that publish often, updating the file weekly or monthly makes sense. The good news is that DefiniteSEO updates the file dynamically as your site grows, so most of the time no manual action is needed.

Q10. Can I view analytics on how AI crawlers use my llms.txt file?
At the moment, there is no standardized analytics system for llms.txt. Some AI companies may provide crawl logs in the future, but for now, you can monitor your server logs.

This file is becoming an essential tool for webmasters who want control over how AI models interact with their sites. With DefiniteSEO WordPress Plugin, creating and managing this file takes just a few clicks no technical skills required.