Hey everyone!
There are now well over a hundred AI crawlers on the web, and they don't all do the same thing. Some collect content for model training. Some power AI search. Some only open a page when a reader asks an assistant to fetch it. Before you can decide anything, you have to know which user agent belongs to which service – and that list keeps growing.
That's why we created the AI Crawler Blocker plugin. It comes pre-loaded with 160 known AI crawlers, already categorized by what they actually do – so you can make quick decisions without doing hours of research. Pick a protection profile, and Publii applies it automatically whenever you generate your site. Your existing custom rules stay in place, and regular search engines keep working exactly as before.
No user agents to look up, and no categories to figure out.
What you get: - 4 Protection Profiles – Strict, Training only, Individual crawlers, or Custom by category
- 160 Curated AI Crawlers – Block or allow them individually or in groups
-
Custom Exceptions & Rules – Add your own crawler identifiers with private notes
- Automatic robots.txt Rules – Written on every site generation, alongside anything already in the file
- Meta Tag Signals – Optional noai and noimageai directives that leave your index and follow settings untouched
-
Emergency Lockdown – For the rare case when you want to immediately block every compliant crawler, search engines included
Available now in the Marketplace. Requires Publii 0.47 or newer, works on an unlimited number of domains, and includes three months of premium support. |
