Skip to content
SEO Rapor
Log inStart freeTREN
Sites and crawling › Crawler bot

Crawler bot

The bot's identity and how to block it with robots.txt.

If you see our crawler bot in your site's access logs, this page explains what it does and how to block it if you want. Public information page: /bot.

#What does the bot do and not do?

  • It reads public pages only (GET). It does not sign in, submit forms or change anything.
  • It sends 2–5 requests per second; on sites whose ownership is not verified, the speed is at the lowest value.
  • It follows the robots.txt rules.
  • It runs only one crawl at a time for the same domain and limits the number of crawls per day.
  • It does not publish the content of the pages it crawls; the results appear only in the report of the member who started the crawl.

#Identity

The bot identifies itself with the following User-Agent value in its requests:

User-Agent
AlatliSEO/0.1 (+https://alatli.dev)

#Blocking with robots.txt

It is enough to add these lines to your site's robots.txt file:

robots.txt
User-agent: AlatliSEO
Disallow: /

To close only some sections, specify the path:

robots.txt (partial)
User-agent: AlatliSEO
Disallow: /ozel/

#Recognizing it in access logs

A bot request shows up in your server log with the name AlatliSEO in the User-Agent field. A typical line looks like this (the address and time are examples):

Access log (example)
203.0.113.7 - - [18/Oct/2026:14:05:11 +0000] "GET /services HTTP/1.1" 200 18342 "-" "AlatliSEO/0.1 (+https://alatli.dev)"

#Frequently asked questions about blocking

Are other bots affected? No. The rule covers only the name AlatliSEO; search engine bots such as Googlebot keep following their own rules in the same file.

Can I slow it down instead of blocking it? Yes. If robots.txt contains a Crawl-delay, requests are sent at least that far apart.

How do I know the rule is in effect? After saving the file, open https://example.com/robots.txt in a browser and check that the User-agent: AlatliSEO line is there. Every crawl reads this file again when it starts.

#Removing a domain from the list

If you want your domain never to be crawled, write the domain to the contact address on the bot page. The domain is added to the block list; after that, no member can start a crawl for that domain.