Crawler

What is this bot?

SEO Rapor is a service that visits a site's public pages, finds technical SEO issues such as titles, descriptions, links, redirects and images, and produces a report. A member of our service (usually the site owner, an agency or a prospective client) starts the scan.

What it does and doesn't do

• It only reads public pages (GET requests). It does not log in, submit forms or change anything.
• It sends 2–5 requests per second; on sites whose ownership hasn't been verified the speed is at its lowest.
• robots.txt rules are respected; it does not enter disallowed pages.
• When a member adds a site or prepares a scan, it reads robots.txt and sitemap files to estimate the page count. If there is no sitemap or it has very few URLs, it briefly crawls up to 100 pages starting from the home page, at the same speed and following the same robots.txt rules; at most once a minute per domain.
• It runs only one scan at a time per domain and limits the number of scans per day.
• It does not publish the content of scanned pages; results appear only in the report of the member who started the scan.

How to block it with robots.txt

Add these lines to your site's robots.txt file:

To close only some sections, you can specify a path such as Disallow: /ozel/.

Removing your domain from scanning

If you never want your domain scanned, write to info@alatli.dev with the domain name. We add it to our block list; after that no member can start a scan for that domain.