FindSignalBot

FindSignalBot is the crawler behind FindSignal. It fetches a small, bounded set of public pages from a website when someone starts an audit of that site, and checks them for technical SEO, AEO and GEO issues.

How to recognise it

Requests carry this User-Agent:

FindSignalBot/0.2.0 (+https://findsignal.simlofi.com/bot)

What it does

• Reads /robots.txt before anything else and obeys the group for FindSignalBot, or * when there is none. If robots.txt cannot be read because of a server error or timeout, it fetches nothing.
• Fetches one page at a time per host, at least 1 second apart, and honours Crawl-delay up to 10 seconds.
• Backs off on HTTP 429 or 503 and stops after a second rate-limit response.
• Stays on the audited origin; links to other sites are counted, never fetched.
• Fetches at most 100 pages per audit, only HTML up to 2 MB each, and never sends or stores cookies.
• Does not run JavaScript, submit forms or log in.

Opting out

Add this to your robots.txt to block it completely:

User-agent: FindSignalBot
Disallow: /

Contact

Questions or problems, or to ask us to stop scanning your site: contact us.