beReferencedBot

Technical SEO & GEO Analysis Scanner

What is beReferencedBot?

beReferencedBot is the web crawler used by beReferenced, a SaaS platform that helps businesses optimize their visibility on AI platforms (ChatGPT, Claude, Gemini, Perplexity) and traditional search engines.

Our bot performs technical SEO audits to help website owners understand and improve their online presence in the age of Generative Engine Optimization (GEO).

How Does It Work?

Respects robots.txt

We check and respect your robots.txt directives before scanning. If you disallow our bot, we won't scan your site.

Rate Limited

We apply a minimum 2-second delay between requests to the same domain to avoid overloading your server.

Public Data Only

We only analyze publicly accessible pages. No authentication, no form submissions, no personal data collection.

Identifiable

Our User-Agent clearly identifies us as beReferencedBot, making it easy to track in your server logs.

What We Analyze

Technical SEO: Meta tags, headings structure, canonical URLs, robots directives

Structured Data: Schema.org markup (JSON-LD) for better AI understanding

Content Quality: Keyword density, semantic coherence, content structure

AI Bot Permissions: robots.txt analysis for AI crawlers (GPTBot, ClaudeBot, etc.)

Site Architecture: Detection of key pages (About, Contact, FAQ, Blog, etc.)

Technology Stack: Frameworks, CMS, and libraries used

User-Agent String

beReferencedBot/1.0 (+https://bereferenced.com/bot) Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36

You can use this string to identify our bot in your server logs or analytics.

How to Control Access

You can control beReferencedBot's access to your website using your robots.txt file:

Block beReferencedBot completely:

User-agent: beReferencedBot
Disallow: /

Block specific sections:

User-agent: beReferencedBot
Disallow: /admin/
Disallow: /private/

Allow all access (default):

User-agent: beReferencedBot
Allow: /

Questions or Concerns?

If you have questions about our bot, need to report an issue, or want to request removal from our scanning list, please contact us:

Email:[email protected]