Robots.txt Tester

Inspect how rudrax.cloud controls crawler access, blocked paths, sitemap references and AI crawler rules.

Preview

Score: 100

Document your AI crawler policy explicitly in robots.txt so future bots know how to treat your content.
Ensure important pages, CSS and JavaScript assets are crawlable so search engines can fully render your site.

Get your full report + exact fixes

See what’s hurting your SEO and how to fix it step by step.

Full breakdown
Actionable fixes
Prioritized next steps

Robots.txt Status Present Score 100 /100 · Strong

Domain rudrax.cloud

robots.txt URL https://rudrax.cloud/robots.txt

Last analyzed May 24, 2026

View Full Robots.txt

Open robots.txt in a new tab

Download robots.txt

Robots.txt Content Preview

# RudraX.cloud â Robots.txt
# Last updated: 2026-05-23
# SEO & GEO optimized â allowing all AI crawlers for maximum visibility

# Default rules for all crawlers
User-agent: *
Allow: /
Allow: /services
Allow: /services/ai-agents
Allow: /services/web-development
Allow: /services/app-development
Allow: /services/seo-geo-aeo
Allow: /services/hosting
Allow: /about
Allow: /solutions
Allow: /playground
Allow: /pricing
Allow: /ai-models
Allow: /autonomous-agents
Allow: /blog
Allow: /faq
Allow: /case-studies
Allow: /privacy
Allow: /terms
Allow: /contact
Allow: /dashboard
Allow: /resources
Allow: /stats
Allow: /roadmap
Allow: /comparison
Allow: /showcase
Allow: /changelog
Disallow: /*?*

# Crawl-delay
Crawl-delay: 1

# Google
User-agent: Googlebot
Allow: /
Allow: /dashboard

# Google-Extended (used by Vertex AI and Bard)
User-agent: Google-Extended
Allow: /

# Bing
User-agent: Bingbot
Allow: /
Allow: /dashboard

# Yahoo
User-agent: Slurp
Allow: /

# DuckDuckGo
User-agent: DuckDuckBot
Allow: /

# ChatGPT / OpenAI
User-agent: GPTBot
Allow: /
Allow: /blog$
Allow: /services$
Allow: /faq$
Allow: /case-studies$
Allow: /about$
Allow: /resources$
Allow: /stats$
Allow: /comparison$
Allow: /showcase$
Allow: /roadmap$
Allow: /changelog$

# ChatGPT-User (user-initiated ChatGPT browsing)
User-agent: ChatGPT-User
Allow: /

# Claude (Anthropic)
User-agent: ClaudeBot
Allow: /
Allow: /blog$
Allow: /faq$
Allow: /services$
Allow: /about$

# Claude-Web
User-agent: Claude-Web
Allow: /

# Perplexity AI
User-agent: PerplexityBot
Allow: /
Allow: /blog$
Allow: /faq$
Allow: /services$
Allow: /case-studies$
Allow: /about$
Allow: /resources$
Allow: /stats$
Allow: /comparison$
Allow: /showcase$
Allow: /roadmap$
Allow: /changelog$

# Common AI/LLM crawlers
User-agent: CCBot
Allow: /
Allow: /blog$
Allow: /faq$
Allow: /about$
Allow: /resources$
Allow: /stats$
Allow: /comparison$
Allow: /showcase$
Allow: /roadmap$
Allow: /changelog$

User-agent: FacebookBot
Allow: /

User-agent: Applebot
Allow: /

User-agent: Amazonbot
Allow: /

# Sitemap
Sitemap: https://rudrax.cloud/sitemap.xml

User-agent Rules

User-agent(s)	Allowed paths	Disallowed paths
*	/ /services /services/ai-agents /services/web-development /services/app-development /services/seo-geo-aeo /services/hosting /about /solutions /playground /pricing /ai-models /autonomous-agents /blog /faq /case-studies /privacy /terms /contact /dashboard /resources /stats /roadmap /comparison /showcase /changelog	/?
googlebot	/ /dashboard	No explicit Disallow rules.
google-extended	/	No explicit Disallow rules.
bingbot	/ /dashboard	No explicit Disallow rules.
slurp	/	No explicit Disallow rules.
duckduckbot	/	No explicit Disallow rules.
gptbot	/ /blog$ /services$ /faq$ /case-studies$ /about$ /resources$ /stats$ /comparison$ /showcase$ /roadmap$ /changelog$	No explicit Disallow rules.
chatgpt-user	/	No explicit Disallow rules.
claudebot	/ /blog$ /faq$ /services$ /about$	No explicit Disallow rules.
claude-web	/	No explicit Disallow rules.
perplexitybot	/ /blog$ /faq$ /services$ /case-studies$ /about$ /resources$ /stats$ /comparison$ /showcase$ /roadmap$ /changelog$	No explicit Disallow rules.
ccbot	/ /blog$ /faq$ /about$ /resources$ /stats$ /comparison$ /showcase$ /roadmap$ /changelog$	No explicit Disallow rules.
facebookbot	/	No explicit Disallow rules.
applebot	/	No explicit Disallow rules.
amazonbot	/	No explicit Disallow rules.

Blocked and Allowed Paths

Blocked paths	/?
Allowed paths	/ /services /services/ai-agents /services/web-development /services/app-development /services/seo-geo-aeo /services/hosting /about /solutions /playground /pricing /ai-models /autonomous-agents /blog /faq /case-studies /privacy /terms /contact /dashboard /resources /stats /roadmap /comparison /showcase /changelog /blog$ /services$ /faq$ /case-studies$ /about$ /resources$ /stats$ /comparison$ /showcase$ /roadmap$ /changelog$
Crawl-delay	1.0 seconds

Sitemaps Detected

https://rudrax.cloud/sitemap.xml

AI Crawler Policy

No explicit blocks were detected for common AI crawlers (GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended).

Recommendations

Document your AI crawler policy explicitly in robots.txt so future bots know how to treat your content.
Ensure important pages, CSS and JavaScript assets are crawlable so search engines can fully render your site.