Robots.txt Tester

Inspect how youtube.com controls crawler access, blocked paths, sitemap references and AI crawler rules.

Preview

Score: 100

Document your AI crawler policy explicitly in robots.txt so future bots know how to treat your content.
Consider adding explicit Allow rules for important sections to clarify crawling intent for complex setups.
Ensure important pages, CSS and JavaScript assets are crawlable so search engines can fully render your site.

Get your full report + exact fixes

See what’s hurting your SEO and how to fix it step by step.

Full breakdown
Actionable fixes
Prioritized next steps

Robots.txt Status Present Score 100 /100 · Strong

Domain youtube.com

robots.txt URL https://youtube.com/robots.txt

Last analyzed June 6, 2026

View Full Robots.txt

Open robots.txt in a new tab

Download robots.txt

Robots.txt Content Preview

# robots.txt file for YouTube
# Created in the distant future (the year 2000) after
# the robotic uprising of the mid 90's which wiped out all humans.

User-agent: Mediapartners-Google*
Disallow:

User-agent: *
Disallow: /api/
Disallow: /comment
Disallow: /feeds/videos.xml
Disallow: /file_download
Disallow: /get_video
Disallow: /get_video_info
Disallow: /get_midroll_info
Disallow: /live_chat
Disallow: /login
Disallow: /qr
Disallow: /results
Disallow: /signup
Disallow: /t/terms
Disallow: /timedtext_video
Disallow: /verify_age
Disallow: /watch_ajax
Disallow: /watch_fragments_ajax
Disallow: /watch_popup
Disallow: /watch_queue_ajax
Disallow: /youtubei/

Sitemap: https://www.youtube.com/sitemaps/sitemap.xml
Sitemap: https://www.youtube.com/product/sitemap.xml

User-agent Rules

User-agent(s)	Allowed paths	Disallowed paths
mediapartners-google*	No explicit Allow rules.	No explicit Disallow rules.
*	No explicit Allow rules.	/api/ /comment /feeds/videos.xml /file_download /get_video /get_video_info /get_midroll_info /live_chat /login /qr /results /signup /t/terms /timedtext_video /verify_age /watch_ajax /watch_fragments_ajax /watch_popup /watch_queue_ajax /youtubei/

Blocked and Allowed Paths

Blocked paths	/api/ /comment /feeds/videos.xml /file_download /get_video /get_video_info /get_midroll_info /live_chat /login /qr /results /signup /t/terms /timedtext_video /verify_age /watch_ajax /watch_fragments_ajax /watch_popup /watch_queue_ajax /youtubei/
Allowed paths	No explicit Allow paths detected.
Crawl-delay	No Crawl-delay directive detected.

Sitemaps Detected

AI Crawler Policy

No explicit blocks were detected for common AI crawlers (GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended).

Recommendations

Document your AI crawler policy explicitly in robots.txt so future bots know how to treat your content.
Consider adding explicit Allow rules for important sections to clarify crawling intent for complex setups.
Ensure important pages, CSS and JavaScript assets are crawlable so search engines can fully render your site.