User-agent: * # 1. Block Internal Search Results (SEO Best Practice to save Crawl Budget) Disallow: /search Disallow: /search/ Disallow: /search.html Disallow: /*?q= Disallow: /*&q= Disallow: /404.html Disallow: /404 # 2. Block Tracking Parameters (Prevent Duplicate Content) Disallow: /*?*utm_ Disallow: /*?*utm-source Disallow: /*?*utm_medium Disallow: /*?*fbclid Disallow: /*?*gclid Disallow: /*?*ref Disallow: /*?*referrer Disallow: /*?*session Disallow: /*?*share= Disallow: /*?*filter= Disallow: /*?*sort= Disallow: /*?*lang= # 3. Explicitly Allow Assets & Main Content (Ensure Google can render the page) Allow: / Allow: /assets/ Allow: /images/ Allow: /*.css$ Allow: /*.js$ Allow: /*.png$ Allow: /*.jpg$ Allow: /*.jpeg$ Allow: /*.gif$ Allow: /*.svg$ Allow: /*.webp$ Allow: /*.json$ Allow: /*.xml$ # 4. Sitemap Location Sitemap: https://vnlibs.com/sitemap.xml