# Skryber — robots.txt # Crawl rules: index the marketing pages, keep authenticated/admin/API surfaces out. User-agent: * # Content Signals (contentsignals.org / IETF AIPREF) — how AI systems may use this content: # search=yes → may be included in search indexes # ai-input=yes → agents may use this content to answer / act on a user's behalf # ai-train=no → may NOT be used to train AI models Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / Disallow: /admin Disallow: /api/ Disallow: /dashboard Disallow: /editor Disallow: /auth/ # v682: internal harnesses + legacy/debug pages (reachable by link, not for search) Disallow: /dev/ Disallow: /mobile-debug Disallow: /youtube-simple Disallow: /video-processor Sitemap: https://www.skryber.com/sitemap.xml Host: https://www.skryber.com