# Zaleso

# --- AI search / retrieval crawlers: ALLOW. These make the site citable. ---
User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

# --- Training-only crawlers: no citation benefit. Blocked. ---
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: CCBot
Disallow: /

# --- Everyone else ---
User-agent: *
Allow: /

Sitemap: https://zaleso.com/sitemap-index.xml

# NOTE: robots.txt is not sufficient on its own. Cloudflare's default since
# 15 Sept 2026 blocks "mixed-use" crawlers from ad-carrying pages. AI search
# crawlers must ALSO be explicitly allowed in the Cloudflare dashboard.
# Audit quarterly. See CLAUDE.md §10.