User-agent: * Allow: / # Explicit access for AI crawlers User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: PerplexityBot Allow: / User-agent: ClaudeBot Allow: / Sitemap: https://shopage.org/sitemap.xml