# goriva.online crawler policy # Search engines and user-requested AI browsing may access public pages. # Automated collection for AI training, dataset creation, resale, mirroring # or bulk extraction is not authorized. User-agent: * Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: GPTBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: CCBot Disallow: / User-agent: Bytespider Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Applebot-Extended Disallow: / Sitemap: https://goriva.online/sitemap.xml