# START YOAST BLOCK
# ---------------------------
User-agent: *
Allow: /
Disallow: /?s=
Disallow: /feed
Disallow: /*add-to-cart=*
Disallow: /?wc-ajax=get_refreshed_fragments
Disallow: /my-account/
Disallow: /shop/?filter_
Disallow: /*?filter_
Disallow: /*?query_type_
Disallow: /*?action=
Disallow: /idscode/

Sitemap: https://www.barcare.co.uk/sitemap_index.xml
# ---------------------------
# END YOAST BLOCK

# --- Added 27 Aug 2026 -------------------------------------------------------
# Throttle Amazonbot.
#
# Measured on 26 Aug 2026 over 24h of nginx access log: Amazonbot made 2,789
# requests, the heaviest crawler on the site by a wide margin and more than 13x
# Googlebot's 206. It feeds Amazon/Alexa search, which is worth little to a UK
# B2B catering supplier.
#
# The Disallow lines below are DELIBERATELY DUPLICATED from the User-agent: *
# group above. A crawler obeys only the most specific group that matches it, so
# an Amazonbot group containing nothing but Crawl-delay would exempt Amazonbot
# from every rule above and let it crawl MORE, not less. If you edit the rules
# above, edit them here too.
#
# NOT throttled, deliberately: GPTBot, ClaudeBot, PerplexityBot, YouBot,
# Googlebot and bingbot. The Pillar 1 AI-citation work depends on those crawlers
# reading the site, and slowing them would work against it.
User-agent: Amazonbot
Crawl-delay: 10
Disallow: /?s=
Disallow: /feed
Disallow: /*add-to-cart=*
Disallow: /?wc-ajax=get_refreshed_fragments
Disallow: /my-account/
Disallow: /shop/?filter_
Disallow: /*?filter_
Disallow: /*?query_type_
Disallow: /*?action=
Disallow: /idscode/
# -----------------------------------------------------------------------------