[feature] Block Amazonbot (#2692)

Blocks the Amazon crawler bot. Closes: #2686
2024-02-27 14:25:08 +01:00 · 2024-02-27 14:25:08 +01:00 · 6528592dd2
parent 9cadc764b3
commit 6528592dd2
1 changed files with 5 additions and 0 deletions
--- a/internal/web/robots.go
+++ b/internal/web/robots.go
@ -64,6 +64,11 @@ Disallow: /
 User-agent: WellKnownBot
 Disallow: /

+# Block Amazonbot, because Amazon.
+# https://developer.amazon.com/amazonbot
+User-agent: Amazonbot
+Disallow: /
+
 # Rules for everything else.
 User-agent: *
 Crawl-delay: 500