mirror of
https://github.com/superseriousbusiness/gotosocial
synced 2025-01-31 17:45:03 +01:00
4604224c4d
This syncs our copy with the current state of the ai.robots.txt repository. Upstream has tightened their scope to be AI-only, whereas before it included a bunch of SEO and "web intelligence" marketing stuff. I've kept those but moved them into their own section.