4604224c4d
This syncs our copy with the current state of the ai.robots.txt repository. Upstream has tightened their scope to be AI-only, whereas before it included a bunch of SEO and "web intelligence" marketing stuff. I've kept those but moved them into their own section. |
||
|---|---|---|
| .. | ||
| about.go | ||
| assets.go | ||
| confirmemail.go | ||
| customcss.go | ||
| domain-blocklist.go | ||
| etag.go | ||
| index.go | ||
| profile.go | ||
| robots.go | ||
| rss.go | ||
| settings-panel.go | ||
| signup.go | ||
| tag.go | ||
| thread.go | ||
| web.go | ||