devol/wallabag
1
0
Fork 0
mirror of https://github.com/wallabag/wallabag.git synced 2024-12-15 09:57:41 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #3510 from wallabag/update-robotstxt

Browse Source 
Updated robots.txt to prevent crawling
This commit is contained in:
Nicolas Lœuillet 2017-12-16 09:02:40 +01:00 committed by GitHub
commit 9b606725fe
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
web/robots.txt
View File

@ -1,4 +1,2 @@
# www.robotstxt.org/
# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
User-agent: * User-agent: *
Disallow: /