Website_jan.su/_source/robots.txt

10 lines
119 B
Plaintext
Raw Normal View History

2019-03-26 23:25:24 +01:00
# www.robotstxt.org/
# Allow crawling of all content
User-agent: *
Disallow:
2023-08-07 22:30:09 +02:00
# No GPT
User-agent: GPTBot
Disallow: /