Website_jan.su/_source/robots.txt

# www.robotstxt.org/

# Allow crawling of all content
User-agent: *
Disallow:

# No GPT
User-agent: GPTBot
Disallow: /
Added some meta txt 2019-03-26 23:25:24 +01:00			`# www.robotstxt.org/`

			`# Allow crawling of all content`
			`User-agent: *`
			`Disallow:`
Block OpenAI Bot 2023-08-07 22:30:09 +02:00
			`# No GPT`
			`User-agent: GPTBot`
			`Disallow: /`