website_template/robots.txt

# www.robotstxt.org/

# Allow crawling of all content
User-agent: *
Disallow:
Some changes 2015-07-27 09:49:42 +02:00			`# www.robotstxt.org/`
Some cleanup and updates 2015-07-27 12:25:14 +02:00
			`# Allow crawling of all content`
Some changes 2015-07-27 09:49:42 +02:00			`User-agent: *`
Some cleanup and updates 2015-07-27 12:25:14 +02:00			`Disallow:`