User-agent: Scrapy Allow: / # XML Sitemap & Google News version 5.3.3 - https://status301.net/wordpress-plugins/xml-sitemap-feed/ Sitemap: https://www.wk.nl/sitemap.xml User-agent: * Disallow: