User-agent: * Disallow: */flyer$ Disallow: */studiegids-aanvragen* Disallow: */contact$ Disallow: /reviewwidget.js* Disallow: /searches/widget* Disallow: /autocomplete/* Disallow: */informatie-aanvragen* Disallow: */extra-aanvragen* Disallow: */download* Disallow: */registerClick* Disallow: */alternatieven Disallow: */training-review* Disallow: */ervaring-bedankt* Disallow: */ervaringen/* Disallow: /users/* Disallow: /mijn-springest Disallow: /members Disallow: */inschrijven Disallow: */inschrijving-bevestigen/* Disallow: */inschrijving-bevestigen-bedankt Disallow: */inschrijven-bedankt Disallow: */betaling/* Disallow: */betaling-bedankt/* Disallow: */betaling-gefaald/* Disallow: */betaling-opnieuw/* Disallow: /mollie-callback/* # https://yandex.com/support/webmaster/controlling-robot/robots-txt.xml User-agent: Yandex Crawl-delay: 30 # http://www.baidu.com/search/spider.html User-agent: Baiduspider Crawl-delay: 30 # https://help.yahoo.com/kb/guide-meta-tags-robotstxt-slurp-sln2213.html User-agent: Slurp Crawl-delay: 30 # http://www.semrush.com/bot.html) User-agent: SemrushBot Disallow: / # http://mj12bot.com/ User-agent: MJ12bot Disallow: / User-agent: Linguee Bot Disallow: / # http://ltx71.com/ User-agent: ltx71 Disallow: / # http://webmeup-crawler.com/ User-agent: BLEXBot Disallow: / # https://www.seokicks.de/robot.html User-agent: SEOkicks-Robot Disallow: / # http://www.exabot.com/go/robot User-agent: Exabot Disallow: / User-agent: seoscanners.net Disallow: / # http://www.sogou.com/docs/help/webmasters.htm User-agent: Sogou Disallow: /