[Housefly]

Can anyone recommend me the best text to be inserted on robots.txt to avoid all the web scrapers and also if possible use tarbits to give a teeth to robots.txt

[switch-IT]

User-agent: *
Disallow: /


it won't stop scrapers but will stop mainstream search engines.

[Housefly]

User-agent: *
Disallow: /


it won't stop scrapers but will stop mainstream search engines.

well i needed something more advance like the one with tarbits and shit like that