Hacker News new | past | comments | ask | show | jobs | submit login

That one got even picked up by wired, this also contains an interview with one of their developers at the time http://www.wired.com/epicenter/2010/08/robot-laws/



Good catch - didn't even know that. I came across this thanks to a tweet by Mikko Hypponen[1]. It was one of the top three or four.

Fox News's robots.txt[2] is pretty interesting too - they deliberately block at least one article from being indexed[3]

And YouTube[4], as usual, shows some humor in the form of Flight Of The Conchords lyrics.

[1] http://twitter.com/#!/mikkohypponen/status/10344080313274777...

[2] http://www.foxnews.com/robots.txt

[3] http://www.foxnews.com//politics/2011/07/21/wynn-slams-obama...

[4] http://www.youtube.com/robots.txt




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: