Hacker News new | past | comments | ask | show | jobs | submit login

But it's the perfect way to circumvent robots.txt! With enough users the law of large numbers applies (and how many hundreds of millions of users are using IE?) so you can a.s. copy Google's index.

In particular, you copy the most popular results, so if you weight the % of google's index that you copied, then it's going to be a huge number, and almost for free.

It's evil. But devilishly smart.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: