How do you do the crawls without getting banned? Are you using like 80legs or AW... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

leej on March 25, 2010 | parent | context | favorite | on: Duck Duck Go is starting to get coverage

How do you do the crawls without getting banned? Are you using like 80legs or AWS for from cloud crawling?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact