Hacker News new | past | comments | ask | show | jobs | submit login

Wish someone would meta search these search engines



There's fairly aggressive bot activity when you run a search engine, which is why many of us are behind cloudflare or similar bot-mitigation services.

Best guess is the bots are attempting to manipulate Google or Bing's suggestion algorithms this way (since a lot of small search engines are basically just forwarding results from bigger engines). Unfortunately they can be very aggressive to the point of almost bringing down your service. I saw up to 30k searches per hour before I cried uncle and got behind cloudflare.

I do offer an API, and I suspect some others in the indie search space might as well (or might at least be willing to set one up). That way I can rate limit on a per-consumer basis without affecting everyone.


Searx and SearxNG support a few of these, but instances get blocked and unblocked left and right.

eTools is a metasearch engine that uses commercial APIs for its search providers, so it does not get blocked. However, each search is a bit expensive for eTools, so users might get blocked by eTools instead.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: