Hacker News new | past | comments | ask | show | jobs | submit login

Yep; there may be a lack of incentive to preserve old sites, but what's worse are the ranking algorithms that prevent their discoverability in the first place.



Both the Internet Archive and Common Crawl have tools that reveal actual crawl dates. Search engines are not really intended to be archives, so it's no surprise that they aren't very good archives.


Is it, though? I think you have to define what your search engine is searching to make a claim like that. Internet Archive and Common Crawl (which I will say has its own incentives discouraging the discoverability of old sites through its methodology and limitations of its web crawling) are search engines in their own right.

What are you doing when you use their services? Searching.


Not really prevented, the huge one is http sites being down ranked heavily by google.

But they are still there. Do a specific enough search and they'll be at the top of the search results.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: