I wonder if someday in 30 years people will look back in the same way and say: wow, you know there was once a time when search engines were major feats of engineering - and Decentralized Networks were deemed impractical!
They just as us, were resource limited. The modern constraints have merely shifted. They came up with amazing tricks to get around physical bottlenecks - just as we do now. We tend to call our current foci engineering instead of 'bags of tricks/hacks'. But when technology progresses till we take things for granted that were once hard, we young ones look back with a mix of disbelief and awe that people used to go through so much trouble for something that is now so trivial.
I think search engines will always be major feats of engineering - the big difference between a spell-checker and a search engine is in the former the problem space rarely evolves.
An English dictionary, for example, only has a a small number of words added each year. A search engine is a different thing entirely: not only are new items added all the time, but the format of these items evolves. Today search engines need to deal with non-textual formats such as videos, images, and audio. This is a very difficult problem.
The fact that there are now APIs and libraries that let developers use things like latent semantic mapping without having to roll their own algorithms and solutions can certainly make writing an intelligent textual search engine a lot easier than it used to be.
But the key difference here is that what we demand from a 'search engine' (and by this I mean Google, or Bing) ten years will be much greater than what we demand now. We'll have new media to search for, new types of queries, new expectations. But a spell checker will always be a spell checker.
At least, that's my opinion. I'm sure someone will disagree!
Sure, I'll disagree :) When you have enough memory bandwidth to compute the relationships of all pages in just a few seconds, then you can re-index the entire corpus according to whatever criteria you want (PageRank or face detection or whatever) for each query. So you have some spiders that add content to the database, and re-index the database for every query.
Good point - I don't know if that solves the problem of getting different forms of media into the system in the first place though. I think as long as people create and new types of media there will be interesting problems to search for them. Searching video based off of text queries, for example, is a much harder problem than simple text search!
They just as us, were resource limited. The modern constraints have merely shifted. They came up with amazing tricks to get around physical bottlenecks - just as we do now. We tend to call our current foci engineering instead of 'bags of tricks/hacks'. But when technology progresses till we take things for granted that were once hard, we young ones look back with a mix of disbelief and awe that people used to go through so much trouble for something that is now so trivial.