Well whether you add “site:.edu” or not, Google still does the thing where they just spit out a pile of extra “matches” that don’t contain your search phrase AT ALL. Just thrown into the list, as if they have any business being there (oh wait, they do have business: Google’s business).
Frankly, if it’s not reliable as a tool, it has no real value and I can’t believe it has come to this. Imagine if every time you ran `ls` in a shell with a glob pattern, it just decided to sort of “add in” a few other files loosely based on your query (or heck, files that aren’t based on it at all)? Oh, now imagine if `rm` did that.
Sadly this happens with lots of search tools now. Why the heck is the default state on a new Mac to funnel what you type to everything, e.g. I searched for “Chrome” and hit Return and the FIRST thing it did was throw me into the App Store and call up some not-even-a-web-browser scam app with Chrome in its name, instead of selecting the Chrome already installed on my computer and opening it? More and more it seems that you have to turn off all kinds of poor defaults to put tools into a useful state, or there is simply no way to get them there at all.
Unfortunately the fuzzy search thing is in a kind of feedback loop in terms of user expectation. Because of Google users are used to typing the wrong thing and getting the right answer. If I search for "how old is Roger Redford" Google tells me that Robert Redford was born in 1936 (it doesn't even display the "Did you mean...?" correction). And to be fair, in 99% of cases the person probably did mean Robert Redford.
But anyway, this is the behaviour I think people are being trained to expect from searches. I sometimes have to show new users our business systems (which manage residential property data) and it's seen as a drag that you have to have some level of precision when searching for anything.
It's like a spellchecker. As they've got better over time you can be less and less accurate with words you're not quite sure on and they still find the word you meant.
In a presentation yesterday, I noticed “Jews” appear in the live closed caption stream of my talk.
It threw me off for a few seconds and I went back to the recording to figure out it took the word “choose” completely out of grammatical context to form a new clause of just “Jews”, as if I’d been speaking in complete sentences, then suddenly just decided to interject a random word between two half-sentences.
Seems like the search market is ready for Google Search Pro, where you can use advanced operators and syntax to get exactly what you asked. Just $9.99 monthly, or buy Google One and get it free!
The loss of "Did you mean..." is painful. I think that little button effectively meaning "I MEANT WHAT I SAID" is great for the general use case of typos and foggy memory, and not having it is such a loss of usability
It's not just search. It's so much user-facing interactivity online. Stop black-box-inserting irrelevant search results. Stop black-box-modifying what I type in the document. Stop black-box-suggesting-as-I-type. Stop black-box-second-guessing what I meant.
I'm a perfectly capable human who can communicate my intent, and make corrections on my own if needed. What kind of deranged hubris must these engineers have to build systems that try to proactively act on what they think I actually meant?
Given that chat texts are used in court as evidence, I wonder what happens to this when the text is not what you typed, but what the AI system "corrected" for you.
Also, I wish Google could simply deliver the information I want without tracking me or gathering information about me, my interests, communication, travel, searches, or contacts.
> Well whether you add “site:.edu” or not, Google still does the thing where they just spit out a pile of extra “matches” that don’t contain your search phrase AT ALL
There used to be a 'verbatim' mode that would do exactly that. The quotes symbol were also a way to enforce verbatim mode.
Sadly this behaviour of "let me assume what you want" is not exclusive of google as I also now experience this on ddg.
As an anecdote, we have implemented a really strict exact match at Kagi (meaning quotes do exactly what they are supposed to do). We did receive some feedback that we should relax it a bit mostly because non-alphanum character matching (some users wanted them ignored) and occasional empty results page (some users were not used to getting an empty result page as Google almost always returns something, even if it is not what they searched for).
I would also be curious if you're able to filter results by site or country domain?
(similar to now depricated google keywords with "site:foo.com" or "domain:.bar")
What I don't understand is that the verbatim option still exists, it just doesn't seem to do anything. And quotes don't seem to work anymore either. It's all very insulting.
Regarding the quotes, I actually notice they do work. However the difference is that it doesn't highlight where the string was matched in the search result index page.
So for example if you google "Food", all the results _will_ contain the string "Food" somewhere in the page (if you don't believe me, try it). It can also match somewhere in the comments or metadata which is useless..
So the feature is still there but it's nowhere as useful as it once was.
I recall a Google engineer who confirmed this once but maybe one of them lurking in the thread can clarify it?
Quotes haven’t worked for about 10 years now. For my job in 2012 I very often had to google obscure part numbers to find documentation.
Even if you added quotes to the part number (such as “foo123-x”, google would return results for “foo234-x” or “foo123-y” and bold them as if they had matched. The real part numbers could be 10-20 characters long, so it was more difficult to spot discrepancies.
I learned very quickly not to trust the results even when adding quotes. If I had assumed the quotes had worked, I would have grabbed bad documentation without even realizing it.
I would be willing to bet substantial sums of money that Google tracks metrics for these alternative results and has concluded that they add value to the user. Maybe not to you, but to most people.
Personally, I'm ok with a bit of "oh, this is what I think you meant" rather than a literal interpretation of my query. It's not perfect but neither are my queries.
> I would be willing to bet substantial sums of money that Google tracks metrics for these alternative results and has concluded that they add value to the user.
And I would be willing to be substantial sums of money that any metrics Google has with respect to "adding value to the user" are actually more directly track "adding value to the business".
I would imagine that "adding value to the user" translates to "adding value to the business" in a lot of ways for Google.
If, on average, users perceive more value from Google, they use Google more, and Google earns more ad rev. Even if this lowers value substantially for the "relative few".
It's basic utilitarianism, and it sucks to be in the "relative few"...
I too, wish that quotes were respected, and Google would stop giving results for what it thinks I meant to search for like it knows what I meant better than I do - even if that is the case...sometimes.
Marginal search results adds value for Google because it keeps people searching, which is how Google makes its money. If you find what you want, you stop feeding money into Google.
Google can do this because it has a virtual monopoly on search. If a strong competitor were to emerge, Google couldn't play these revenue optimization games, and would have to go back to being a search engine.
I certainly have moments where I don't quite know how to ask what I'm looking for and badly describe it in a query: e.g. "flat thing you use for cooking" - google seems to understand what I'm looking for, whereas bing/ddg guide me more towards cooking tools with the word "flat" in them.
I usually use ddg but I do find google useful for the more weird queries I have.
Open source alternatives are the cure (in the case of Mac hacking their search results to drive some arbitrary metric so some random PM gets promoted somewhere)
A lot harder to track me using linux instead and constantly pushing my company to allow people to use linux machines for dev
I remember the days when internet as a whole were simpler in terms of contents and users, and search engines reflected that simpicity. I am not sure if it's possible to go back to those days.
Or a search engine with zetabytes of data and the world's largest collection of AI PhDs. It could be possible that they have the advantage over some scam artist smoking cigs in front of a screen at 3am, considering it's their system.
I wonder if google could even return to its heyday if it wanted. Even if they stopped playing dumb games with search there a alot more malicious actors whose entire career is to fuck up search results now.
I feel like low-quality advertising results, low-quality-match results, and scammy results are at least 3 different bad search result types to get, and folks here are talking about the former
of course, google gets paid to show them, and I don't think anyone here realistically expects google to voluntarily stop intentionally fucking up search results unless it somehow means a larger paycheck to them
Reducing yearly revenue from 250 billion to 200 billion seems like a reasonable tradeoff for not hobbling and annoying more than half the population of the earth and also maintaining brand longevity. Right now they are further and further ripening for disruption by an upstart.
Ah yes, the good ol' days of content farms copying from stack overflow, or did you mean the good ol' days of buying back links? Or the good ol' days of web rings?
Let's face it, the heyday of honest results are just as mythical as the political "good ol' days".
I honestly don't get your snark. Those were the good old days of the Internet, when SEO spam and PageRank wasn't a thing. Then Google came and made the Internet even better, then it turned to shit, and here we are. And web rings were great, you take that back.
So yes, there was a period of time when the Internet was in some ways better than nowadays. Now I'm not saying that everything that happened since is bad, but Internet search and signal-to-noise ratio has definitely fallen off a cliff.
Let's enjoy the good days of siloes and SEO optimization. I doubt it'll get better.
Notably, around `08 or `09, these cycles stopped. From the outside, it looks like Google stopped trying to fight spam and just gave a bunch of huge sites a permanent boost in rankings instead, so at least some of your results might not be spammy garbage.
Also honest content moved into video and podcasting formats (that are harder to index) or inside walled gardens like Twitter or Facebook. It's like if the internet was a sea and Google was the ship used to navigate it, the water has dried up and now it's just craggy shoals we have to navigate around to get anywhere.
I honestly switched to google from alltheweb and altavista initially because of the easier name and muscle memory taking hold. To this day I believe Altavista was better when google first took off.
This is nitpicky but the tire changing page doesn't actually fit your rational for choosing .edu pages
>Professors are paid generous salaries to share knowledge with the paying customers of the University (students). So, let’s find some paid unpaid University knowledge on how to change a tire by using our new trick!
But the bottom of the page says
>Prepared by: [students] For Dr. Bruce Magee's English 303 (Technical Writing) Class, Winter 1997-1998.
The professor is actually an English professor and the page was written by students as an assignment. I looked into this because I was wondering, "what would a professor be teaching that he's writing notes on tire changing?". There's probably some professor out there that does that in an academic context, but there's probably a lot of things that no professor actually gets paid to write about. Maybe they would write about something miscellaneous like that anyway and put it online, but they would be doing so out of the culture of web savvy professors having personal pages where they upload stuff they think is helpful.
It isn't nitpicking at all, I had the same question and glad you uncovered this answer. I think site:*.edu is just one tool in your search toolbox that you should know about, but I wouldn't expect it to work for every topic. Still, I had never thought of it so I'm glad I read this article.
That's how a search engine should work: it returns only what you asks of it, knows how many results there are, can be forced to use your query verbatim.
This has recently been my desperation move when e.g similar bugs to what I'm facing were solved a decade ago (but were irrelevant) were coming out as top results rather than bugs for a recent version
Consider using alternative SE's like you.com or neeva, they give you the ability to rank sources and e.g. set Reddit as a preferred site so you don't have to type site:reddit.com every single time. Works great and saves time.
Yes, it matters - a writing assignment would be graded on writing quality and style. I would assume the writing professor is not a tire changing expert. You're there because you don't know how to change a tire, so there's no way to know if the information is correct.
It's certainly better than nothing but there could be specific gotchas for your situation. Most common would be needing certain tools or techniques for your model of car. It'd probably be best to either look up the manual for your car online or find a youtube video for changing the tires for your model of car.
Everything is based around SEO spam. If someone has a website, and they're producing content, their point is to monetize the browsing of the website itself, or promote their main product via the website.
This incentivizes a certain kind of content, which is marketing content.
These days I'm back to searching social networks like Reddit or Hacker News for (at least somewhat) unbiased information.
Google is a fuzzy-search address bar at this point for me.
I keep seeing this excuse but it doesn't explain why google doesn't downrank obvious bad actors that have been showing up at the top for years, or returning results that don't have the most important word in your search terms, or why they don't offer settings to permanently filter some sites. It's a problem with incentives, not SEO spam.
Most likely because the only reliable on-page indicator for detecting low quality content is the amount of ads/trackers on page which is probably the only thing that they can not directly punish a site for.
Indeed. Google used to have a mechanism to report pages - was annoying when they removed that. Even if it was just the illusion of control, it was nice to think they at least pretended to care.
They don't curate the search index by hand, and "obvious bad actors" aren't really that obvious from a "well-definable heuristic" point of view. It's obvious to a human, but it seems hard to identify via an algorithm
The SEO spam exists in part because Google does a really bad job at surfacing any site that isn’t content heavy. This leads to sites needing to churn out keyword rich fluff content in order to get any organic traffic.
I have first hand experience with this. My niche, hyper local online service (of which everyone loves) was completely undiscoverable through Google despite the niche having nearly zero relevant results of any kind, and zero competition of any kind. It seems Google didn’t like the concise, no BS one-pager for my service.
My options were to continue to spend on Facebook ads perpetually or to create SEO spam content. Within literally a month or so of generating fluff articles I was gaining a ton of organic traffic and could stop ad spending altogether.
A good example of larger company doing this is Digital Ocean. Fortunately they’ve been extremely ethical about it and the content is genuinely very good and helpful.
By hyper local I mean something relevant to a small geographic region like a city or state.
To be clear, that aspect was mostly tangential to my point.
I think the point is Google is really bad at understanding web apps, online services, or really any site that isn’t information rich.
The majority of web apps for example only really need a landing page and a sign up form. But if that’s all you have Google’s algorithm is going to show zero interest. And it seems that’s the case even if you put in the work and optimize the content and provide plenty of meta data.
Imagine you own a pizza joint, but the only way Google shows your site to any of your potential customers is if you dedicate to publishing an article about pizza every month. That’s basically the boat a lot of us are in with web apps, and other online services.
> I think the point is Google is really bad at understanding web apps, online services, or really any site that isn’t information rich.
You keep using that phrase, but it doesn't select for information richness, it selects for verbiage. This is at best orthogonal and more usually opposed to information richness.
Verbiage is the result of bad copy writing and/or lazy keyword stuffing. I don’t think they select for or reward verbiage, but for sure they’re not doing enough to treat it as a negative signal.
Well not directly, but you're heavily penalized for not keyword stuffing and then passing it through an AI tool until it is 'simple to read' (ie. says what you're trying to say extremely badly five times with almost-correct words to avoid anpiece of jargon).
As a result even the content made with earnest intent to communicate has to read exactly like blogspam in order to rank.
The other side of the coin is that users are so used to the "free" model (read: ad model) that they refuse to pay for anything directly.
Look at youtube. They show ads and there is a non-stop stream of complaints about how long/bad/annoying they are. They offer a paid subscription to remove all ads and people laugh at the audacity youtube has to ask for money.
Users DO NOT want to see/watch ads. Users DO NOT want to pay a subscription for "free" content.
Just look at the story of Vid.me, a youtube competitor start-up that promised ad-free and sub-free videos. It was a huge hit until it hit bankruptcy.
I wouldn't be so sure, agree that there is an enormous challenge to convince people to pay for something they think they are getting for free, even when it's really not free ie paying for it with time, attention, privacy.
But create enough value, provide truly differentiated and unique experiences that ad-supported models can't, include privacy etc and there is a case to be made that over time a sizable enough segment will pay for that value that you can have a sustainable business.
That is certainly our bet at Neeva, and so far we have seen a lot of growth, interest and folks opting for the premium (paid version) over the basic. Even without a full push to encouraging the premium.
YT Premium is expensive. Much more expensive than users expect.
Most people underestimate the value of their attention. Youtube earned 7 billion USD in ad revenue last quarter.
The value of our attention is a shock to most people. I suspect that YouTube may actually be losing money (as compared to ad revenue), even at the $12 / month--a price many users find outrageous.
I am calling this the 'ad gap', which is difference between what an advertiser is willing to pay, and what the user is willing to pay to avoid the ad.
One way to look at it is that users are irrational. But it can also be seen as a testament to the effectiveness of advertising.
Exactly. YouTube Premium is loaded. Its pricing is based around selling YouTube Music and YouTube Originals. I don't want either. I'd happily pay 5$/month for a YouTube Premium experience.
We’re used to free because google decided free was the answer and they had so much clout, no other answer could compete. Now that their time is ending there will be room for other answers.
I was in mood yesterday and I tried Google versus Kagi and You Code. Results of N out of N queries in other search engines were miles better in other search engines than Google.
I have now switched completely to code.you.com and Kagi from Google.
Kagi is jaw-droppingly amazing. With Google search quality declining, Bing being on average 95% as good as Google, and DuckDuckGo being 90% as good, I was doubting there was any way that search engines would ever improve. Kagi has removed that doubt completely. I ask for stuff and it finds it. I couldn't believe there could be such a useful product.
Apparently it uses Bing plus some secret sauce although in my experience the results are a bit worse than pure Bing. Having said that I have not performed a rigorous study, this is just my guess from using all these search engines over a prolonged period.
It was mostly Bing and Yandex. The latter was responsible for the odd Russian language results you'd get at times but presumably produced English results as well.
Bing may be that good for some, but every time I use it it stinks and I end up back on Google. Same for DDG.
Then again, my queries tend to be pretty specific and are probably not a typical use case.
I really expected code.you.com to be tailored to coding, but the results are nonsensical. The $ got stripped, as well as any other characters like %, #, or _. Casing is also ignored.
Searching for $PATH results in this. Yes, I definitely wanted these results:
> PATH (People Acting To Help), Inc. is a comprehensive Behavioral / Mental Health and Intellectual Disabilities center located in Northeast Philadelphia.
> Better health moves humanity forward | PATH
The scope of our work is vast because it must be. Billions of people are still underserved or marginalized by inequitable systems. To close the gaps, we advance progress in dozens of health areas, from epidemic preparedness, HIV/AIDS, and malaria, to maternal and newborn care, sexual and reproductive health, tuberculosis, and more.
I'm a product manager working on code.you.com. This is good feedback and an interesting suggestion. We'll work on adding those special characters, variables and casing to make the search better for coding.
We're just getting started but working really hard to make this something special. There is a fairly active slack community if you have any other suggestions and want to share. thx.
I don't know if I can agree, yes SEO spam is probably worse but it's grown side-by-side with Google, both sides have more money and resources.
SEO spammers decided to improve their tools/techniques.
Google realized they could earn just as much by replacing their grep-for-the-web search-engine with some NLP, ML and fuzzy matching on what I strongly suspect to be a tiny subset of the index they had before.
I understand the reasoning and I admit it does a really good job at correcting mistakes but I often wonder if anyone working on the search-engine actually uses it to find development related information.
Maybe I'm in some kind of A-B testing hell but my results often show 1 relevant random url and a tiny a stackoverflow box with other relevant information and then always the same 10-20 stackoverflow/github copycats. Scrolling down just loops those same sites over and over again. Similarly I've never had Google link to the official python docs, it's always some spammy website that copy pasted a docstring and decorated it with ads/share buttons.
I don't understand how that's not bothering people that actually work on the search-engine.
You right, but Google at least can try to fix it. For example: allow users to hide sites from search results, allow users to downrank sites in search results, etc. etc.
Then it's going to be gamed, again. This time by businesses who would love to drown their competitors search results, or malicious players who want to attack an entity's web presence.
I understand that to some degree this is an arms race between business SEO optimization and Google's SEO ranking algorithms but I don't understand why Google upranks websites with obvious annoyances like the 3 popups in the case of OP's website, excessive amount of adtracking, and unreasonably large website sizes. In those cases, I can only suspect Google upranks them because they provide ad revenue to Google.
I switched to using kagi.com a while ago, and I've never looked back. I've tried to use duckduckgo before but just ended up using '!g' most of the time.
Kagi is in a whole other league. It's on par with, if not better than google search. I highly recommend it. When they introduce the paid tier I'm sure as hell going to subscribe to it. At this point I would trade my Netflix subscription for it.
I've been using Brave Search for months now, much better than DDG. I had a Kagi invite sitting in my inbox, just signed up and set as default search.
Will give it a spin, but so far I like what I see.
EDIT: I immediately boosted all `github.com` and `reddit.com` results. That's a killer feature!
EDIT 2: better than Brave Search on mobile: there's a search button to enter a new search, and the textbox doesn't move around like Brave or Google do causing you to click on the first suggestion. I bloody hate that. Well done Kagi.
Yeah I've running Kagi recently and in most areas it outshines Google.
Love how I can easily banish garbage seo-only sites from results
I appreciate how it groups results into a more compact section if they're from forums or review sites. The UX give you the ability to scan for relevant links quicker
I really like what I've read about Kagi, and I'll likely end-up paying for it too, but I _hate_ the metered fee.
I've never run a search engine, but I don't imagine that offering "Unlimited" searches where "unlimited" means "do not abuse" would meaningfully hurt their margins. As a benefit, you avoid losing customers that then need to ask "Do I make more than 5*80 searches a month? How many is that a day? What do I if I go over?". It also just feels like I'm being nickle-and-dimed.
I love that they're charging for the service, it makes me feel like they're building something sustainable. I just don't love the fee structure.
They're not going to do a metered plan. From an email sent today:
> Kagi will come as a free version with limited use; and an unlimited use, paid option at $10 a month, both versions having great search results with less spam and completely ad-free, tracking free, and with none of your search data being retained
$10/m for a household, if this is as good as people are saying, would be a no-brainer subscription for me.
Not paying $10/m per person in my household, though. That'd be way too high.
[EDIT] Though I guess with no retained search history, we could just share one account anyway, so it doesn't matter. I was thinking about keeping those separate. Unless there are per-user settings that we might not all want to share.
I've been using it during the beta and plan on paying for it once they start charging. Previously I had tried switching to DuckDuckGo, but found myself using !g half the time because I knew the DDG results would be garbage compared to the Google results.
I haven't found Kagi's results to be significantly better than Google's results, but unlike DDG they're not hugely worse than Google. It's all the extra stuff that makes me want to use Kagi instead of Google:
* Ability to banish sites from results forever. E.g. I never want to see the Concordia Lutheran High School's website when I search for things in the Common Lisp HyperSpec with clhs.
* Ability to weight sites higher. E.g. there are two mirrors of the Common Lisp HyperSpec that appear in results, one at http://clhs.lisp.se and one at http://www.lispworks.com/documentation/HyperSpec/. The lisp.se mirror is hilariously slow for me (5-10 seconds to load) while the lispworks one is instant. Google always ranks lisp.se first which tricks me into clicking it and wasting my time. Kagi lets me weight that site lower so the lispworks site comes first. Banishing lisp.se would also work, but just upweighting it means I can still find it if the other mirror ever goes down or something.
* I reported a minor bug to Kagi and they fixed it within a few days. I doubt that would happen with Google.
* Supports DDG-style bang queries, which I use occasionally.
* No goddamn ads/tracking (I use uBlock Origin, but it's the principle of the thing). I've almost entirely de-Googled my life at this point (Youtube is the only holdout for me).
So it's not that it gives better results than Google, it's that the results aren't any worse and the extra quality-of-life features are really nice.
Basically everything you wrote above is what I'd say - except I haven't submitted a bug report or feedback to Kagi...
Basically, when I used DDG, I found myself frequently using !g to search Google instead. For Kagi, I haven't really found myself doing that anywhere near as often.
So yeah, you have to fill out a survey (took a couple of minutes and I got an invite the same/next day), so it's not the easiest to recommend right now. Two weeks and they go free+premium according to an email today.
I think it's just that I, like a lot of HN, have been getting super disillusioned with Google lately - and most importantly the preponderance of obviously machine-generated zero-value spam pages for everything I search for - so I was desperately trying to find something better. DDG constantly disappointed for my searching. Kagi doesn't do that. So I'm happy. $10/month happy? I'm not sure.
Generally, there's a growing group of people who are feeling about Google the way a lot of us felt about Facebook 10+ years ago - it's time to stop relying on it and maybe cut it out entirely (and yes, I know there are plenty of people who felt this way about Google for years already). So I pay for Fastmail now. I guess I'll likely be paying for Kagi now. Quality is getting expensive, but at least I'm paying with my money rather than my data.
Not only requires a login, but also Joe Regular cannot use it - beta means here closed invite-only, so it's just an internal tool for some group of people. So any news about it should be taken with a grain of salt.
There’s no way to test without giving your info away and taking a long survey from what I remember. It’s kind of like posting Facebook links I suppose.
Our goal at this stage (invite only beta) was not to get users who just want to try it out, but who actualy wanted to contribute by reporting bugs and suggestions as beta-testers. This is why we have a survey form, as completing it signals wilingness to invest time in the product. Soon (in two weeks) we will move to public beta.
Even then we won't have search without an account, among other things because bots would probably kill us.
Also, it predicts a video is best to answer this question and gives 3 YouTube thumbnails listed at the top of the results. They all seem generally helpful based on the title.
These top results are identical to the results I'm getting from metager.org which is just a meta search engine. Both those results came from Bing in the case of metager
this right here is why you never make any progress and never fix anything and you never understand why everything sucks, you keep rewarding perverse incentives and greed and then try to work around it as if greed were not the central problem.
making money is not noble, making money does not represent increasing value, it only represents cornering tokens of others' labor time, which you have just observed takes the form of profiteers intentionally seeking ways to waste your time.
improving the world is noble, and whether you make money or not, you get to live in a better world.
linus was awarded a salary because people were EMBARRASSED that they were profiting so much off his free work.
Making money is the practical reality and the entire foundation of the modern world. Do you really think trading chickens or loafs of bread is more practical than exchanging money for goods and services?
How ridiculously naïve! You should be ashamed that you have the utter privilege to hold such utterly ridiculous thoughts without the fear of starving or not having shelter.
get fucked eric, all slavers must be killed, you have no idea how secure food or shelter are for me, motherfucker. you think because i'm able to get here and talk to you that i'm in a situation like yours, and not facing homelessness or starvation? fuck you scum
It's probably location dependent, but I wonder if there's a bit of randomness in there as well to give websites with equivalent information similar exposure.
Me too. I think often the complaints against Google expect a bit much from it. I mean use uBlock Origin to block some of the ad nonsense and if the first search term doesn't get it try something a bit different. Also set it to 30 results not 10 and scroll a bit.
I recently found https://commoncrawl.org/ and started working on my own personal search engine. I'm surprised how easy it is to actually get decent results with just metadata that show up nowhere in Google/DDG.
It's the aggregate result of the sort of hiring they do in there. Take for example they rejected the guy who created homebrew because he couldn't invert a binary tree to the satisfaction of some jobsworth in there (1). It's jumped the shark and gone full on corporate bs, not because of any changes to what they do, but as a result of who's making the decisions in there these days.
I feel like Google could filter out a lot of SEO spam by simply ignoring or downranking heavy sites. Everything that loads more than "n MB" is probably bloated and made to trick you into buying something or generating traffic without providing value.
Another method could be to analyze how many external requests a page makes and put that into the calculation.
Of course, there are exceptions, but I think this might help.
Downranking these sites will overwhelmingly mean downranking those who spend money on Google ads. It will not happen because it is exactly contrary to Google making more money.
Of course Google has PageSpeed, which supposedly influences rankings by downranking sites that are very heavy to load, so I mean, that's an aspect to take into account, but I still believe the above to be largely true.
The presence of ads, analytics and affiliate links are a good proxy for "spam site" and is very hard to game, so downranking them is a trivial solution.
Google will never do it because those ads & analytics can be Google's and benefit their bottom line.
Those sites generate plenty of value ... for Google. Even if they're not showing Google adverts, those sites normalize the idea that "websites have adverts", so users don't question it on other sites.
If Google started returning sites with low numbers of adverts first then users would start demanding more sites with fewer ads. Google really don't want to show fewer adverts.
I was confident they already included page speed in their rankings, but who knows what the status of that is? After some (ironically) googling, I've found three blog posts from May 2020 [0], November 2020 [1] and April 2021 [2] announcing it, the last one saying it went live in August 2021. But I was sure they talked about it years ago, hence the PageSpeed Insights and Lighthouse tools.
People literally curate lists of websites that are only spam, and provide no value in search results[1]. Even if Google only cared a little bit about blocking the worst 1% of websites they don't already block, they would simply copy these lists, or use them as a starting point. Let alone devoting actual resources of their own. But they don't even do that.
Thought the same, but it would be unfair to web sites that are actual web apps and are JS-heavy.
Google is hopeless anyway, so here's something for the next search engine that wants to bring back the Web: downrank by the number of 3rd party domain requests, attempts of tracking, and possibly also popups.
If you show an example of something that pops up right after the initial load, and that a user actually desires, we'll think of ways. I can't think of any right now. Things should pop up in response to a click, generally.
Okay, but iris does have two meanings and sometimes you need to be specific in your queries. Your original query would not have been perfect if you were in fact wondering about the flower. "what is an iris anatomy site:*.edu" is enough to bring up this as the first result:
Sidetrack, but I can recommend the Firefox/Chrome extension "Don't Track Me Google" which cleans up Google result links to be the actual URL, without Google's tracking redirect.
Hey, I just came from the company that hosts that dealer website!
The amount of money that dealerships pour into SEO and advertising is unreal. They also chase every single thing that Google says with complete abandon. (Remember AMP? ya, that was something we chased because dealers asked us to and then forgot about 12 months later.)
I realize this is about Google but I think there will always be a struggle until everyone can boycott playing the game and I just don't see that happening.
Reddit has very strong biases and its own bubbles as well. Don't expect to be exposed to a wide variety of viewpoints there, but what appeals to a very narrow demographic (18-30 year old low-income white progressive American males)
There are countless small(er) subreddits where a Google 'site:' search works wonderfully. Often, I don't specify a subreddit at all if the query itself is narrow enough.
Unfortunately, I find Reddit results to generally be of very low quality. For tech related searches, stackoverflow and direct links to documentation are far superior. Where reddit is occasionally useful is when searching for things like "what's the best service for X?", but even then I find myself wading through many posts and many comments on those posts before I come across a sliver of useful information.
As an aside, anyone knows if you can still find "who links to?". I used to do this to hear how people feel about my site/app. I would do something like `link:foo.bar.com` I don't remember if it was link: or linkto: but it seems it no longer is supported? :(
I dunno about "who links to" functionality in the search engine itself, but this information can be found if you set up Google Analytics, which is probably what they want you to do anyway because that data is worth a lot to them.
Unlike Analytics at that at least does not require you to give Google any more information about your users - just to verify that you control the domain.
I think it's more about training your google algorithm, i searched the same things they searched and my results are answers to the question.
how to change a tire, nets videos of tire changing and the first 5 results are all different sites explaining how to do it.
how are pearls made, gives me an inline box answering the question.
half the things I search it offers me a "did you mean (my search) reddit" which I usually did...
google isn't great, but it's still the best by far (I like the algorithm that knows what i search and assists me with it), i'm not sure what this person has done to theirs...
The reason that cookies are not a browser setting is NOT because of a technical limitation. Besides, if it was a browser setting, I would hope most people would have default "no, you can't store cookies except for the functional ones" and not "yes, store a thousand tracking cookies".
It is because of a power issue.
It would have to be a protocol change. Google owns the largest browser. Google doesn't want it to be a browser setting. End of story. Unless you gather political power to force google and others to have this, it will not change.
There's a Firefox add-on that allows you to block specific sites from Google's search results... not every practical for the whole Internet but it is for the worst offenders. (https://addons.mozilla.org/en-US/android/addon/ublacklist/)
> If you simply google: how to fix a flat tire the first website in the search results is this piece of bloatware, MillerAutoPlaza:
I attempted to reproduce the experiment and I got a full page of useful results, all of them videos or numbered lists of the steps as in
> Step One: Find the Puncture. Once you're in a safe place, hop out of the car, and find the flat tire. ...
etc.
Maybe Google read HN and fixed that query but how to replace flat tires yields similar results.
Maybe it's because I'm not logged in into Google and they are less aggressive on showing me ads?
Maybe it's because I'm not from an English language country and they know from my IP address. Let's try with come cambiare una gomma bucata. Other useful numbered lists and videos.
I'm not always happy with Google's results but there seems to be something wrong in the experiment here.
In this specific case, it's a great example of how Google's attempts to keep people on Google has made search worse.
It's showing this result as a featured snippet because it's easy to distil into steps, and Google is hoping you'll just read the result and not leave Google.
Of course, Google's attempts to stop you leaving, and hoover up as much of the ad market as possible, also drives publishers to ever more desperate things to make money.
lol @ musk thinking cookie settings should be in the browser. Does he not know that they used to be until a certain advertising company took it out of their browser, and Mozilla then followed?
At this point I actually think it is. I believe that DDG does tweak the result they get from Bing, and mix in a few other sources. Still, even if it is just Bing, then yes, Bing is now better than Google.
Seems like a long way to go to you know, not just switch search engines. Google's had a nice run so did yahoo, hotbot, excite, alta vista... the question isn't how to fix Google. The question is...
... or use Firefox (Tor Browser after disabling tor) + uBlock Origin, and avoid all the unwanted crap most websites throw at you today. And train yourself not to automatically search for everything on Google. Instead use IMDB, Wikipedia, Stackoverflow / Stackexchange, Reddit etc. etc. by adding their search to your browser ( https://superuser.com/questions/7327/how-to-add-a-custom-sea... ).
That's google's personalisation. I like freaking ppl out when I just ask them to google a famous politician's name and compare the results. Some times they get similar results on WiFi and very different on mobile data afterwards.
Ok. I bit the bullet and typed exactly those words: "how to fix a flat tire".
This is what I got in return:
Row 1: 2 buttons marked "on a bike" and "scooter"
Row 2: single scrollable row of images of tire fixing kits with the prices.
Row 3: step by step recipe on how to fix flat on a car (7 steps long) followed by the link to the corresponding article and small image on the right side.
To me it looks straight to the point and one can't bitch about single row of ads of a relevant products.
I've given up on google for search and I'm not going back. I've been beta-testing kagi search [1] for the last couple of months and it does 95% of what I need - accurate and low-distraction and it doesn't try to sell me stuff or sell me to anyone. They're about to go into public beta and announce their pricing, and I'm pretty sure I'm in.
Kagi has been fantastic to me. The search results seem better. The option to search by EDU or discussion or by "literally anything that doesn't have 5000 trackers" with a single click is indispensable. Showing results from the internet archive and old blogs is a nice plus. Shoving all the listicles into their own compact section is great.
If this is the quality I can expect going forward, I have zero problem paying the $10/mo down the line
How come nobody is doing social search yet? Why can't I up and down vote results and maybe even comment on them or something? I don't want to ever see some of these recipe sites that show up again... I can't imagine I would if people were helping curate. Seems like an "easy" solution to SEO crapola?
I don't think there is the need of curating content in a social search way, if the SE itself is just good and alternative SE's that are coming up seem very promising.
e.g., you.com lets you pick preferred sources, so that you can tailor your search experience to your needs.
Google tried it in 2009, failed.
Microsoft tried it in 2013, failed.
Startups across the board have tried it.
I think it is because "social" usually implies to a user that they are searching for a person, not the content they create. And previously, there wasn't a ton of UGC that was relevant to a query.
Disclaimer: I'm a You.com employee working on search.
I was trying to find pre-made hand railing mounting plates, I thought to try and use google shopping because I didn't have an existing part number.
I enter my search term hit enter, for a split second I can see the exact products I want to browse but those results are quickly swapped out for a bunch of unrelated products.
.edu is a TLD for US based educational bodies. So the reason you don't see a cookie banner is because the majority (AFAIK) of US states don't mandate a cookie banner, unlike EU and UK based sites (like the Natural History Museum, based in the UK). Does this mean the Natural History Museum is a less authoritative resource than anywhere else? I'd say the opposite, actually.
And for the "change a tire" example using a .edu TLD. That's a fantastic example of how to find search results that are almost completely useless for anyone outside the US. What's a "tire", anyway? Oh, you mean "tyre". What's a "flashlight". OK, you mean a "torch".
In summary: there's a world outside the US. It's lovely out here.
You're annoyed with him because his target audience is only the United States?
Why stop at British people? I can't believe this guy didn't consider Japanese-speaking people! Why didn't OP make an article for Chinese people?! Wow, can you believe OP didn't make an article for every single possible person on the planet?
Honestly, you make a good point at the start but I don't understand your rage at the end there.
It's a little sad we have to '.edu' filter searches just to not be directed to SEO sites at this point. Kagi's and You.com's searches are really smooth and painless from this perspective. You.com also has built in site filtering which helps a ton.
Instead of fighting Google, better "just" stop using it. Leave it for good, even if it feels better than competition right now. Let's face it, Google is not going to improve. We are not their customers, and they are not going to make their product better for us. That boat has sailed long time ago.
We need to reset the web. Try out as much of alternative search engines as you can. Use the one, or multiple, that suit you better. Maybe they will get some traction in a couple of years, maybe in a decade, or so. We'll be there where we were before Google, using Yahoos, Altavista, Excite et al. and waiting for the history to repeat in a form of a new incumbent that will again make search usable.
Also, I created a pure JavaScript (no dependency, tracking, or downloads!) bookmarklet that kills all the links in a page and turns it to plain text with a visual optimization algorithm applied to make it easier to read: https://locserendipity.com/Hyper.html
The internet is much better without all the cruft.
I'm sorry; but it makes no sense to limit yourself to results form .edu websites. Ideas take way too long from the point they have been conceived to land on .edu websites. We have to find a better solution to Google.
Has anyone else noticed a significant uptick in negative posts about google search lately? Is this because it’s gotten noticeably worse recently or it’s trendy to complain about? Thinking over the last six months or so.
An Algolia search for "google search" would suggest that there's more recent traffic amongst the most popular hits, though you can find complaints dating back over a decade.
I had to look up some govt pdfs and tables recently and the government sites hosting the files hardly ever show up on the first page on Google anymore. It's a sad story.
This title bothers me because he’s clearly going after the wrong target. It’s not that Google is unbearable. It’s that people with money are more likely to invest on ranking high in Google than those who don’t have money.
This would be the case with any commercial service with financial goals, given enough time and the right motivations.
If commercial content is your problem, say it. Leave Google out of it.
At least some of the problem here needs to be laid at the feet of users relying too heavily on the web to learn everything. Your vehicle's owner manual has instructions for how to change a tire, with the bonus that it will even know where the tools that come with your car are located and whether you have some extra features like locking lug nuts.
To be fair my result in Germany is a very good article that even has a video and a good schema. Before the result I even have google's automatically generated summary of the page that's quite good to understand if the page's actually useful.
I hate Google Search for many reasons, but this example is really ill-conceived.
There is no perfect answer for a query.
There can only be contextually meaningful answers.
They can be from highly relevant to far off.
It is very hard to prove a search engine as bad in general.
It may not work for you. But it may still work for a lot more.
Google has indexed the entire Web. This is as large a span as one can take.
This is why I left Google to join Neeva. Check us out. Unlike the alternatives, we are truly investing in our search stack.
Unfortunately there’s just too much inertia with the ads-based business model making it too hard for Google to not let itself get this way. Similar challenges with other ads supported businesses.
The mentioned queries provide quite good search results for me
The first link clearly explains how to fix a tire and right underneath the first link, there's a link to Youtube video with described sections which you can click on immediately
The sites are full of tracking and ads, but that's hardly Google's fault imo
Any search engine with a profit motive is inevitably doomed as one value overtakes the other. To have a search engine that works means having one where scaling endlessly in pursuit of profit is discouraged.
I got surprised recently by more predictable Yandex search is. I started comparing it to TikTok, the algorithm is more predictable and feels that you can direct it into the pathway you want.
For searching in Hacker news, I saved the URL 'https://hn.algolia.com/?query=%s' with the search key 'hn'. So when I want to search something around here, I just type
hn something
which then replaces the %s variable with 'something'
https://hn.algolia.com/?query=something
There is Javascript code that allows for two or more arguments, but saving that as a bookmarklet isn't as neat.
I just realized that the authors suggestion is basically to implement a custom bang (using !g on ddg will send you to the google search, !a to amazon, etc.) for .edu.
I'm sorry but in no way should you limit your results to only those published on .edu websites. There's got to be a better solution to this. Ideas only end up on .edu websites after a long period from the point it was conceived.
imagine what it will be in a thousand years, all search will be impossible as all content will be encased in layers upon layers of SEO optimizations and adds ...
Google is so unbearable because they are no longer interested in you finding the answer to your query; they're interested in selling your query and its context to the highest bidder. That conflict of interest manifests in what Google has become.
It is sad because I spent nearly 10 years at Google and nearly all the people I met were well-meaning, seemingly harmless, engineering-oriented, and data-driven. But the configuration of people that makes up the superorganism that is Google nowadays is driven by both local incentives and a broad top incentive to grow. Scale to more computation, more storage, more bandwidth, and with lower overhead--morally, those are neutral to positive, IMHO, but they are put in service of the real goal at the end of the day: swallow more money. More revenue, more markets, more ads, more engagement, more watch time...muscle out the competition, dominate. 20% YoY growth for nearly 25 years running, and suddenly you get huge, scary numbers. And you have to do insane things to keep that exponential curve from regressing to an S-curve like the laws of physics dictate it must.
The overall problem -- and this is not specific to Google -- is that managers at all levels are incentivized to grow head counts.
Google has at least 10x as many employees as it needs. That's based on decades of revenue being decoupled from costs, and from monopoly profits. It's the double-whammy of being in tech (where costs and revenues don't couple since incremental costs are close to zero), and having a super-profitable monopoly.
You can't fire 90% of the workforce; even morale and culture problems aside, at this point, the organization is structured to need too many people. On the other hand, organizing over 100k employees turns into this superorganism problem.
At >100k employees, the influence of anyone -- up to and including the CEO -- is very, very limited. Dynamics take over.
I've been bearish on Google for a long time, but I haven't shorted the stock. Market irrationality can last a lot longer than my wallet. Fundamentally, though, one could muscle through building a new, better Google for much less than the valuation of Google.
Google could also be worth a lot more with a breakup. There is negative synergy between, for example, advertising, Google Workspace, Android, hosting, etc. If those were independent businesses, with partnerships around things like data sharing, the total value could be much higher. Alphabet was a move in the right direction, but the move should have been much deeper. Right now, Alphabet is Google + a bunch of tiny startups.
> The overall problem -- and this is not specific to Google -- is that managers at all levels are incentivized to grow head counts.
No one is incentivised to grow head count. They are incentivised to do more, produce more revenue, and expand the reach of the company; most of these goals require the application of more smart people to a problem. They are not getting a promotion or salary bump because their group increased from 200 to 300 people over the previous quarter, they are receiving rewards for being able to effectively direct those people in order to achieve some specific goal. Increasing head count by 50% while delivering 250% growth in revenue/engagement/view/etc is what is being rewarded.
- If I have a job at Google as an IC, I can look for IC jobs.
- If I have a job managing 10 people, I can get a manager-level job.
- If I have a job managing 100 people, I can get a director-level job.
- If I have a job managing 1000 people, I can look for a VP-level job.
- If I have a job managing 10,000 people, I can look for a C-suite job.
Average tenure in a position in the tech industry is around 3 years. No one will know or care what objectives a Google manager achieved, or whether those were hard or easy to achieve. If you want to grow rapidly in your career, the things which matter are:
- Connections, network, and references
- What shows up on an interview (e.g. self-improvement)
- What shows up on your CV (e.g. how many people you managed, or the brands you worked on)
The place techies get stuck in career growth is by trying to do the "right thing." By the time you're trying to rise to the top of a corporate ladder, the competition is extreme, and the people who succeed play the game as optimally as they can.
I might be able to look one level up. An IC might be able to manage a small team. However, your odds of having an IC move directly into the C-suite require nothing short of a miracle (a Nobel prize, being the author of Linux, or something similar).
Respectfully, I think this is a little bit naive. While the job ladder doesn't explicitly specify org growth as a requirement to get ahead, leading more people is a classic way to get ahead as a relatively junior manager. If a manager's peers are all more senior but are leading similar sized or smaller orgs, it will seem natural that the junior manager should get promoted to the same level. Building orgs is a classic way for managers to get ahead.
Actually delivering value is a huge advantage too, but as far as I've seen is not prerequisite for getting ahead.
> Actually delivering value is a huge advantage too, but as far as I've seen is not prerequisite for getting ahead.
Appearing to deliver value is strongly advantageous. Delivering value helps you do that. Other methods:
- Good salesmanship (e.g. making easy things look difficult)
- Stealing credit (can be personal, or e.g. for market conditions)
- Lying (e.g. cooking books)
Delivering value is probably the easiest way to appear to deliver value at the IC level. The higher one moves in the corporate ladder, the more important the others become.
If you believe they are neither vices as well, then I can see your point — they just are.
Perhaps this is the point you are making: if one is data-driven and engineering-oriented without regard to the consequences, that could be seen as a vice.
Despite the amazing engineering/science, it is hard to come to peace with von Braun or Teller (just to pick two examples).
Yes exactly. But until this is clear and accepted, these terms are apt to be misdirections—for example here, in a discussion trying to understand how things went so wrong at a leading tech company.
Hopefully we can get to the point of putting purpose (I think “mission” is a bit too hackneyed and neat to work here) first in discussions of culture, whether corporate or engineering culture.
I think most of us want to view "engineering culture" as just problem solving, amoral. If these companies we worked for didn't have such an outsized effect on the greater world we live in that would probably be a reasonable way to view engineering culture.
Some years ago there were some serious Warnings issued from Google: Make your website fast and small! Popups will be PUNISHED! as will mobile-unfriendliness!
But in hindsight, this was a ploy from them to try and upsell AMP, which was in a lot of cases heavier than non-AMP websites, and it was a means to keep people inside of their ecosystem - and inside of their advertising network, because not all websites Google linked to had Google adverts on it.
They've given up on AMP. They've given up on improving search results quality. Dare I say they've given up on improving the web? I mean they're still putting some money into Chrome and HTTP/3 and the like.
> I mean they're still putting some money into Chrome and HTTP/3 and the like
Looking at the misnamed "Privacy Sandbox", Chrome is not just there to improve the Web (and therefore google usage), but also to control it (and steer advertisers to Google even if it undermines the Web as a whole).
I use Safari for everything except front-end web development on MacOS now. I started because it's more energy efficient but I got quite used to the UX and now I'm glad I switched.
I'm hopeful there are a lot of Mac users like me out there.
I agree. What the search space needs is a different business model. I have no idea what it could be. I'm quite sure a subscription model would fail. Micropayments have not taken off. How can we fund meaningful search divorced from an ad business?
Why would a .edu help you learn to fix a tire? That seems like a YouTube question, not something a school would teach, and if they did, not something that would go online.
The article we're discussing is about limiting your search results to ".edu" domains to remove bloat, and gives an example of an https://www2.latech.edu/~bmagee/303win97/Group3/2245.html which does have this information on changing a tire.
Too bad .edu seems to be restricted to US-based schools. There's plenty of great stuff hosted by universities in other countries but none of them are on the .edu TLD.
There are other national domain schemes with educational subdomains. 'ac.uk' and 'edu.au' being two of which I'm aware.
A listing of top global educational institutions would only have a few thousand entries.
It's not without some irony that I note that the early, and much belated, Internet was largely edu domains, along with a handful of tech firms, government agencies, and military entities.
I've tried the same and don't find the results super similar. Both search engines seem to think that you're trying to repair the old tire. However the second Google result is goodyear.com which, while trying to sell you tires, does explain how to use the spare:
On Kagi you have to go until the 10th result to get something from Bridgestone, which explains the same thing while being less aggressive about selling tires:
Just 1% of Google's market might be enough to sustain a non-bloated, VC-free company and make a good product.
The Bloomberg Terminal is extremely expensive and yet companies pay for it because it makes their traders' jobs so much more efficient. I don't see why companies wouldn't do the same for a search engine that filters out all the bullshit so their employees don't have to waste their time wading through it.
You are right. And the actual number is lot smaller. We just need 0.001% of Google users to be sustainable with the current team size. (or about 50,000 users out of 4.3B Google users). And if you find 50k users, you will find 100k. First 5k is tricky!
I've been using Kagi for a month or two now. Love it and very excited about your work. Unlike alternatives such as DuckDuckGo, it has easily and painlessly replaced Google Search for me. Can't wait to see more from yall
I'm a Kagi private beta tester and love it. Will become a paying customer.
One question: I do remember a page that told me how much I searched and what that would mean once Kagi becomes a paid service. I can't find that info anymore. In light of the mail us Kagi users got today, does that mean that it's going to be a flat rate for starters?
Yes that is what we intend to try ($10/mo flat fee for unlimited). Most users told us they are more comfortable with this model vs pay-per-use which is why we removed the "Consumption" page as is not relevant in this model.
We do not yet know if the flat fee will be viable for the company and we hope to learn that very soon.
Another user here who see's huge value in just renaming the consumption page to a history page. Would be VERY handy! Especially if I could restrict a search to my own search history.
Google has 4.3B users, so 1% of that is 43M. With a $5 subscription, Kagi would have $215M in monthly revenue, enough to pay for 14,000 developer salaries.
A more reasonable target might be 500k users at, let's say, $10/mo.
Categorized search results are also data. Bloomberg is no different to Google in this regard, it doesn't generate the data, it just fetches and aggregates market data from various sources which anyone could technically do with enough effort and yet people pay for it.
At this point I’d pay for it. Every search I do these days is a fight with Google to find pearls buried in the overwhelming muck. I’d sincerely like to see how search curation would change if the intent was to keep me as a paying customer, not to show me more ads.
Every month I see my user experience decline with Google, in so much a paid search solution would be a blessing.
Imagine a search engine that truly is private, containerized search and does not need to sell you anything. It only wants to serve you true relevance and accuracy. That engine also skips over all monetized sites that serve more than 5-10% ads. Prefers cookieless sites. That search engine would be bliss.
Working on much of this at Neeva. We have a free tier, no ads cluttering the results. Just focused on the user experience. We also offer a premium tier for those who are keen to support us.
We have been working on exactly that at Neeva and the response has been very positive in terms of growth and adoption. Neeva offers a freemium model (free basic and premium for $5/Month which includes paid versions of VPN and a password manager), you can connect third party apps like dropbox and email putting all your personal docs in one search along with websearch, both versions are ad-free and private, it's available to anyone in U.S. and we are building our own independent stack to make the search experience unique and better.
Give Neeva a try (disclosure I work with them). We offer a freemium model (free basic and premium version for $5/month that includes paid versions of VPN and password manager), it's ad-free and private, it's available to anyone in U.S. and we are building our own independent stack to make the search experience unique and better.
Just a user of Neeva here but a happy one. Also very excited to hear about the work toward an independent stack! Enjoy the spaces and immersive search UIs for various types of searches. Neeva make search fun again. ;)
I am using their beta search service. It does seem nice enough. I am not sure yet how it compares with DDG, but I am probably going to sign up as a paying customer.
> We plan to fund Andi through a freemium business model, with free anonymous search for everyone for ever, and paid plans for professionals and businesses, including APIs and using Andi for corporate information, with supplemental revenue from anonymous referral link attribution. We're focused on building the product right now, and will figure out the details once we're further along.
> We are committed to staying 100% advertising free. Commercial considerations or partnerships will have zero impact on our search results or recommendations. Our recommendations are always made based on the best results we can find for our customers, and are never subject to commercial influence.
> By sharing search revenue with content producers and media companies that join our network, our mission is to provide funding to great content and reduce media's reliance on invasive ad-tech. Contact us at info@andisearch.com to get involved!
There are only 2 employees[1] of the company so there is a low amount of revenue they need to be self-sufficient
Why does web search need to have a business model at all? Why can't we just provide the Library of Congress funding to run something like it? Google got its start as a library project, after all, and it's the natural place for such a service.
Why stop there? Social media suffers from the same conflict of interest as search. Heck, even mainstream media online has this problem: the public's confidence in journalism has plummeted due to the rise of clickbait, fake news, and sensationalism.
What I really want is a decentralized, non-commercial, open source web. As part of this I would like to see search and communications moved to the client, rather than the cloud. This seems like it would take a huge effort to build, however, and even more monumental effort to bring people into the network. So it feels like a pipe dream at this point. I do think there are a lot of other people out there craving for something similar, based on nostalgia for the old days (90's and earlier).
I've been sort of thinking there really ought to be a sort of open source manhattan project to disrupt the search ecosystem and offer real competition to big tech.
There's a lot more to search than just search in the traditional sense, and I think multiple cooperating services offering small complimentary functionality slices could probably offer reasonably similar capabilities (save for like image search, maps and translation) with a significantly smaller hardware footprint.
My own search engine is fairly janky, but a lot of its problems are understood and could probably if not be fixed, at least mitigated. That's one functionality slice, searching documents. You could have a slice for forums, one for reviews, one for high value sites like stackoverflow and wikipedia, one for facts, one for popular links in social media and news, one for source code, one for mailing lists, and so on.
As a whole, I think you might actually be able to put together an actually useful information gateway this way. Would be a lot of work, but I think the amount of work probably exceeds its difficulty.
> based on nostalgia for the old days (90's and earlier).
Indeed. The internet in general and the beginnings of the web were a very different place back then. I remember about 1996 or so seeing a URL on a Pepsi can and thinking, "oh crap, it's over."
> How can we fund meaningful search divorced from an ad business?
The only way you have not listed is gov't funding. And i dont think i would trust a gov't run search engine either. May be a well meaning librarian institution could make it work.
> I'm quite sure a subscription model would fail.
this hasn't really been tried yet - it could work, who knows?
Hi there, you should give Neeva a try. Disclosure, I work for them, but we were the first to introduce a subscription, interest/growth has been really strong so far. Neeva offers a freemium model (free basic and premium for $5/Month which includes paid versions of VPN and a password manager), you can connect third party apps like dropbox and email putting all your personal docs in one search along with websearch, both versions are ad-free and private, it's available to anyone in U.S. and we are building our own independent stack to make the search experience unique and better.
There are two versions 1) Free basic which is free -- full search, ad free, private, and personal connectors. 2) Premium for $5/Month includes all the free basic + additional connectors and paid versions of VPN and password manager.
Do we really need it to be a business? Does it need to make money? Torrent communities and DHT don't make any money and yet they're pretty good. Would it be possible to build a decentralized solution around finding things on the internet? Something with trustworthiness ratings and user controlled blacklists perhaps?
It doesn't matter what new model you come up with. As long as capitalism is around and pushing for ever increasing profits this kind of behavior will always be on the menu.
There are many ways laws could be written to would effectively ban the current online advertising business model. One could be to require written permission each and every time a company wants to transfer user data of any kind to anyone else. Google could try keeping all Google site collected data internally but all analytics data collected from non-Google sites would be forbidden without written permission for each and every instance.
Every site would have to run their own data analysis and sell their own ads. Advertisers would be unlikely to take Google or anyone else's word, so audits would be necessary but difficult to conduct. It wouldn't completely end online advertising but would turn it back into something closer to the old traditional model where those selling ad space had a handful of semi-generic personality models for their readers/users and advertisers would select one that best matches the profile of their target customer. It destroys the economics of centralized data collection.
There are many other laws that could accomplish a similar end but through different means. The biggest barrier would be getting politicians to not care about the demands of huge tech companies that dump endless amounts of money into lobbying and campaign funding.
There are only two choices. Either Google are no longer interested in finding the best answers to your queries. Or as you point out: They don't care, that is no longer what they do.
I cannot imagine it being easy, but I find it hard to believe that Google aren't able to weed out the worst offenders. For a large number of queries the results are all SEO spam. Try searching for anything related to health and get a useful result. It's no longer possible.
> Try searching for anything related to health and get a useful result. It's no longer possible.
Medicine researching on the internets always was among the hardest topics. And anyone who really understands Medicine has to write on any advice not to consider it as a real medical advice... I agree that google has degraded down to unusable in comparison to non-global searchable websites (like SO) but Medicine is very bad example to your point.
At the same time, it would be the easiest to solve with something like a "white list" (I know, not a good term to use these days). With one stroke you shut out all the bad actors.
In fact, I wonder how far a search engine could go with a curated list of "good actors" you exclusively serve up results from. Add a "search everywhere" link for people that prefer SEO hell.
When Google started, the web was honest and innocent. People linked to the pages they liked, and Google could crowd source those links and raise up the most popular pages.
Now most of the web is lies meant to confuse search engines. Figuring out what pages are good is no longer a feasible task for software.
> Figuring out what pages are good is no longer a feasible task for software.
Of course it is. It shouldn't be at all difficult to train a large natural language AI model on SEO spam pages. But nobody will do it unless they can figure out a business model that makes it pay.
Or a non-profit organization is formed to do search spidering for the benefit of the whole world, much like Wikipedia or PBS.
I don't doubt the narrative as you relay it, but I question how hard this can be for one of the largest corporations with one of the largest engineering pools in the world.
Google has SO much money, they could probably afford to curate the web and white-list websites one at a time and still make money. Personally, I think Google should get out of the search-indexing market entirely and rely on website submissions only.
At this point, it's no different than an appstore in terms of volume. Ironically that's where we're heading with most websites being "3.0" and just SPAs and now PWAs even.
OT, but Amazon is in the same boat. Their problems with counterfeit merchandise and fly-by-night Chinese junk manufacturers have been going on so long that it's become clear they don't want to fix the problem because it's now a major revenue source for them. Apparently someone at Amazon decided the Aliexpress marketplace was more lucrative than selling good products.
They don't care about fixing it because people still flock to them.
And who can blame us? Other sites still suck worse. I can't believe how bad walmart.com still is - the user experience is night and day. Walmart - the retailer embracing computerized tracking of spending by credit card number and other advanced data analytics a full decade before Amazon was even a glimmer in Bezo's eye took forever to realize the power of using their stores as local fulfillment centers? Provides a 3rd party marketplace that's less reliable than craigslist? Who the hell is in charge of online operations at these big companies and how do they still have jobs?
Then again just look at Sears - the company that literally invented mail order *in the 1800's* and how they completely missed the Internet - the ultimate mail order environment.
I'm using Google less and less in favor of Youtube. It's not always convenient to get answers in video form, but I'm finding it's better for more involved answers. Also, google is just littered with Answers/Quora/StackOverflow/etc dodgy clones that make searching questions a nightmare.
While I overall agree, I ironically find the search engine on YouTube to be somewhat limiting when I'm looking for specific and/or fresh things...
For example, I wanted to watch a bunch of videos of Mark Zuckerberg speaking from before 2010 and the YouTube search engine only showed me a limited amount of things I had already watched— it even had unrelated suggestions below those results based on my viewing habits.
So I went to Google and searched for videos and interviews of Zuck and found a bunch which were not easily surfaced on YouTube itself.
Same thing with Ina Garten/Barefoot Contessa cooking videos.
Now that I've found some fresh videos, the homepage recommendation algorithm has gotten better at surfacing new content related to these topics.
This makes no sense. You make it sound like Google has no financial incentive to fix this problem, which is patently false. Google isn't selling the organic search results, which is what this article talks about.
The problem is that Google is engaged in a war with SEO optimizers, and losing (or at least, barely breaking even). No business model will fix that. It's a hard problem that any successful search engine will struggle with.
But does Google care about the organic search result quality?
For many search terms, the entire above-the-fold and sometimes multiple scroll heights are filled with:
- Ads
- Info cards
- People Also Ask
- Map/Business info
- Videos
- Related Products
- More Ads
The organic search results start after that and on mobile they may as well not exist.
If everyone is clicking the directly or indirectly sponsored/algorithmic content then what incentive does Google have to give you good, straightforward links to relevant webpages?
I am not a Googler, but I know some Googlers (including at least one that worked on search) and... Google doesn't seem to work like that? The search team doesn't make take direction from the adsense team.
I think they are genuinely struggling in an adversarial environment. Imagine every hacker in the world specifically focused on your system and the payoff is millions or even billions of dollars.
I think that’s totally reasonable as far as the organic results go.
But clearly there are also a lot of motivations to de-emphasize the organic results on the search page in favor of ads, “algorithmic answers” that give you an answer without clicking away from Google, and redirects to Maps, Youtube, and Shopping which are revenue-generators for Google.
Maybe that also comes with a budget de-emphasis for fighting the war in the organic results vs surfacing relevant content from these other sources.
"Google" is 100+k people. "Google" doesn't respond to incentives; individuals do.
A typical employee at any large company is incentivized to:
- Maximize their salary
- Maximize their future earnings potential (e.g. CV)
- Have fun
- Have work-life balance
... and so on. That goes all the way up to the CEO. Pretending a corporation thinks and responds to incentives is folly. Running a corporation like Google is like herding cats. If everyone marched in the same direction, Google would cut through SEO optimizers like a chainsaw through butter. There is no wayh to do that with 100+k people.
Google fundamentally has no financial incentive to fix any problem, since organizations aren't intelligent, sentient beings which act on incentives. They have dynamics from the interactions of 100k+ individual agents.
There is no magical fairy which makes large organizations act in their own best interests or even try to survive. The only reason they do survive is that's as true of their competitors as themselves.
Did you write that, or did your 85 billion (approx.) neurons?
Corporations are intelligent, sentient entities which act on incentives. They don't have a conscious sense of self, but then neither do many natural organisms. That doesn't keep them from responding to their environment as if they're aware of it, and acting in incentive-driven ways.
Corporations seeking to maximise profits are exactly like humans attempting to maximise income. The problem is that corporations - like humans - have the wrong incentives, further diminished by limited predictive ability and poor heuristics.
So maladaptive behaviours emerge and stick. Both humans and corporations find a nice local minimum and don't move from it.
More intelligent behaviour would be able to predict and avoid existential threats without having to experience them first. But it's a question of predictive ability, not a binary absence of all intelligence.
So it's not that Google can't do better because it has too many employees. It can't do better because its cultural heuristics - which include it internal culture and the external culture it operates in - don't allow it to.
A megacorporation doesn't act like my 85 billion neurons. A corporation acts like a forest of trees, or at best, an anthill. Corporations (and simple organisms) respond to their environment, but not necessarily in incentive-driven ways. Often, corporations move opposite incentives not because of some local minimum or whatever, but because it, for example, personally benefits the CEO, a few board members, or a group of employees.
I can set up "incentives" for an anthill, in the form of a mixture of sugar and borax. Emergent behavior takes over.
The behavior of corporations, at the scale of Google, Facebook, or similar, is almost entirely emergent dynamics. Same thing for countries, for that matter.
Not TV. It's almost exactly like Altavista, Lycos, Yahoo, Excite and other "search portals" of yore. The thing they swore they would never become, yet here we are now.
This is why I switched to Neeva, I want my search engine to make products for me that help me find the best results for what I am looking for. When an advertiser is the customer instead of the end user the entire product road map is just trying to take you back to that search results page as many times as possible.
Neeva also lets me index my personal documents and includes those in search results as well. They are an early startup but I am excited about what they can build when they are not beholden to advertisers.
Another Neeva user here... yeah, the personal document indexing was far cooler than I expected. I held off doing it for it a while but glad I gave it go. Just the GitHub indexing alone is cool. Can do some fun hacks with that too. I think they need to surface these personal results even better in the web results than they do (and I hear that is in the works?? hope it is).
Corollary: Web page creators are no longer interested in answering your query; they're interested in getting you to 1) view their page for ads 2) click on their affiliate links 3) buy the product they're selling.
Google has an impossible task in trying to answer a query like "Portable air mattress reviews", because 99.99999% of the results are SEO-optimized crap
We don't know if the task is impossible because Google has no incentive to do it at all, so we can't assume they are really doing their best. They created this situation and they profit from it just the way it is.
Build a better mouse trap etc, if the task was possible, DDG / Bing / Whoever could absolutely dominate the search market by providing a ”no crap results” option
They would only bother to try if they could monetize it, and the only model for that which won't create the same incentives is paid user subscriptions. The market for that may just be way to small for the big guys to consider.
The top two organic results for that precise query are pretty informative articles from Wirecutter and Good Housekeeping. Unless I'm misunderstanding and you are expecting something better than this: https://www.goodhousekeeping.com/home-products/a25473658/bes...
I paid for Which? which is the UK equivalent. Would've subscribed again when I needed it (and there's been a few times since) but their dark pattern upon unsubscription put me off from going anywhere near them again.
I also spend 50£/month on Patreon to support the creators I watch regularly.
Got me thinking - Google has became what it is - this colorful commercial ooze - by parasiting on the pre-Google web. Is there a search engine that searches sites without any Google code in them?
They also have no focus, as you and a reply essentially say.
For example, all the stupid aliasing of search terms, and dropping of search terms.
This is helpful if you want to optimize for "sorta close, sell things!" such as "maytag A4453 fix timer", but you want advertisers to latch on to maytag=washing machine, and ignore the rest.
And it also helps when websites with google ads on them them get higher SEO, because suggesting that site might get a click, and profit!
EG, suggest a blog/spam site, because google ads are on it.
And it also helps with google voice search! Did the speaker mean here/hear? Did they mean Danny/Daniel/Danson when they said Dan?
Google has too much genericization in its search, and it is for profit naturally, but I think it is lazy profit, and it is becoming worse yearly.
While obviously not universally true, this bothers me about a large swath of current UX culture and it’s pushing me towards other types of design. Anyone who’s heard a UX conference keynote in the past ten years would think UX folks go into work every day making unbiased data, using it to identify users’ genuine needs, and championing those needs at every stage of product development. Sure, the suits call the shots, but does our software landscape indicate anybody really does that?
Many product folks just don’t seem honest, even with themselves, about what benefits users. Do you really think your users benefit from ”more relevant” advertisements? Because your users don’t. Are new services and content so beneficial to users that they’d forfeit their privacy to fund it? If so, why obfuscate your having made that choice for them? Why not make it opt-in? Does that dialog box popping up just at the right time really give that overstimulated and frustrated user a choice? Either answer those kinds of questions honestly or admit that you’re just finding the smoothest path to maximize revenue. The hypocrisy is infuriating.
I still use Google every day. I was really hoping this advice was for real. Like maybe how to filter your Web results by size since they started off with example of a 9 MB website maybe you could filter your queries down to websites that are 2 MB or less? Since I use pi.hole and the first results are ads my top links are always dead. It is time to switch - Is DDG the best alternative?
DDG has been my default search engine for probably ~7-8 years. In the early days it wasn't that unusual for me to still do some searches on Google for better results, but nowadays that's extremely rare (probably partially because DDG has gotten better and partially because Google has gotten worse). But I still hear people complain about DDG's search results quality, so I think it really depends & YMMV, but it's a pretty low-effort thing to try for a while, so maybe give it a shot?
My loosely held theory on the wide variance in people's judgement of DDG is that some people really like/have come to rely on how much Google personalizes results based on all the other data they have on you, even if they are in the "Google has gotten bad" camp. DDG doesn't do any of that: it's much more like going back 20 years where you have to put all the context you care about into a query, you're not searching "in a bubble". For me, that's a very positive feature, but others may find it frustrating and not even be able to articulate why they're not happy with the results. DDG does do a lot of the nice "enhancement" stuff Google pioneered long ago, like embedding the intro of the Wikipedia page in the sidebar for a query with a strong match, e.g. a person's name. So for me DDG is just the right level of "smart": it understands common queries well enough to immediately highlight useful details, but it doesn't try to be so smart that you as a user stop being able to understand what it's doing under the hood.
I think a subscription model could succeed for a niche group of people if the price was right, and the quality was good enough.
Also, I think Google could make a ton of money if they sold one ad per language at a time. As in, for the next 5 minutes the ad for all English speaking users will be for coca-cola, and then it will be for boscovs. Just like TV ads.
That is to say, a search engine could sell advertising without betraying it's users or compromising its own quality. It is possible, but would need constant vigilance against the temptations of greed.
$5 / mo for a premium Neeva subscription or just use the free tier to experience ad-free search focused on the user. I left Google to join Neeva so I could be part of building a better search experience for people. Check us out. Would love feedback.
Adam Savage went on a rant about 2 weeks ago about this very thing. He was trying to find a large format laser printer and the typical "best laser printers of 2022" blogspam were all filled with inkjet printers.
Only 13% of adults in the US have an advanced degree. Coddled engineers live in an emotional bubble ignoring the very real world that does not feel an obligation to a minority of coddled engineers.
When a brain is trained to follow along from childhood, I don’t really see how it’s an option to avoid it.
Also the 13% are not all engineers. Many are historians or similar who babble nonsense. We built many big buildings and bridges before web devs came along.
Centralized ledgers used to dictate agency and what effort we must put in (computed by elites with a currency monopoly; which in the US is legally a speech monopoly thanks to SCOTUS) are not really the same thing as a bridge.
There’s a big difference between things we need to engineer and TV 2.0. Whole lot of IT is rent seeking.
> Oh, and also, Miller is hiring if you’re interested in joining the team. I want to change my tire! Get off my back, and stop trying to make money off me for one second while I solve this tire crisis.
So, you want someone to help you and you are pissed when they don't want to do it for free?
The problem is that there are people who have published free, no-bullshit guides on how to change a tire for the sake of spreading knowledge, and these results should be prioritized over businesses who have a profit motive.
I came to find out how to change a tire, not to get a job in dealership. If the page said "we have 5 different guides on how to change a tire with different level of details, so you can choose the one you preffer. Just pay us 5$ to get download link" without forcing an account, it would be much better.
No. We are not looking for hire someone to do a job. We just want to be able to search for content online without middlemen interjecting themselves in the hopes of monetizing the interaction.
The millions of people that write and produce content without being tied to this Protestant mentality that something is only worth doing if it is profitable.
In fact, I'd wager that if we had a way to filter for content that is written without the goal of being monetized, we'd have a better web and us consumers would be more inclined to reward and give back to the creators generously.
I thought this was an interesting topic, so I read a bit on Protestantism and Protestant work ethic and what makes things worth doing. [1, 2] The interpretation that something is only worth doing if profitable seems somewhat uncharitable but does seem to be possibly rooted in truth.
The Kagi search engine which is close to entering public beta has lenses which tailor the search experience. There is a non-commercial lens. The lenses are basically on or off all the time and affect the overall search results.
Where did you find this feature? As a beta user, I have a hard time finding documentation for the features in Kagi. Ironically, searching about Kagi in Kagi is not helpful.
The irony is that the garage is doing a fantastic job up until that point.
If someone is so clueless about changing a tyre that the best place they can start is google, then organically taking them to the garage's page is a win. Maybe they'll find the information useful, maybe they'll decide they're out of their depth and call the number on the screen.
For them to show up as the first organic result is a huge win. All they have to do with that win is not screw it up. And 9 meg of trackers, a page full of popups, and pointless CTAs (do you want to hire someone who can't change a tyre?) is screwing up that easy win.
You don't have to be scummy to turn search results into profit. That garage came so, so close of being a great example, before they threw it away and made a bad example.
Ironically, it's the ones who were doing it for free who tended to excel at providing information, and the ones who're busy monetising their presence who've revealed the Sadim Touch, turning all that's gold to shit.
So I will reiterate my little theory, this is not entirely Google's fault, regulations, especially KYC are responsible for this. KYC is incompatible with having a free zone. This is actually the aim of KYC, to remove free zones.
If you dont get it, let's take an example. The flat tire
mustard-yellow webpage. This page is not KYC'd. Are they laundering money through their website? Are they rallying people for a bloody crusade? A search engine cant know that (basically) but it knows a website that display ads is KYC'd so google can display them more easily.
For sure Google is pressed to help make a regulated web and stop the free zone. Knowing it's in their interest anyway (ads), they embrace it. I dont get why the internet dont get that already: it's all KYC :)
Frankly, if it’s not reliable as a tool, it has no real value and I can’t believe it has come to this. Imagine if every time you ran `ls` in a shell with a glob pattern, it just decided to sort of “add in” a few other files loosely based on your query (or heck, files that aren’t based on it at all)? Oh, now imagine if `rm` did that.
Sadly this happens with lots of search tools now. Why the heck is the default state on a new Mac to funnel what you type to everything, e.g. I searched for “Chrome” and hit Return and the FIRST thing it did was throw me into the App Store and call up some not-even-a-web-browser scam app with Chrome in its name, instead of selecting the Chrome already installed on my computer and opening it? More and more it seems that you have to turn off all kinds of poor defaults to put tools into a useful state, or there is simply no way to get them there at all.