The filter bubble thing is overstated. There are times when you want a filter bubble, when you found a piece of information and want to dig deeper on related subjects, a "conversational" interface that has at least short term memory offers big benefits. Do you really want to ask questions to an entity which acts like it has anterograde amnesia? Can you imagine Captain Picard having to continue to remind the computer of the enterprise about context?
Google's 'long term memory' offers lots of benefits too, and 90% of the time, the filter bubble is the correct answer. Like when I search for a particular business on the desktop, and then later I search on my mobile device in Google Maps, and the very first auto-suggest after typing 1-2 characters it the business I searched for yesterday. This is awesome given how irritating it is to enter stuff on mobile devices, especially when in motion, the fewer characters typed the better.
Most of the privacy issues can be solved just by opening up an incognito window. You can choose with 1 stroke whether you want a bubble or not.
The privacy issue I have is that governments can come in and request all data companies have about me, and there is nothing companies can do about this. There is no privacy promise they can make to me that can withstand a government request.
The only defence is to ensure the data does not exist.
In practice, this means using an incognito window almost all the time. Simply the fact that I'm on the other end of an IP logged in to a service is information I'd rather not share. And that's just too onerous, so instead I ensure my main browser isn't logged in to any services, amongst other defensive measures.
Ahh, but if the NSA got a secret court order commanding DDG to start collecting data quietly, would users know? Do you think DDG would go the way of Lavabit and shut down voluntarily?
That's a really good question. My impression is that Gabriel Weinberg and team are building something on the side of privacy and users. I don't know with any certainty what Mr. Weinberg would do in such a face-off, but at this moment in time, I personally feel more comfortable with DDG than Google. My feelings are simply mine and based upon what I've observed so far with DDG and Google. Google just won't close shop - DDG might close shop. I trust DDG more.
And a domain from Guernsey, who are unlikely to mess around with things. It always baffles me when I see startups using things like Libyan (.ly) and even Syrian (.sy) domains to make some cutesy word.
I really think we need a search command to break out of the bubble on google, you are right 90% of the time its the right choice but the other 10 your screwed, some trial and error with removing the offending terms but sometimes it just doesnt.
And startpage.com is located in the Netherlands and not in the USA like DuckDuckGo.
I can not trust USA services anymore.
DuckDuckGo is located in the Amazon Cloud and Amazon is the next target of the NSA.
startpage.com includes ads from Google and forwards you upon a result click to google.com/aclk for tracking purposes.
Please do not spread this site saying it does not submit data to Google. It does guaranteed for the ads, and knowing the search engine scene a bit I would also bet that they submit your IP and useragent to Google for every search query.
It seems recently everybody accepts those claims about privacy protecting search engines without checking anything.
As a sibling comment says, I didn't actually make any privacy statements.
However, startpage's help pages claim not to pass on IP addresses, useragents, cookies, or other identifying information. This claimed-independent report says so too: https://www.european-privacy-seal.eu/awarded-seals/de-110022... — verifying such statements is hard; believe what you will.
I agree, it seems clicking on the Google ads (rather than search results) does put a request via google.com/aclk. Privacy conscious users could block ads to avoid such mistakes. Of course startpage must send some data to google: the search terms are needed to get the search results themselves (as well as for selecting the ads). That could be a privacy leak, for example if simultaneously searching for a name and a sensitive topic. It doesn't seem nearly as bad as tying the search terms to a unique identifier across searches however.
Google can't not track which results are clicked for which queries. It'd be impossible to build a decent algorithm without measuring how well it works for people.
Exactly, the bubble it's not bad in all use cases. Nevertheless it would be really great if you actually have a real control over it: switching from one "bubble" to another for different contexts or disabling it altogether.
I've switched from Google and Chrome to DDG and Firefox. Very happy with the switch. I've started recommending to my non-techie relatives to switch as well.
We've seen this sort of thing before. I've gone from no search engines, to Webcrawler, to Lycos, to AltaVista, to Google, and now DuckDuckGo. No biggie.
I've made the exact same switch. I'm loving the combination of FF & DDG. I've also set DDG as the default search engine in Chrome and use it on iOS from the app and Mercury browser. DuckDuckGo deserves all the great publicity it's receiving.
DDG is my default search engine, I do find myself needing to occasionally throw in !g if I'm not finding what I'm looking for. Results are generally decent though, and the trend seems to be that they are getting better and better (though I have not objectively measured that in any way). The other ! shortcuts are worth reviewing, as they can dramatically improve the results if you're doing a focused search.
So true. One of the nice things about the !bang syntax is that it's pretty intuitive. Although one can accomplish something similar in the browser via smart keywords, the DDG way lets anyone jump right on to any machine and focus a search on a particular site. By picking the first letter or a two letter combination, e.g. !hn=hackernews, I don't have to look up the exact !bang for a site.
They've changed the behaviour of that in the address bar now. Perhaps I have done something wrong, but it appears that what you type in the address bar searches by the search bar's choice. So if I have Wikipedia selected, it searches on Wikipedia from the address bar. This behaviour was initially very confusing.
At first I thought I had done something strange to my Firefox (on Linux), but then I noticed both on my workstation (Windows) at work, and on my laptop (Linux and Windows). I hope someone else can confirm this.
Same here, FF and DDG. Might have to use Bing or Google a few times but Google isn't /wasn't perfect on every search either.
I now do get referrals from DDG on a few on my websites so anecdotally and from a very small sample I see the increase. Google has become very commercialized and too tied up with the FED big boys now.
Don't get me wrong, I'm all for keeping my private stuff - well, private. But, really, will this matter? I mean, how many people used Gnome's browser in the first place? and how many people, after trying DDG won't simply switch back to Google?
Had Gnome's browser been no.1 or no.2 in the browser arena, the gesture may have had some impact (albeit just symbollicaly). But right now... do they think Google would even care/notice?
The gesture is noteworthy not because of market share but because of who the user base is. It's exciting to see folks in the technology field (often the same ones who were educating others about Google in the early days) promoting DDG now. A browser that ships with DDG as its default, adds immediate cred to its brand image, IMO.
Sometimes we need to be the ones willing to sacrifice a little bit of "convenience" to stand up for products and services that are at least attempting to travel a better road for users. Maybe I'm being a bit naive, but I believe that lines are being drawn and those products on the side of users, have a renewed opportunity to shine.
DDG is right there - in a great position. I get the results I want from DuckDuckGo without bloat and "personalized" crud. If I really want to massage the DDG query results, I can add !sp and get a "Google" perspective with privacy.
The !bang syntax just makes DDG more powerful.
All in all, I think that Gnome's decision is more than just a gesture. It's the beginning of a shift and the big players can either read the message that's being sent or ignore it to their data loss.
I think they are aware of what you're suggesting they wrote:
"Then again, knowing the humble size of our userbase, we should let neither our expectations nor our imagination run wild on this front."
Still it is the first browser which chooses DDG as its default search engine that I am aware of. This is quite a thing for such a small company like DDG. Now others can follow (or not) when they see that it worked (or not) for Gnome. And even if on Linux more people are aware of the fact that they can change their search engine in the browser, still there is a fair amount who just use what is there. It is therefore browsers nowadays are developed, do sell the default search engine position.
I remember the days when IE had 98% of the market share.
But then you consider that in the mid 90s, the raw number of devices regularly running Netscape Navigator was probably on par with todays usage statistics of Opera.
I'd bet its even less than that. I remember watching the comets fall on that stupid icon and waiting... and waiting... only the truest of nerds tolerated that nonsense.
I've been using DuckDuckGo for months now and over the past few weeks it's results have gotten considerably better. I wonder if this is because a massive amount of new users are consistently using it overall improving it's results?
I'd say it's improving for different categories of searches. Because for a long time users were largely technical, it's search for tech stuff has been pretty good for a long time (in general, there are a few sub-categories in that that have lagged[1]). Where, searching for an author or book might have terrible results. So, when these people say it has improved, they mean it has improved for the kind of searches that that person does. As more people use it, and suggest improvements[2], it will approach being good for all kinds of searches.
[1] It still isn't very good at searching for specific error messages. It definitely used to show obscure projects on GitHub way too often. You'd do a search for 'getty' and some 3-commit project with a name containing getty would be the first hit. Also, I think it's actually gotten worse at finding Rails API stuff.
[2] It's been a while since I tried, but if you tweet at them with a search term with a poor result, they will try to figure out how to tweak the algorithm to make it better. It's really cool to see them act so directly on your suggestions.
I wonder if there's an element of users learning how to optimize their query for Duck Duck Go. I've been using it as a default for some months now, but I often fall back to Google for software error message strings and other things.
It just doesn't give as accurate results as Google in some searches, I assume due to the rafts of information collected about users - a trade off I will make happily.
If the search engine isn't giving me results I expect isn't that the same if not worse than it being inaccurate? If I type cucumber in google, it knows since I'm always search technical information I probably mean Cucumber BDD and it gives it to me as the second result right after the wikipedia article about the gourd, that's what I expect. DDG, however, having no clue who I am, gives me 8 results about the gourd before finally, below the fold mind you, giving me Cucumber BDD.
Ah but Google does return me CakePHP if I search cake (and CoffeeScript for some reason, coffee and cake go together I guess I dunno). I don't agree with your example about french braids, at some point you are getting a little too generic.
It's definitely uncomfortable, I don't know if it's better or worse but for me I couldn't adjust and found myself going through !g until I just switched back.
You're fooling yourself. Google and Startpage return Cucumber BDD as a second result whether you're logged in or not. And in foreign countries such as France, it's even the first result.
Compare to Bing: in France, "cucumber" returns a porn(!) site as the first result (note: I don't visit porn sites or use bing).
DuckDuckGo and Bing simply aren't good enough, but it has nothing to do with whether they respect your privacy.
Working for a company that takes privacy very seriously, I can say that it can be a PITA. Worth doing, but still a PITA. We had to resort to some clever trickery just to count monthly active users without any sort of unique identifier. I bet DDG faces similar challenges.
Google uses a cookie to tie visits to a certain user for analytics.
The other solution I've come up with is to use a hash of IP + other identifying info, so not recording the IP or anytihng to tie it to a person, but still recording a relatively unique ID. What sort of techniques are you using?
Well, we have daily update requests (it's not a web app), so we knew the number of daily users already. But we really wanted to know the monthly users, gives a way better idea of the actual user numbers. What we ended up doing is to transmit the timestamp of the last update check with each update check, that way we can see which requests are the first over an arbitrary period.
Granted, the situation is probably not that similar to what DDG faces, but it was just one issue that came up. What we use for web analytics is based on awstats, which operates on access logs with anonymised IPs. Not as good as Google Analytics, but as useful as it gets without JS/tracking, I guess.
That's assuming that search preferences are homogeneous, that everyone in the world uses DDG, or that the people currently providing click data are a representative sample of the world. The first and last things might be true for a few terms, but even then, the order of the search results is a very tricky thing (e.g. NYTimes or WSJ editorial page first? the difference in traffic between 1st and even 2nd or 3rd on the search results page is huge).
The only real benefit is that you and your neighbor are in the same filter bubble.
There are upper limits to global optimization. We all don't want the same results for a given query; to get beyond some particular bound, you can't treat all users the same.
Google can still track you, cookies are just one example. So DDG uses only the data which can not related to a single user, essentially "users who searched X clicked most often on the second link" while Google builds a profile, that is "the user with cookie value Y usually searches for computers."
I'm not sure if they are still as responsive on Twitter as they used to be, but if you tweeted at them with a specific search term that gave poor results, they would act directly on that. It was cool to see your own input make a difference.
In addition to aggregate usage stats, there's also the indirect (and slower) route: more users -> more ad impressions -> higher revenue -> more employees or other resources available to improve the service.
Very welcome step. User privacy should be of utmost importance for all GNU/Linux distros. Unfortunately, users of Ubuntu are subjected to privacy intruding malware (that sends their queries to Amazon). Its about time the distros followed GNOME's step and override Firefox's default search engine to DDG as well.
"User privacy should be of utmost importance for all GNU/Linux distros."
Why? It's a feature which takes a large amount of work to implement, has a series of poor tradeoffs, won't actually increase the size of the user base much, is incredibly hard to do well and is very hard to market to users. Not to mention the fact that a large number of GNU/Linux distros are aimed at markets where such user land improvements are largely immaterial (e.g. on servers).
The simple fact is that most users don't care about privacy in the same sense that DDG cares about privacy. If they did, then there would be a huge economic incentive for everyone to make these sorts of changes.
In light of the recent leaks, there is a rising consensus that privacy protection should not be underestimated. If it is, then it is prone to put to use by a third party without taking any permission from the end-user, which is unethical. This is all the more important in the open-source world because many people use open-source products because of their ethical and 'open' paradigms. Proprietary code by their very nature bind us to their whims, but open-source is all about freedom.
Even if your hypothesis is true, ie even if most users don't take privacy very seriously today, I bet it is pretty soon going to change.
I'm a new Ubuntu user who's not super savvy but am considerinng moving to another distro due to these malware issues. Which distro would you recommend I move to? Mint?
The easiest thing would be to just turn off the internet search option from the dash, or maybe uninstall the Amazon lens (but maybe there are other similar lenses...). Or change from Unity to another Desktop Environment (Gnome, Xfce, KDE, Cinnamon or whatever).
Unless you want to make a statement, then changing the distro might be the gesture you want.
Don't be fooled by the FUD. It's easy to turn off and not that big a deal, but people who are already biased against Ubuntu like to use this to try to scare folks away.
Yup, it's easy to remove that specific functionality. But let's think of the big picture. The introduction of this privacy-violating functionality signifies the direction of Ubuntu as a whole.
I highly, highly recommend Linux Mint (with Cinnamon) to those looking for an Ubuntu alternative. Cinnamon is a great desktop environment and Mint, in many ways, is Ubuntu without the privacy issues.
I'm not entirely sure how the upgrade process works in Mint. I think of it as a "transition" distro in many ways. Most of the people I've recommended it to moved on to Crunchbang or similar after a few months of use.
Depends, I guess. I'd check out Debian and maybe, yes, Mint.
The thing to realize is that you can use any kind of desktop environment on any of the main distros. With Ubuntu, moving from its default DE can sometimes result in certain hiccups. Other distros are more DE-modular; that's the feeling I get when working on a Debian desktop, at least.
Overall, Debian is a very user-friendly system (or rather, it exposes the same kinds of interfaces as Ubuntu does, once the leaky UI abstractions in Ubuntu do start leaking. ;)
I have heard of DDG intermittently, but never remember to try it out with any consistency. I generally have my default browser page set to about:blank, but I just updated it to DDG. Maybe that will prompt me to try it for a while.
But, here's the thing: As an avid Android user (phones and tablets), it feels a little "insufficient" to just switch over my search engine in the name of privacy. After all, I use the big G for nav, contacts, and other stuff. Of course, it is easy to use their sevices by default, and I don't know what a viable alternative stack woukd look like. I disabled Verizon's Navigator because its click agreement seemed at least as onerous as what I have seen from G.
So, even in trying to find alternatives, it does bring up the questions: to whom am I willing to provide my info, how much am I willing to provide, and in exchange for what?
It is relatively easy to run your own contacts/calendar server. Just buy a Raspberry Pi, set up DynDNS, and hit apt-get install owncloud. This is really the way to go here.
Also, there are plenty of email providers out there that are not Google (but it is often unclear whether they are any better privacy-wise. Also, arguably, no one is providing as slick a web interface for email as Google). I am not living in the US, so it makes sense for me to choose an email provider in my own country.
Mapping is more difficult. Tomtom etc. have great navigation apps. But if you just want to look at a map, GMaps is still far superior to most OSM based map apps. Skobbler has a few great ones.
Depends on where you live. In my city and its surroundings, OSM data is far better than GMaps. And on my Android I paid for both OsmAnd and Skobbler. These aren't as polished as Gmaps, but at least they work offline.
This is trading imagined harm for a real harm. You trade off a loss of privacy that has zero practical implications currently for almost everyone (a few terrorist suspects might have to worry) against lost time due to inferior search result quality.
I don't follow the logic. It seems like idealism for the sake of idealism to me.
> This is trading imagined harm for a real harm. You trade off a loss of privacy that has zero practical implications currently for almost everyone (a few terrorist suspects might have to worry) against lost time due to inferior search result quality.
So you argue that slightly inferior search results are "real harm" and mass surveillance is "imagined harm"? I'm afraid we don't agree on the definition of the word "harm".
> If you think everything that generates records the government can eventually access is "real harm," what exactly are you doing on HN?
I post my comments knowing full well that anyone can read them. I do not think my web browsing and searching behaviour is quite the same thing. I cannot decide what to share and what not.
To come up with a stupid analogy, it's like saying: "If you don't like being watched, why do you go to the supermarket?" It's not the same as being watched in my bedroom.
This might be workable for some people, I tried DDG and results aren't on the same level. DDG is getting better every day, until then my temporary solution is using "Google proxy" (anonymised Google search) like https://startpage.com/.
I think the NSA have proven if a particular data provider (in this case Google) doesn't agree or can't provide the info the NSA wants, the NSA have the capabilities to sniff the traffic on a multitude of levels. Privacy is an illusion, but I can attest to the fact DuckDuckGo is the more secure option.
DuckDuckGo is nothing but a bing front-end. I do not see an upside with this switch, so basically, you are making M$ more stronger on Web. I am not being snarky, I am concerned.Google is evil sure, but is M$ better?
KDE has a web browser too, Konqueror which uses KHTML and KJS which then Apple forked into Webkit to use in Safari. Google then got into the Webkit action with Chrome. Now Google is forking Webkit itself.
So yes, nothing good ever comes out of rarely used web browsers.
The Web browser is WebKit via WebKitGTK+ (the reference is a little oblique, but it's mentioned in the article, or you could do a basic search), so no need to fret over other people working on a browser you don't even use.
No, the package management system on just about every distribution running gnome is the best way. A web browser is a good way to get info on why the package manage is broken though.
DuckDuckGo is great. I always end up after a while using Google though as I find Google generally has better results and I enjoy the 'sort in the last...' functions for example that DDG lacks.
I have nothing against the entities involved but reading the comments here I get the impression that many simply ignored the "cooperation" bit i.e:
Cooperation: It's been some time now since we were first contacted by DuckDuckGo regarding the possibility to partner with them in order to share a percentage of the revenue that they make from the traffic originated on their search engine links ...
So DDG made a deal they could afford and got their partner to hold water for them and link to one of their PR websites, they've been ramping up their marketing efforts to try and capitalize on certain fears, which I suppose is logical.
Generally speaking I think ddg went with the "we don't store data" route because it's the niche they were left with and that ultimately means their search won't improve beyond a certain point.
>Generally speaking I think ddg went with the "we don't store data" route because it's the niche they were left with and that ultimately means that their search won't improve beyond a certain point.
I'd have to agree with this sentiment. After using ddg for about two years, I was kind of expecting something more to come from the whole zero click thing that could change the way users interact with search and how websites and information is discovered (like a different take on SEO).
Now they could totally ride (and should) the "we don't store data" wave, but I wonder how far the ddg team thinks it will take them and does the general population of people equate that to being a better search engine?
Why would storing personal information be necessary (not beneficial) to improve a service? There are plenty of ways to improve user experience, including actually communicating with users.
"Generally speaking I think ddg went with the "we don't store data" route because it's the niche they were left with and that ultimately means that their search won't improve beyond a certain point."
As they will all be too busy not storing peoples data, or how does this line of reasoning work?
Search is based upon machine learning algorithms that use stored data as signals. Google is not only REALLY good at what they do BECAUSE of the amount of data they have on the individual doing the searching, but the benefits of machine learning are that they will get even BETTER as more and more data is incorporated and stored.
Without proper machine learning on the individual level, DDG will never improve beyond a point. They're already kind of shitty, and even if they get millions of more queries they won't be able to improve at the rate that Google does.
It's a trade-off ('privacy' vs. 'quality') that 95% of the world doesn't give a shit about. Therefore I see this sort of move to DDG as just short-sighted and unreasonable overall.
This pretty much captures it. Google knows a lot about you, and it knows a lot about the people you hang out with and the kinds of things you read and the kinds of things you don't read. In knowing all of that stuff they can ascertain that you are part of a community (well a cohort at least) with certain preferences and proclivities, and as soon as a new user identifies as also part of that cohort they can tweak the user experience to "improve" it.
And don't get me wrong, it does improve. I'm glad that I'm not bombarded with ads for TJMax on Google properties but think Digikey could back off a bit.
From a feedback perspective though it gets weird. And you can see that when you search without any identifying information enabled. I found the whole 'Don't Bubble Me' campaign quite creative.
The challenge for search engines though is that being the reference librarian's favorite tool for objectiveness doesn't woo the mass market. And search is a game of numbers if nothing else.
No. Google is a great search engine because of what it knows about you. Trivial example: search on DDG for "ramen": useless generic information about ramen. Search on Google for "ramen": general information on the right, TEN laser-accurate local results for ramen shops. Search on DDG for "tires": ten results for sites that sell tires. Search on Google for "tires": shopping links for tires for my motorcycle. Nice.
Google Now indexes things like your mailbox, calendar, contacts. If you have a local motorcycle shop or something in your contacts or whatever, [.......]
Note that DDG is small enough that they're not even doing their search crawling/aggregation. They have to rely on garbage data they get from Yahoo and other crawlers. Their quality, I think, is much less from the lack of smart machine learning algos, as it is their lack of funds in not being able to afford to crawl the web.
Recent survey conducted by Washington Post-ABC News:
1. 75% of [Americans] are concerned about personal privacy
2. 40% say it is more important to protect privacy even if that limits the US government’s ability to investigate possible terrorist threats
Privacy appears to be a concern to a large percentage of Americans. That said, it's not like all of these people are going to delete their FB account, switch to DDG tomorrow, etc.
However, the numbers do suggest that DDG has a sizable opportunity assuming they execute well and that privacy concerns continue.
"Therefore I see this sort of move to DDG as just short-sighted and unreasonable overall."
What do users of Gnome's browser care about? If users of Gnome's browser are concerned about personal privacy, then the move to switch to DDG for search is a sound choice.
This analysis relies entirely on DDG's privacy being a material thing, rather than an illusion. Nobody on this site nor anywhere outside of DDG has any idea if DDG is capable of resisting coercion or intrusion by state agencies and other bad people.
You can only make your results so relevant if you limit yourself to returning the same results for everyone for a given query. Google uses its data on you to (among other things) tweak your search results based on what it thinks you specifically are likely to be looking for, which DDG will never be able to do.
DDG has (rather successfully) spun that as a critical feature rather than the tradeoff between privacy and utility that it is.
Bad move. It creates an extra step in the installation process. It forces users to then reset it back to Google search, which most everyone will do. Just like when you reset Bing in IE to Google.
Google's 'long term memory' offers lots of benefits too, and 90% of the time, the filter bubble is the correct answer. Like when I search for a particular business on the desktop, and then later I search on my mobile device in Google Maps, and the very first auto-suggest after typing 1-2 characters it the business I searched for yesterday. This is awesome given how irritating it is to enter stuff on mobile devices, especially when in motion, the fewer characters typed the better.
Most of the privacy issues can be solved just by opening up an incognito window. You can choose with 1 stroke whether you want a bubble or not.