The "even in Incognito" part of this is certainly the biggest result I see. And I agree on the study limitation; attempting to clean up localization effects after the fact doesn't feel like a strong fix. It should be possible to isolate device and location effects by using multiple devices in one location, then VPN-ing one device to multiple 'locations'.
One thing that caught my eye was Google's response about Incognito:
> The company did confirm that it does not personalize results for incognito searches using signed-in search history, and it also confirmed that it does not personalize results for the Top Stories row or the News tab in search.
Since it's a corporate reply, the standard question is what's not present: a statement that Incognito isn't personalized, or isn't personalized beyond device type and location. Perhaps I'm too cynical, but "we don't personalize using X" parses as "we do personalize in other ways".
it does not personalize results for incognito searches using signed-in search history
To me this sounds reasonable. A very large number of searches are locality based, and it is entirely reasonable to localize them based on IP address (and - as you note - the device type).
It's also reasonable to customize based on recent (session based) search history (refinements, spelling corrections, etc).
The difference between this and personalization seems mostly about semantics IMHO.
> To me this sounds reasonable. A very large number of searches are locality based, and it is entirely reasonable to localize them based on IP address (and - as you note - the device type).
I wish this was trivial to disable. I regularly search for things where I want the global result, and instead get weird local results that I don't care about. It's much easier to narrow a global search to a local one by adding an appropriate region name to the search than it is to expand a local search to a global one via search terms.
Overall I agree with this, I definitely see why Googlers are frustrated at having all of this framed as 'personalization'.
But broadly, I see three bases for objecting to these changes.
First is the lack of user control. Like many other people in this thread I often want to turn off or 'rehome' localization, not just for weird developer use cases but for obvious stuff like "I'm about to travel and want results for that location". Disabling session-based changes is a rarer desire, but comes up sometimes when a correction or topic change is interpreted as a refinement that's biasing results. Fortunately, resetting Incognito should manage that. (I've never actually wanted to bypass device type adjustments except for dev work.)
Second is inadvertent bubbles. It's easy to imagine content-neutral rules like "show fast and mobile-friendly pages to smartphones" correlating with a meaningful content difference, and the same for location. Hard to really blame Google here, but again it'd be really nice to have the option of a "stop helping" setting.
Third is Google-driven bubbles. Some of the DuckDuckGo examples showed effects like national newspaper articles on a search for 'immigration' getting reordered, or pushing above and below non-news sources. (We can't know if that was caused by location or device type, but let's look at the case where it was.) That doesn't look like basic localization, it looks like non-local results being adjusted based on user location.
This wouldn't have to be anything purposeful; if you add location into your training set and reinforce on the usual 'success' metrics (e.g. first result clicked, final result clicked), you could easily learn that people in NYC and Houston have different behavior patterns and display accordingly. It's open to debate whether this is a bad thing, but it's certainly not what most people (including the Googler who responded to the article) mean when they say "localization".
It's not a secret, but I don't think we're doing enough to keep this on the forefront of people's minds. Every time I hear this, I am shocked! Only to then remember I already knew this but somehow let it slide...
I mean, what's the difference between this and customizing billboard ads based on where the billboard is?
IMO the issue is not google using IP-based location info. The issue (if there really is one) is people assuming/believing the internet hides their location.
> There's a difference between Signed out, and Incognito (no cookies)!
Is there? I was under the impression that Incognito and its cousins generally still accept and preserve cookies for the duration of the temporary session. This means that for this purpose, there isn't really a difference.
Incognito is supposed to give you a completely clean session that doesn't carry over the cache, cookies, etc. from your normal browsing context.
In this study, my understanding is that result personalization carried over from a normal browsing session into the clean, Incognito session, likely due to IP correlation or possibly through User-Agent strings. So while Incognito has its own context that is wiped once the session has ended, the result personalization didn't need anything saved in the browser to recognize who you are.
Hmmm I wonder if using a second browser, maybe based on a different renderer, in incognito with a different user-agent set up would be enough or if it would still get enough info. At that point it would still have your outgoing ip address at least is there anything else they could still match to a signed in session? I guess you could also route all traffic in the unsigned in session through a VPN too.
Incognito will accept new cookies but to my understanding it won't serve existing cookies that pre-date the Incognito session. A fresh Incognito window is supposed to be like you cleared your history and cookies before opening it, and then cleared them again when you close it. But yeah, in between the browser acts as normal.
In a normal (not Incognito) browser window, you don't have to be logged into Google for Google to read Google cookies. Logging out doesn't make you anonymous; they still know who you are.
+ People generally tend to miss the point that Incognito doesn't prevent sharing the IP of the user.
+ I think DuckDuckGo's study missed out using VPN in their analysis. i.e., SignedIn vs Incognito vs (Incognito+VPN)