Does anyone here understand how it is that Bing is powering DDG?
Does DDG just generate lots of Bing searches and scrape the result? If so, why wouldn't MS block them?
Or did they strike some sort of deal? If so, how can that be in Bing's interest? They're just creating a Bing competitor, right? Given that Bing reasonably wants to be the #2 search engine, supporting a formidable competitor striving for the same spot doesn't sound like a winning strategy for Bing. Right?
Microsoft offers a Bing API [1]. So Microsoft is getting paid by DDG for using Bing. Given Google's domination of the search market, I think it makes sense for Microsoft to sell their search API and let other companies help chip away at Google's user base.
Correct. The list price for the Bing web search API is $30/10,000 searches. I wouldn't be surprised if DDG was also able to negotiate a volume discount.
In addition, DDG serves ads from the Microsoft ad network, so they will also make money there.
Also, worth noting that Google provides the same type of service through API. I built a search engine [0] where Google web results are one of many sources used.
If DDG gains sufficient marketshare, MS may just buy them out, for a steal perhaps since they have the ability to tighten the screws. So DDG is little more than an MS proxy aimed at Google’s search business. It’s not really an underdog, just two behemoths fighting for control of the search business. Sounds more like “Meet the new boss, same as the old boss.”
This is only true if DDG isn't investing into its own backend as it grows.
If all the users go to DDG.com, it won't matter if Bing has a great backend or not.
I've recently lived this experience in another context. We built a great backend for a partner that was good at publicity. When they redirected users to their internal backend, we were dead in the water. It didn't matter that the partner struggled with their backend and ours was better. We hadn't invested resources in independent publicity, so users couldn't find us. It was a hard lesson to learn.
And to that point, DDG does have its own crawler, and I believe leverages other third-parties as well (I know Yandex was historically in the mix, for example). So if Microsoft did try to strongarm DuckDuckGo, DDG would have other options, even if less-than-ideal.
There’s no one close to Google in the US. After a big drop, there’s no one REMOTELY close to Bing for 2nd best results. Bing dropping DDG in 2020 or 2021 would be a death kneel for them for US search results. Probably many other countries too. Yandex is too far off right now to be a replacement. I haven’t checked Yandex in a bit of time though. Perhaps they drastically improved.
People in tech circles always say DDG has a crawler. I know they do. No one ever says how big it is. I have a hard time believing it’s crawling more than a couple percent what Bing is crawling.
One complicating matter is that it specializes in DDG's "Instant Answers"; apparently¹ the "source from other search engines" approach really only applies to the traditional results list, while DuckDuckBot is focused specifically on the Instant Answers, effectively optimizing for the absolute most relevant first result possible. So in its current state DuckDuckBot ain't exactly comparable in purpose.
Still, I don't see much that'd be stopping DDG from ramping this up and having it generate the traditional list of results, too. Yeah, obviously the results for quite a few queries will be far from ideal, but if the query already hits one of DDG's curated data sources, I suspect the results might already be better if anything.
> for a steal perhaps since they have the ability to tighten the screws.
You think they don't have some tight contractual relationship with built in renewals and extensions? This is really similar to how corporations lock up retail real estate locations. They don't sign a 1 or 2 or even 5 year lease and then hope the landlord doeesn't rake them over the coals when there is no where else to go to. (Which could often be the case..) You build in renewal options as part of the deal. Those can last a long long time. May even have (probably does) a 'right of first refusal' and so on.
I believe DDG used other solutions pre-Bing; it’s just that, after they switched to Bing, it seemed to work well enough that they kept it. Could they afford the hit in result quality resulting from switching away, if MS started playing rough? That’s a good question, but I’m sure the DDG folks have thought about it.
Depends on the user I imagine. Most of my searches start on DDG and I just got used to expecting certain types of "human phrase searches" to be !g'd, or more recently for me, !s'd.
So if DDG degraded in quality I'd be fine, because I don't expect perfection. Though I suppose it depends on how much, of course.
DDG launched in 2008, and according to the below, the deal between Yahoo and Microsoft didn't happen until 2009 at the earliest.
In July 2009, Microsoft and Yahoo! announced a deal in which Bing would power Yahoo! Search. All Yahoo! Search global customers and partners made the transition by early 2012. The deal was altered in 2015, meaning Yahoo! was only required to use Bing for a "majority" of searches.
DDG was still working significantly with Yahoo! as late as 2016: https://news.ycombinator.com/item?id=12013264 . The links are dead but you can desume from the conversation what it was about.
Note: I know Yahoo is Bing behind the scene - I'm just saying the technical implementation at the time was not tied exclusively to "bing.com".
Good point. Though my bigger point was that DDG has essentially always been dependent on Bing/Microsoft.
2016 is the year Yahoo was sold to Verizon. 2 years later, Verizon writes down $4.6B of the AOL Yahoo subsidiary. A year after that they sell Tumblr in a fire sale. Verizon stopped caring about their web media empire when they did the write down/sale too. Effectively throwing in the towel as a major web advertising hub even though they are likely still top 5 in the west after Google, Facebook, Microsoft, Amazon. That’s peanuts to Verizon though.
I can’t imagine Verizon cares much about their deal with DDG. If Microsoft cared, it makes sense that they could have Verizon and Yahoo do what they want over lawsuit worries from DDG.
I’m being pedantic at this point and totally guessing with my assumptions. I admit the continuous praising of DDG on the web has gotten to me and likely making me feel and react this way.
Maybe DDG is hoarding the results of every API call to Bing to build its own database. Then one day they can turn off the Bing integration and nobody will notice.
2) MSFT has clauses in its contract to address this. If a commenter on HN thinks of this one minute after reading the article, you can be sure that their legal team has thought about it too.
Now I'm curious, how many search queries does the average person make per month? I've been more and more dissatisfied with results I'm getting and would happily pay $10-20 per month for search. Especially if that means I'm not getting tracked and not seeing ads. Or maybe just seeing ads that aren't being in-lined.
Can't answer that question, but it's worth mentioning they also add their affiliate tag to Amazon product links and possibly others to get some revenue.
For those curious, "[Google] Custom Search JSON API provides 100 search queries per day for free. ...Additional requests cost $5 per 1000 queries, up to 10k queries per day."
It seems runnaroo would therefore be limited to 10k searches/day?
That sentence could also mean "$5/1000 applies up to 10k queries per day (and you get to talk to a sales rep if you do more than 10k queries per day)".
Not necessarily. Startpage uses Google for all of its results.
> You can’t beat Google when it comes to online search. So we’re paying them to use their brilliant search results in order to remove all trackers and logs. - https://startpage.com
Does anyone know how one can get whatever deal Startpage has?
Just wondering, are you a one man show running a search engine. That is extremely impressive. I did some relevant local searches and was able to find what I was looking for in the first page.
Some people do. I do. On places with vocal minority (or in general sometimes) like HN, reddit, Twitter, you’ll see many more people tout the trendy/cool DDG. As you said Bing/Microsoft aren’t cool.
I’m talking about the majority of these people who outside of basic ad/tracker blocking and using DDG, don’t stick to caring about privacy above most other things.
A lot of what I’m saying doesn’t count for HN’s crowd as much as other bigger sites.
I don’t have any backing for this statement. Just what i believe from common sense looking around.
It's a pretty great deal for Microsoft, because they get paid twice. Once in dollar value for access to their API, and once in user queries. Even anonymized user queries have value in terms of learning what users are interested in for search purposes in general.
Bing also gives Microsoft:
- a relational knowledge graph
- a way to annoy Google
"Annoy Google" is a "commoditize the complement" play.
Bing (and the Bing powered Yahoo + DDG) create a viable alternative where otherwise there is none. Google now has to constantly spend to keep their search tech ahead, and Mozilla and Apple can now threaten to switch default search engines forcing Google to pay protection money. Altogether, Bing forces Google to burn around 15-20 billion per year. This is money Google can't use to poach Microsoft engineers, or push G-Suite against Office, or Chromebooks against Windows, or Google Cloud against Azure.
Microsoft already has open sourced many key Bing technologies, and actively pays consumers to search with Bing. The primary goal isn't revenue, it's commoditizing search.
Yep, but with the msft focus on the cloud, their priority is to take from goog.
Additionally, people buying OSX computers isn’t really a bad scenario for Microsoft. MS office is quite popular on OSX and iOS and people spend more on that than they do on operating systems.
If Apple really invested in OSX it could be a threat to windows, unfortunately, combined with the high cost of the hardware, OSX's downslide means it's stuck to its current pro market...
Apple is making investments in OSX: Catalyst, T2 chip /security, Touch bar support, iPad mirroring, airplay, arm64 support (speculating), SwiftUI. Metal/GPUs. Core apps: logic pro, xcode, playgrounds and brining more iOS apps (news, stocks, apple tv app, apple arcade with subscriptions). I also suspect with arm64 will come better cameras (lidar perhaps), beefier GPUs and neural cores. all of this running on macOS.
What kind of investment do you think they need to make to be a threat with Windows? If you think they should be trying to "match" windows with things it does, it would be a bad strategy that isn't winnable. Throwing away money. At least with their current approach they are levaring strengths (verttical integration, mobile dominance, etc) to build something that could leapfrog windows one day. Especially if the user journeys that span into other products (hw accessories, phones, etc)
DuckDuckGo pays money to Bing to use their API [1], presumably enough to outweigh any loss of revenue. DDG also uses Bing's ad network, so Bing makes money regardless of what engine a search is made on.
Also, there's speculation that Microsoft is starting to give up on getting wide adoption for Bing among consumers [2], and given Bing's low market share, Microsoft has probably determined that Bing is never going to get widely adopted enough to the point where having a deal with DDG would be a bad idea.
My guess is the companies DDG partners with (I believe Yandex and a couple others, in addition to Bing) need the revenue from such a deal, and probably consider the enemy of their enemy their friend. Google having a search monopoly hurts all search engines that aren't Google. So any search engine that cuts into Google's share is helpful to all of them.
I imagine Microsoft and friends are also still getting some idea what queries people making on DDG, if those queries are being forwarded to them to get results. Not user-correlated, and hence, nowhere near as valuable as if someone searched on Bing while signed into their Microsoft account. But it's probably got some general value to them on search behavior.
Well, not user-correlated via traditional means. DDG is proxying user queries over to MS, but unless they're taking steps to actively fuzz the queries, there's all kinds of secondary signal MS could possibly be sampling out by correlating queries across time (especially since DDG also vends Microsoft-provided ads in the sidebar; it's unclear what signals MS is receiving about a user's session from DDG via server-to-server communication, or how MS might correlate that data. If MS vends an ad and the user clicks through, and MS gets signal that the user clicked on the ad from DDG, MS can correlate that the user who saw ad Y did search X).
Yeah, Yahoo used to have their own search index, and very developer friendly search API called Yahoo Boss. That was actually the main source of DDG results originally.
Yahoo has since given up on Search, shut down the Boss search API, and is now also fed by Microsoft Bing.
Presumably DDG either pays MS for the search feed or has some sort of ad sharing agreement. So MS makes money either way and is banking on DDG cutting more into Google than Bing market share. Strategically I suspect MS cares more about Google not having a search monopoly than the money Bing makes them.
You are missing the entire point, "search" is sadly about advertising, not about the search itself :-)
Bing is interested serving DDG, Qwant, Ecosia and a lot of other unknown search engines because of the aggregated reach they provide for their ad network. Ad-networks only work if the aggregated audiences are massive, otherwise advertisers do not bother putting their ads there, only the top-3/5 ad networks get to see any action. So Bing wants/needs a bigger audience just to be on the game. They can grow in 2 paths: 1) increase Bing search reach (difficult), or 2) use partners with different value propositions.
Bing charges little for 1K query, 1USD officially but it gets cheaper, to zero :-) The real thing though, is that if you display Bing ads, you get a 70%-90% rev-share of the ad-revenue, which varies from country to country, something between $5 to $20 per 1K queries.
So, DDG basically gets around 5$ to 10$ net for each 1K, and can spend all that money on distribution and marketing so that they get even more users. Bing gets the rest, money, and what's more important, their ad-network continues to be competitive.
Everybody wins, right? :-/
Search is so cheap 1$/1K queries and people makes 2/3 queries per day, so $1/year/user (average). It makes no economic sense to build an alternative. Unless of course, you are building out of "ideals".
I work in the search engine space. Pretty much everyone is either Google or Bing under the hood, including DuckDuckGo. Deals are made on ad revenue shares generally. If I recall correctly DuckDuckGo just monetizes on product affiliate links though.
Fun fact: DuckDuckGo used to use Yahoo, but Yahoo just has a deal with Bing. In other words, DuckDuckGo paid Yahoo, to pay Bing, to deliver Bing search results. :)
Now they use other resources as well, and also have their own internal web search.
Yahoo search is largely powered by Bing, as well. That's kind of funny since MSFT wanted to buy Yahoo at one point.
Also, there is the concept of dependent competitors. In old days, GM sold transmissions to American Motors, for example. So American Motors was a dependent competitor of GM.
> We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).
That kind of stuff still happens with automakers -- maybe even more so these days. Heck, sometimes competitors buy an entire platform from another manufacturer and put their own branding on it.
Does DDG just generate lots of Bing searches and scrape the result? If so, why wouldn't MS block them?
Or did they strike some sort of deal? If so, how can that be in Bing's interest? They're just creating a Bing competitor, right? Given that Bing reasonably wants to be the #2 search engine, supporting a formidable competitor striving for the same spot doesn't sound like a winning strategy for Bing. Right?
What am I missing? :-)