Hopefully this will let each of them compete on their own merits.
I’ve been tossing up moving our workloads to Elastic Cloud anyway, because AWS ES Service is a source of constant headaches for us. Feels like at least once a week a server ends up in a state where we can’t fix it, and AWS engineers have to manually fix their internal state.
Their standard response is “add more nodes”; well, we did that, and it is costing us an arm and a leg, and it didn’t fix the problems. (Plus, now we have new problems where networking blips appear to be causing quorum problems and sending the cluster into a death spiral.)
Purely off the customer experience, it feels like Elastic Cloud has to be better; the whole licensing debacle has definitely turned me off Elastic though.
I also had a serious problem with AWS Managed ES. In my case, some of the heap didn't free on every garbage collect which would eventually result in cluster failure. This was likely a JVM misconfiguration and was most easily observed by viewing a shrinking sawtooth pattern on the memory graphs. This resulted in a multi day marathon of sleeplessness keeping the cluster alive by continually rolling it every few hours (We initially assumed we had done something wrong and investigated ourselves first... eventually I shunted traffic to both my own cluster and the managed cluster... my cluster did free heap as expected and we successfully switched over without downtime but wow was it ever hairy during high traffic periods).
We saved a bunch of money and gained performance by using our own cluster. That cluster hasn't gone down since... years later.
It's very difficult to debug these problems when you don't have direct access to elastic search's configuration... what would normally take minutes to verify can take hours to isolate.
I'm fairly confident it was because I ended up finding a ticket later where very similar behavior was isolated to a jvm.options configuration problem.
Effectively a newer config file was lacking a jvm.options line that changed the behavior with an older machine setup. I would not be surprised if AWS deployed a new config to an old environment.
Unfortunately, not having direct machine access, I could not confirm whether this was the case in the failing cluster.
FWIW that experience is exactly the same as my team has... But we're on Elastico hosting. Constant black box breakages that can't be debugged, nodes getting restarted for opaque reasons... We've been considering switching to Amazon's hosting to avoid these.
Sounds like it might be a "grass is always greener" scenario. ...maybe I'll spend more time looking into self-hosting...
Ditto. Really want to support Elastic but have found their cloud offering just randomly breaks.
Latest problem is we lost a node and the new node couldn't recover its shards from backups. Constant crashing until we upsized the instance to have more memory. Makes no sense and support wouldn't lift a finger to help.
We had similarly terrible experiences with AWS ES, so we moved to self-hosted Elastic. It's better but still pricey and requires more manhours dedicated to tuning.
Their move away from open-source has been unfortunate. For that and some other reasons we've ended up more impressed with Logz.io and Splunk SaaS.
What kind of tuning do you do? We spend time but typically on upgrades… very rarely have we spent much time on tuning I’m curious what kind of tuning are people doing beyond mlock and having enough memory / nodes?
I mean, logz.io is okay and all, but it's quite expensive, and has a loooot of outages. I don't remember a week where we didn't have issues, the most common being log ingestion lag, which take hours to fix. That, and their API is bad: you can only query over a 2 day period and their API keys allow full control over everything.
I think the AWS ES Service is the worst AWS service I've ever used. WE have constant issues with it and AWS staff don't seem all that comfortable working with it - which is rare in my experience. All in all, it's the most hands-on 'manged service' I've used.
Like the others have said, definitely seems to be hit or miss. I inherited an old cluster running on elastic cloud, migrated it to AWS managed and it reduced monthly costs by nearly an order of magnitude on top of going from a 5-15 minute outage per day for master node election to having 3 over 2 years.
We’ve rewritten most of the interactions with the older version of ES into a new service and use a self-managed OpenSearch cluster on Graviton instances and it’s the most stable Elasticsearch/Solr solution I’ve ever interacted with
I hate to say it but your problem here is ES itself.
If you just need "Lucene but clustered" I highly suggest looking at Solr instead, it's design is much more straightforward and has pretty much all the most important knobs for actual indexing that ES has.
If however you are tightly coupled to ES API or use it with 3rd party systems you are sort of up shit creek without a paddle...
Ran ES at large scale for many years, eventually gave up and only use Solr or custom built search engines these days.
We run our own ES cluster it’s not very hard… ec2 instances. The hardest part is just reading the docs carefully before upgrading… we also don’t modify schema eer mappings very often…. Oh and we run 3 masters, and 3 clients and then the rest are data nodes… since doing that knock on wood super stable
> Their standard response is “add more nodes”; well, we did that, and it is costing us an arm and a leg, and it didn’t fix the problems. (Plus, now we have new problems where networking blips appear to be causing quorum problems and sending the cluster into a death spiral.)
This. I have had similar experiences and adding more nodes had just amplified the issues; nodes were all also barely loaded.
Looks like pretty good news. Have worked in AWS before. So AWS is very famous for making money using open source products without contributing upstream.
One very good example is Amazon redis. Amazon figured out that redis asynchronous replication didn't work at scale so instead of fixing issues upstream they chose to develop Amazon redis in house and monetized it.
Disclaimer: ex-AWS as well, worked very closely with MemoryDB, but not paid to shill!
You're entitled to your opinion but your line of reasoning for how MemoryDB for Redis came to exist or the reasoning about why it isn't in upstream Redis is not factual. MemoryDB's architecture uses Amazon's home grown log replication services as pointed out by Werner Vogel in his blog post about MemoryDB[0]. This architecture is fundamentally incompatible with upstream Redis. The real reason why MemoryDB for Redis exists is far less juicy: MemoryDB came about by meeting customers where they were. Customer's love Redis, especially for its non-caching features, but replication is a headache with all the existing solutions today.
Also as far as I know one of the lead committers to Redis is from AWS.
I’m a total AWS fanboy, but even I cringe when I read that superficial, customer-centric sound bite. You know who meets people where they are? FOSS maintainers.
“Also as far as I know one of the lead committers to Redis is from AWS”
Conveniently cryptic, what does from AWS mean? Do they still work there? Did they specialize in log replication?
Disclosure: I work for AWS, but I don't work on Redis or managed services based on Redis.
Madelyn Olson, at present an AWS employee, is one of the members of the Redis core team [1]. The invitation was extended because she had "been actively involved in Redis development for several years, contributing numerous changes throughout Redis, including bug fixes and features."
I think it’s the fact that the implementation details don’t change the optics; AWS had the resources and talent to both monetize AND be a Good Samaritan by contributing to the upstream. The contributions could have been RFCs, GitHub issues, etc. With that said, I can imagine the rumored culture of AWS engineering doesn’t support wondering souls that typically nurture thoughts of community service for the upstream.
There were contributions sent upstream for Elasticsearch bugfixes and enhancements. Some of those PRs are still open, for example [1].
A sampling of additional PRs can be found in this blog post [2].
Further contributions upstream to Apache Lucene have been growing over the years. The new Approximate Nearest Neighbor support in Elasticsearch 8.0 comes from work that was sponsored by Amazon in upstream Apache Lucene [3].
AWS had proprietary infrastructure that could solve the problem at hand; it's not reasonable to expect them to invest an incredible amount of effort to solve it again. AWS customers do not net-benefit from that duplicative effort.
GPL wouldn't prevent Amazon from forking and then providing managed instances of the project without releasing the source code of the fork. AGPL would, though.
But yes, this is the reason I won't work for free on my own or others' BSD or MIT licensed projects.
BSD and MIT don't grant patent rights; this might be one reason Facebook uses them, though they did try to get predatory in the only way Facebook can: https://news.ycombinator.com/item?id=14779881
ianal, but imo, Apache License v2, Mozilla Public License v2, and xGPLs v3 are better at protecting the rights of the consumers (including contributors).
Sometimes fixing upstream is hard / not possible, when maintainers don't want to accept others' proposals/vision, or are cautious to change architecture with breaking changes.
I don't know much about this particular case. However generally speaking that is not true. They would just need to make the fork public. Whether the maintainer accepted the changes upstream is irrelevant.
Unless they actually tried to upstream it, this point doesn't really matter. From what I can see, they never actually tried or intended to upstream their changes, so this is irrelevant. They didn't even release the fork's source, let alone try to upstream it.
No, not everything negative is the new Oracle. AWS has very little in common with how Oracle has operated historically.
Oracle didn't build their company in the style of AWS Redis, that cloning maneuver. Oracle's database was a pioneer. Oracle didn't get where they are by cloning open source and claiming it as their own. Despite the numerous bad things that can be said about Oracle's culture, that's not one of the key negatives about Oracle.
Had OpenSolaris developed a sufficient community outside of Sun/Oracle, this would have been much less of an issue. Which is why community is a big deal. A large company can always decide that some project is no longer interesting and, if they're the only ones supporting it, it's going to die. They certainly have no obligation to support it.
At this point in AWS' life, enterprise sales is king. Not surprising that there's shades of Oracle / Microsoft in them. May be, Google hired Oracle #2, Thomas Kurian, to head GCP for similar reasons. Like it or not, Oracle-sized shadow looms large over BigCloud.
Why? They're all enterprise software companies. They participate in open source to greater or lesser degrees. They seem absolutely part of the same category. This isn't either positive or negative commentary but just observation of what they do as businesses.
I was sort of curious, so I went to see what impact this had on the customer experience at AWS. Searching for 'elasticsearch' in the AWS console services dropdown now yields:
'''
Amazon OpenSearch Service (successor to Amazon Elasticsearch Service)
Run and Scale OpenSearch and Elasticsearch Clusters (successor to Amazon Elasticsea...
'''
This seems like a petty, small win from the Elasticsearch people. I understand AWS has a history of gobbling up OSS and productizing it, and that that's detrimental, but it's hard to see Elastic, Inc as anything but sore that they got their lunch eaten here. Maybe that's justified. But it comes off as incredibly petty.
(disclaimer: i used to work at aws, but not anywhere near the referenced offerings).
It’s “detrimental” to companies who perhaps shouldn’t have chosen a license that allows AWS to do exactly what they did. Those companies can’t simultaneously claim to be competent and to have made that choice without knowing what they were doing.
IMO, they chased the benefits of being open source and then changed course when the costs to them exceeded the benefits (which is fine for code going forward), but trademark concerns aside, I can’t see AWS as the bad actor here.
If anything AWS has increased its co-operation with F/OSS businesses of late and this clear shift in strategy was apparent in product / partnership announcements leading up to 2021 re:Invent. AWS, I believe, realise the F/OSS ecosystem mustn't be taken undue advantage of. I mean, AWS stands a good chance of getting caught in a vehement backlash (let alone sporadic bad PR) from the developer community, who ironically form the basis of an entire industry AWS sells into and operates in.
With Microsoft + GitHub intensifying their investments in F/OSS, AWS had to play ball. It is smart, not petty on anyone's part.
Judging from the tone of the article, I am glad Elastic is content in their current business relationship with AWS. Hopefully, the companies also find an agreement to have AWS' OpenSearch fork merged back in, as well.
If the rumours here on HN are true they need to start working on giving back some of their internal patches too. I think they also need to start providing funding to FLOSS projects too, and doing something like Google Project Zero.
> It’s “detrimental” to companies who perhaps shouldn’t have chosen a license that allows AWS to do exactly what they did.
Yes, it is. Why the scare quotes?
Lots of things are legally allowed that would be very detrimental to me. Or you.
> Those companies can’t simultaneously claim to be competent and to have made that choice without knowing what they were doing.
I'm glad you can tell the future.
> IMO, they chased the benefits of being open source and then changed course when the costs to them exceeded the benefits (which is fine for code going forward), but trademark concerns aside, I can’t see AWS as the bad actor here.
You can at least see how the system has big problems, even if you don't want to blame Amazon for them, right?
You say "system has big problems"... so if you had magic power to pass any law and have everyone follow it, what would that law be?
I am guessing you didn't like how Amazon was profiting from ElasticSearch's work.. so would you retrospectively change ElasticSearch's license to SSPL? Or maybe help all companies: prohibit all permissive licenses (Apache, MIT, BSD etc..) entirely and force people to either AGPL, SSPL or fully commercial? Or make a law that no free software license may apply to companies with >$1 billion in revenue?
While we are at it, how about prohibiting commercial flat-rate unlimited site licenses? The huge companies can take advantage of those too.
The thing is, I bet Elastic's original decision to use Apache was to get more users, at the expense of also getting more competition. Now they have their users but they don't like the price they paid for it. This may have been a bad business decision on their part.. but I don't see "big problems" there, companies make bad business decisions all the time.
Don’t forget that the core of elastic is lucene, the full text library that is also the core of solr and that has an Apache license, without lucene elastic wouldn’t exist.
I can see several cases where open sourcing your core product will be very helpful to your company at one phase and harmful in another. Whether that’s a big problem in the system is a topic that people will differ on.
I get it, you see this as complaining about how things shifted when this is a tradeoff they embraced from the start.
But there are situations where it's not that things changed, with companies that would love to be open source but it would hurt them at all phases.
If you have to be super proprietary to make any money off software, that's bad for everyone.
When a piece of code is responsible for enormous amounts of money and none of that goes to the author, that's a bad way to get more code and a bad way to support valuable authors.
Does the presence of AGPL address most (almost all?) of your concern? It seems like that would offer individual users freedom to use and modify the source while precluding Amazon from monetizing it effectively.
Those same properties of AGPL mean it’s generally viewed as not as effective at attracting a community of early users, but in a case where the alternative is proprietary-only and the company “wants” to be open-source, AGPL seems a better option for them.
This comment is in every thread about this, but there has to be a solution that satisfies FOSS purists as well as casual users without allowing a massive, evil megacorp to just stomp on every company built around open-source solutions.
> there has to be a solution that satisfies FOSS purists as well as casual users without allowing a massive, evil megacorp to just stomp on every company built around open-source solutions.
The reason for everyone not selling the software in question to prefer F/OSS is exactly that it requires surrendering the kind of exclusive control that is also what allows you to build a company around the software as a product.
Yes, this means that F/OSS as such is unlikely to be the key product of a successful company. That's always the way it has been with F/OSS.
“But if I can’t center my business on F/OSS-as-product and I can't center my business on commercial software for the same use because F/OSS products, even if somewhat inferior themselves, develop more robust ecosystems, then how am I supposed to compete with free?” one might ask. And the answer is this: “Maybe you’re not: you are entitled to a business model.”
How is AWS stomping all over it? Being able to fork for your own needs is a good thing. You don't make something open source and accept the world's free contributions without acknowledging that.
You also never hear anyone complain about the thousands of hosting providers that build their business around the LAMP stack. I wonder if it is because Amazon is so big, or just the lack of competition.
I think it's a general sense of unease with the probability that future F/OSS looks like "{Project}, sponsored by Amazon" due to cloud market dynamics and size.
Coupling that with the fact that, in the past, F/OSS project control or domination by a single party hasn't turned out well.
Again, you’re leading with the definition-of-FOSS argument.
Step back from FOSS for a second.
I think most people would agree that there’s somewhat of a moral issue with just taking someone else’s open source software and just hosting it and making billions, with nothing for the creators, because you are a megacorp who is good at hosting.
Now, is there a way to solve that and have the benefits of FOSS?
Both Mongo and MariaDB have tried to address this with licensing - MariaDB seems to have done this much less clumsily than Mongo, but both still had FOSS advocates shrieking
---
EDIT to include response to below comments so everyone doesn't keep repeating the same things:
> Either your product is fully free and you accept that people can fork it, even Amazon, or you choose a restrictive license. You can't make "free" (as in libre) compatible with "but".
> I can totally see that you think this is unfair, but they did allow it and they were perfectly happy when being FOSS brought them market share. You have to take the good with the bad
> If you have a moral qualm with bigcorps using your work for free, you don’t license in such a way that they can. Make your own license, or slap AGPL3 on it - either way, no bigcorp touches it.
> But you cannot be mad when you say “I release this code under these terms” and AWS takes you up on your offer.
If everyone stomps their feet and says "there's no solution, otherwise it's not OSS!" then the end result is only going to be a lot less open-source software.
I'm not asking if someone who wrote code and chose an existing OSS license that allows any possible usage has any real recourse once they want to make money off of it - obviously not.
I am asking, can we find a way to ensure software creators are compensated while still having the generally-recognized benefits of FOSS?
> Don't you think there's a moral issue expecting free contributions to something which only you are allowed to monetize?
Nowhere in OSS licenses does it include the expectations of free contributions
> And how can you satisfy users to the greatest extent while also preventing them from using the provider that is best able to meet their needs?
Again, can we just find a way for the creators to be compensated?
I'd love for the OSS to be available as a hosted solution to be available on every cloud provider, including if the original creators choose to provide a cloud hosting.
I'd also love for the original creators to be able to get some sliver of the money AWS/Google/Azure are making off of it.
---
To me it seems like if you were to start a business around your open-source code right now, you would be smart to do one of these two things:
1. a MariaDB/CockroachDB BSL approach, where the newest versions are BSL, with a restriction that you can't provide a product where you just host our software without paying us, then all code automatically is fully open-source (GPL or whatever) after x amount of time, or x major version releases.
- this comes with the danger of all the bad PR and gnashing of teeth that comes with non-pure-OSS licenses, or people just not adopting it because they dislike or don't understand the license
2. Open-Core with non-OSS-licensed Enterprise versions, and lean really hard into developing hosting and enterprise integrations to make sure you can make some money off the nonfree portions
- this seems even more dangerous, that you have to be distracted on such things that are not the core product in order to defend against ending up like Docker, with everyone using it and no one paying you a dime.
You’ve got it precisely backwards. The license placed on a software encodes what you wish to allow other people to do with it.
If you have a moral qualm with bigcorps using your work for free, you don’t license in such a way that they can. Make your own license, or slap AGPL3 on it - either way, no bigcorp touches it.
But you cannot be mad when you say “I release this code under these terms” and AWS takes you up on your offer.
Disclosure: I work for AWS, but they don't pay me to share my personal opinions on FOSS licenses here on HN. I'm speaking for myself.
Personally, I am a fan of AGPLv3. But I am uncomfortable with advice to apply APLv3 code in an expectation that BigCorps will not touch it. There are many very large companies, including multinational companies based in China, that have no fear of AGPLv3, and are also prepared to meet the obligations of the license.
Many times I have read (including in comments down thread) that "big companies fear AGPLv3." That may be true in some cases, but not all. And the state of adoption or rejection of community licenses changes over time. 25 years ago, many companies "feared" GPLv2.
That changed, and now GPLv2 is widely accepted. The same could happen for AGPLv3, and personally I think that the movements that advocate for Software Freedom will be closer to achieving their goals if that happened.
Maybe I got the timelines wrong - isn't that what was originally happening with Mongo and AWS - AWS was hosting Mongo when it was still under AGPL?
Also the Google policy seems to stem from:
1. It's never really been defined very well what "interaction over a network" means for AGPL and how truly viral it really is - although I'm sure ever there were ever landmark cases in the US establishing it, things would magically end up in Google's favor
2. Google purposely spreading FUD around more restrictive OSS licenses to discourage their existence
Interesting - so a clean room implementation is somehow more attractive than complying with the AGPL.
So AGPL is open source, but in practice it seems like it's so much more restrictive than something like BSL that is not open source.
This just makes OSS vs not-OSS seem like even more of an theoretical line in the sand.
As for
> Hence, Mongo relicensed under a new license, the SSPL, that says if you implement an SSPL-covered API, the SSPL extends.
That definitely looks ugly on Mongo, implementing an API is such a fundamental part of software compatibility. Seems like the exact wrong way to go about it.
Amazon didn't implement an entirely new database. As you can see on https://en.wikipedia.org/wiki/Amazon_DocumentDB, they just built an (admittedly clever) PostgreSQL plugin on top of Aurora PostgreSQL.
Aurora PostgreSQL is based on Aurora, which amusingly traces its roots to MySQL, but that was a long time ago. You can decide how many planks Amazon must install on the ship of Theseus before you're willing to call something "a new database", but the fundamental point is that none of those planks came from MongoDB, thanks to AGPL3.
This is one of the big reasons I think that discussing "open source" is dangerous and tricky. It's not a binary, and there's no one monolith that decides what it is/it isn't. Even ignoring the traditional "FSF vs BSD" holy wars, there's plenty of depth. For example: is the JSON license[0] open source? It's certainly permissive enough, but lots of different groups have different interpretations as to why it isn't: the FSF[1] says it violates their freedom zero, that a user has the right to run a program as they want for any purpose; the OSI says it isn't because their licenses cannot discriminate around intent - quite literally, the OSI supports Evil here ;-)
I've released code under permissive licenses (Apache, BSD, and MPL that I remember off-hand), and at least one feature I know made its way into a commercial product. In those cases I simply didn't care because I didn't consider the code particularly novel, important, or special. It scratched my itch. Occasionally it scratched my employers itch, so I was already paid to write it, which was nice.
If I ever write code that I consider novel, important, or special, I plan to release it under AGPL3 - while you may think Google is overreacting on accident or to create FUD around it, the plain text of the license makes the intent exceptionally clear:
> public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version.
This alone is enough to make the AGPL3 blacklisted anywhere that wants to keep their source closed: you may believe the courts would decide for the big corporation if push comes to shove, and you may even be right, but practical evidence shows that almost nobody wants to risk it.
Part of the problem here is that a generation of programmers seems to think that open source is all the same, so a license is something that you just whack <enter> to when your scaffolding tool suggests one. But in reality, talking about "open source" is useless at best and dangerously imprecise at worst, talking about licenses is the only way to get people on the same page.
> That definitely looks ugly on Mongo, implementing an API is such a fundamental part of software compatibility. Seems like the exact wrong way to go about it.
It did the trick for them, at least against AWS: as expected, they successfully forced the wire format to diverge after AWS launched DocumentDB. It also got them pulled from Debian, Fedora, and I don't know how many other distributions for being non-free. But hey, at that point, they were already a known quantity, so it probably won't hurt in the long run.
> If everyone stomps their feet and says "there's no solution, otherwise it's not OSS!" then the end result is only going to be a lot less open-source software
There's going to be less people trading on the idea of open source software while relying on the exclusivity of control of proprietary software—which in fact does not solve the problems which lead users to prefer open source software—to enable monetization, so we’ll be back to the status quo before the last handful of years where open source software was peripheral projects and supporting infrastructure funded by its users either as internal projects or via external foundations, rather than the central products of startups that have no business plan consistent with the product remaining open. But that’s okay.
I mean it seems to me like of these two situations:
1) there's a lot of software that's mostly-open-source-except-Amazon-isnt-allowed-to-run-a-hosted-version-of-it-without-paying-the-authors
2) all the software from 1) is completely proprietary
a lot more people benefit from 1 than 2. But somehow it seems that OSS purists prefer #2? It seems like they would rather less Open-Source Software in the world, all to maintain the purity of the meaning?
> there's a lot of software that's mostly-open-source-except-Amazon-isnt-allowed-to-run-a-hosted-version-of-it-without-paying-the-authors
That's...not mostly open source. Being able to pay whomever to run a hosted version for you and not have that be exclusive to the original creators isn't, especially in the age when hosted services are in high demand, a peripheral feature of open source. It's integral to the value proposition.
I said nothing about having hosting "be exclusive to the original creators"
If AWS/Google/Azure had to pay some fee to the original creators to provide a hosted version, they would definitely do it for a popular enough product.
Everyone would still get all the benefits, except AWS/Google/Azure get a little less money (down from some hundreds of millions) and the creators get a little more than $0.
I fail to see the negative effects on any entity there except what would amount to rounding error in AWS costs. Doesn't actually seem so integral to the value proposition of open source. Definitely seems "mostly open source".
Yes, because OSS software enjoys a massive popularity boost -- there are a lot of people who will always chose an open source product if one is available. So in the option (2) some software will turn proprietary, but I bet that overall number of OSS users will not change that much... it will be just re-distributed.
For example, if ElasticSearch would have been closed source? People would go to Solr. If Solr closes? Postgres and a lot of swearing.
You have to remember that while AWS ElasticSearch was a bad thing for Elastic Co, it was absolutely great for all sorts of much smaller companies who did not have enough resources to run their own clusters, as Elastic's offering is 2x price of AWS one [0].
I don't think it's a matter of maintaining the purity of the meaning, it's about maintaining the intent. The point of FOSS isn't to let people benefit up to the point where they start being able to outcompete you, the point is to let people benefit freely. If you want to achieve the former, you can just release it for free for non-commercial purposes.
> Now, is there a way to solve that and have the benefits of FOSS?
The answer is simple - no. Either your product is fully free and you accept that people can fork it, even Amazon, or you choose a restrictive license. You can't make "free" (as in libre) compatible with "but".
I can totally see that you think this is unfair, but they did allow it and they were perfectly happy when being FOSS brought them market share. You have to take the good with the bad.
If everyone stomps their feet and says "there's no solution, otherwise it's not OSS!" then the end result is only going to be a lot less open-source software.
Well, it's just the free speech issue all over. Either you censor speech or you risk people saying things that you really don't like. This is exactly the same situation: Either you go FOSS, with all the benefits it entails, and risk getting forked by a major player or you go proprietary and risk not getting adopted at all by major players and open source enthusiasts.
Don't get me wrong, I feel with the Elastic team and I can understand why they're unhappy (even if I don't agree with their pettiness). I also don't particularly like the move Amazon did here. But that's why we want free software: If the current owner of the software does something you don't like, you can still use the software. That's the whole point.
And you're still free to pay Elastic to manage a cluster on AWS for you; Amazon might have forked the project, but it's still the customer that needs to switch to OpenSearch.
The end result may be that there's a little bit more proprietary software; so be it, people could already make proprietary software. We'll still have plenty of Open Source software that's actually Open Source. That's better than losing Open Source altogether.
The question was not "would it be FOSS". The question is whether you can have the benefits of FOSS, or as much of them as possible.
Imagine just as an example, a license for personal software that is exactly the same as GPLv2 except billionaires can't use it. Functionally that has the same benefits as FOSS, even though it's not.
Honestly it seems like the only downside there would be all the OSS purist/billionaire-defender HN commenters would have a field day, then you could move on.
One of the key benefits of FOSS is that if I don’t want to (or am not capable to) do something with the software, I can pay someone to do that part for me. Maybe that’s fix a bug, extend/modify the software, or maybe it’s operate and host it for me.
I’d like to choose AWS to do this hosting in a lot of cases, assuming they’re willing. Barring them from hosting it is impinging on my freedom as a user and therefore with respect to whether I think that software is Free and Open.
That’s the exact freedom that Elastic is trying to remove here, for their own (totally valid) business reasons (they don’t want to compete head-to-head with AWS hosting), but in so doing, they’re removing an essential freedom from users. This is not about defending billionaires; it’s about having choice in the use of the software purporting to be free and open.
I don't think they want to stop you from doing those things, and hiring out to do that, I think they want it to not be offered as a prepackaged product.
So it's a reduction in freedom, but it's not a reduction in user freedom.
Those are just differences in timing and system efficiency. In one view, RHEL is a pre-packaged product. In another view, it’s a bunch of users outsourcing a portion of FOSS management to an expert company because they don’t want to do it.
I don't think I understand your argument. Even if it is just timing and efficiency (I'd argue it's a few other things too), wouldn't that mean it's not just a matter of perspective?
Fair. I retract “just” as I agree it’s a few other things, but in one world, I can contract out my Linux packaging to Redhat or my Elasticsearch hosting to Amazon and in another world I can’t. I think that’s a significant difference in freedom to me, as the user/potential user.
“You have the freedom to contract for help with our software with this company but not that company” or “you have the freedom to contract for hosting in inefficient arrangements that we can beat in competition but not in efficient ways that we’d rather not compete against”.
From a purely practical angle, how am I to ascertain why they chose Apache 2.0 as the license and how should I weigh that knowledge/guess against the crystal clear grant terms of “Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.”?
If I license any of my code under that license, I hope someone builds a billion dollar business on it. If someone else does the same, how should a licensor understand the difference and act accordingly?
Don't you think there's a moral issue expecting free contributions to something which only you are allowed to monetize? And how can you satisfy users to the greatest extent while also preventing them from using the provider that is best able to meet their needs?
EDIT: regarding your inline response,
> Nowhere in OSS licenses does it include the expectations of free contributions
The expectation is not encoded in the license, it is the reason for choosing the license. Why would anyone want to afford you the benefits of FOSS so that you can personally monetize it and nobody else?
There are plenty of reasons for choosing an OSS license - people can easily audit the code, people can self host for their own purposes, etc.
There is plenty of code that is only open-sourced after it has been fully created by a single organization/entity - no free contributions needed!
If we're talking about expectations not encoded in the license it's very clear that developers would just like to be able to release software with all those OSS benefits, but also to have a way to be compensated when AWS takes it, charges people to have it run it on an AWS server, and makes a ton of money.
I'm not denying AWS is adding value, but it sure is odd that they're the only ones making a dime there.
I don't think those are really reasons for choosing an OSS license, they are reasons for choosing a source-available license (of which OSS licenses are just one kind).
> I'm not denying AWS is adding value, but it sure is odd that they're the only ones making a dime there.
I don't understand what's odd about it. If ES were never open source and was a proprietary product from the beginning, there would be nothing stopping Amazon from paying for it and providing it as a service under contract from Elastic, if they thought it added enough value to make that worthwhile. Then they would both be profiting in a monetary sense from the relationship. But it's not a proprietary product (or at least, it wasn't previously)
There is a talk by an OSS person (I forget the details), where they make the argument that AWS selling hosting for a particular piece of software increase the market for that software, which means that while AWS takes some share of the pie, the size of the pie is bigger so the original author's share goes down but the size of their slice goes up. So I'm not sure in every case AWS selling hosted software is actually a bad thing for the original authors.
I want to afford anyone the benefit of an F/OSS license so they don’t have to waste their life energy recreating a wheel that I’ve already created. If it helps you and saves you time, have at it! In other cases, I contribute back patches so that companies can ship better software to their users (sometimes so I benefit; other times just so someone else doesn’t have to fix the same bug I just fixed).
This license case isn’t about “you can personally monetize it and nobody else” but rather “you and anyone else who wants to can try to monetize it”.
Right, I mean to say why would potential contributors want to afford those benefits to the original author so that only the original author can monetize it?
But which copyright owner? If the open source project has multiple contributors (not an unusual scenario in the slightest), it's not just a matter of one person's copyrights. Plus you get into the issue of the projects dependencies, and who owns those?
* requiring code to be assigned by the author to a single person/company, consolidating ownership
* building your entire base on friendly licensed software (3BSD) and ducking any GPL
So it’s very easy to say, for example, that the Mozilla Foundation owns all the code that makes up Firefox - they won’t accept your pull request otherwise.
you know that responding to comments tends to make a discussion flow, rather than stealing all our work for your own comment, incorporating them without credit? ;-)
Would it be fair to do something where you start off with the BSL approach where older code automatically becomes fully open source (like GPL or Apache), with the exceptions that the first N versions or N years of work will start off fully open, then the BSL stuff will kick in later?
I think this would be fair, and it has been done. Ghostscript had two branches, Aladdin Ghostscript and GNU Ghostscript, where GNU Ghostscript is approximately one year behind.
The idea behind dual licensing is that hosting providers would pay for a license and others wouldn't. The arguments that AGPL isn't strong enough are pretty speculative.
But, if the value-add is hosting more than the software itself, why would you prevent the 'value-adder' making the bulk of the money?
I'm like OP, trying to take a step back. What do we want here? As users? As developers? That no one does too much (or any) money hosting our software for other people willing to pay? Profit-sharing? On what basis?
Really, naively, apart from the use of the Elastic brand, that I might conceive could cause problems, who's hurting whom?
I want anyone to be able to pick up development of the software, not needing permission to do so. If elastic the business goes under, hosted solutions should still be available, with changes still be made on it
I'm thinking this would first be an API compatibility problem? The service can be implemented in different ways, by different teams. See redpanda vs kafka or wire-compatibility for postgres?
Is there a way to get the benefits of OSS AND for the creators to get more then zero dollars when someone makes a shitload of money off of just putting their software on an enterprise cloud?
> Is there a way to get the benefits of OSS AND for the creators to get more then zero dollars when someone makes a shitload of money off of just putting their software on an enterprise cloud?
You're basically asking if something can be simultaneously open and not open.
The benefits come precisely because someone can come along and create a successful business using your software without owing you anything. That is the reason for the benefits.
Licensing software accomplishes that, but you lose the freedoms OSS provides.
You also have to take into account that companies use OSS to gain adoption and pull off once they're established. This is why GPL and AGPL are essential. It ensure everyone has to contribute back.
From what I understand about trademark law, unlike copyright, trademarks must be defended at risk of losing them. So it may seem petty, but might actually have some necessity behind it.
IANAL, and certainly not an expert on trademark law.
Hopefully someone with more legal knowledge can provide some resources.
Even if it is just petty, it's petty against amazon, and I for one don't really feel the need to have sympathy for them.
Most people don't really get what trademark defense is -- having Amazon label the trademark in some places as "Elasticsearch is a trademark of <whoever>" is enough to count. Even linking the elasticsearch software repo or elastic.co in conjunction with the trademark name is enough.
Source: I have a best-selling book author friend who has been defending her trademark on a somewhat popular pop-culture term this way for a few decades.
That makes sense. I do sort of wish there were something to take away from this though that might lead to better outcomes for future endeavours, so that people could learn from Elastic's blunder, but if there's anything I don't see it.
I don’t think it’s petty as such. Amazon’s naming was likely to cause confusion. This is a good win for Elastic to protect their trademark.
Of course, Amazon has every legal and ethical right to continue providing their fork under a new name as they’re now doing. That’s the whole point of open source.
This is kind of unrelated but "Amazon OpenSearch Service (successor to Amazon Elasticsearch Service)" is such a verbose product name. If you mark it as one of your favorite services in the AWS Console, it will take up a large chunk of the navigation bar (diminishing its purpose).
There's a constant struggle between allowing free speech versus restricting it via trademark laws. In general, they want to make sure companies can talk about and mention their competitors by name and not be silenced by trademark lawsuits, while still ensuring the trademark isn't being infringed to the point of confusion. "Amazon Elasticsearch Service" sounds an awful lot like they have permission to use the Elasticsearch brand, while simply referring to it in parentheses indicates it's a competing service to the actual elasticsearch.
Full disclosure: This is my tool that I'm using to generate the insights.
When OpenSearch was announced, I shared some insights into how both Elasticsearch and OpenSearch were evolving, and I'll share some more up to date insights here.
Looking at recent pull request activity, OpenSearch had 52 contributors
The metric that I'm most interested in, is knowing how many people committed within the last 14 days compared to those that committed more than 14 days ago. For Elasticsearch, they had 87 contributors which accounts for 68% of all contributors. OpenSearch had 20, which accounts for 67%. With these numbers, I can ball park how many people are working on Elasticsearch and OpenSearch full time and I would say Elasticsearch at the present moment has probably 5 times more people working on it fulltime vs OpenSearch.
An important thing to note is, Amazon has other projects that are related to OpenSearch so these numbers don't necessary give the full picture, but it is pretty obvious that Elasticsearch is evolving at a much faster pace and time will tell if they (OpenSearch) can keep up.
What is interesting about these numbers, is they clearly show a lot of people (57 authors) took the time to create a small pull request, which goes to show how popular the project is.
I think part of the challenge looking at these metrics is the project versus repo structure. For instance, contributions to security are included in the elasticsearch repo but not in the core OpenSearch repo e.g. https://github.com/opensearch-project/security/graphs/contri...
But I still think having a bunch of folks contribute isn't trivial and worth highlighting (57 is a ton for many projects) and anytime I see a truly OSS project at this scale I think it's a good thing.
> I think part of the challenge looking at these metrics is the project versus repo structure
For my tool, I can analyze any combination of repositories so getting the bigger picture isn't challenging. Out of curiosity, I decided to index all the opensearch repos which will take a few hours to index and I'll update this post if I can or just reply to it when the other repos have been indexed.
Elastic is already a mature product, with little to improve. Even 8.0 release is focused on speed and footprint. Many companies still use 2.x happily. License implications are not worth it.
Having interacted with Elasticsearch a lot, I assure you there is a lot for them yet to improve. Frankly, I think Elasticsearch followed the MongoDB model of "make it easy for developers to adopt and then make it good." MDB did that by making it easy to throw "schema-less" documents into their database and then later working on "advanced" things like avoiding data loss [0]. Elasticsearch did something similar in that you don't have to run a separate cluster coordination server like Zookeeper which Solr requires. It's also easy to get up and running with indexing unstructured data. Running it at scale with heavy workloads? There are still plenty of rough edges there. It's a good way to gain market share and both products have definitely improved, but I wouldn't want my day job to involve the operation of a large Elasticsearch cluster.
Indeed, there is a lot of room to improve. I managed a cluster that grew from 9 to 40 then to 60 EC2 Servers over 2 years, with all updates and support we could afford.
It was a a nightmare to manage. The amount of cluster rebalancing, magic nodes that claim to be leader and then do not behave like it is amazing. I don't want ever to get back managing it again. Last thing I did (ES7 at the time) was to split the cluster in many smaller clusters, reimplement all counters on relational databases and use as little of ES as possible. As a former colleague used to say: Elasticsearch is MongoDB for people ashamed of using MongoDB. Hope AWS and Elastic tension helps them to do a better job there.
I believe Amazon does most of their current feature development in separate repositories for their plugins whereas Elastic has their plugins (x-pack) merged into their core repository. Would be interesting to see that taken into account too (along with the actual lines committed, i.e. 10 commits vs 10 commits but one has 10x the code).
I'll be curious to see what happens as well. I ended up going with Opensearch since I use AWS. I figured I'm already locked in to that environment anyways, so why not
Somebody that does consulting work with Elasticsearch mentioned that they had some pretty innovative features in the pipeline and these are the things that can really pull people away from OpenSearch. It could also be that, people only really want basic search functionality so who knows right :-)
I’m increasingly pessimistic on ES as a solution. Everyone is coming to eat their lunch. APM? Datadog. Product search? Algolia.
I think the next wave of search tech will be
1. Leveraging automated ingestion optimization.
2. Capable of searching across mixed media: text, audio, images to start out with. This is only possible because of optimized ingestion. The ingestion layer will perform vectorization of the inputs. The vectors will be the entires stored in inverted indices.
3. A great jump from two to an optimized data store for searching mixed media.
People obsessing over elastic/splunk features are lost. Nobody will care in 5 years. Both will be legacy enterprise tech.
i think that for all the people which just want a logging system in their k8s, opensearch is pretty good.
lots of software ist stable and just works. everyone uses bash. let the elastic guys make their new software more shiny, good for them. i am now glad i can use a real opensource version at home and for small to midsized projects. if opensearch won't cut it? no problem, the customer can pay elastic :)
Thanks for gathering some data here. Very insightful and it confirms some suspicions I've had. I'm one of the people that is no longer creating pull requests for Elastic. I was never a big contributor and I only did a handful of small PRs. But all little bits matter and I was OK with doing that under the Apache 2.0 license and putting my time in and doing my small part. That's not a thing for me anymore. I charge money for commercial software development.
In practical terms, I now have to worry about supporting both Elastic and Opensearch as a consultant (always looking for new customers to help, if you need some advice) and I can't really recommend people stick with Elastic with a straight face either because Opensearch really is a decent alternative and increasingly the default for new users; as I have noticed with my own customers.
I maintain a Kotlin client for Elasticsearch and I have started work on making that work for both Elasticsearch 7, 8 and Opensearch; something that is complicated by the fact that Elastic intentionally made sure their Java client (which I depend on) no longer talks to opensearch. They also deprecated it and introduced a completely new one recently. And of course the recently released Elasticsearch 8 complicates things with a few new features, compatibility breaking changes, etc.
On top of that, Opensearch is starting to get its own features or alternatives to Elastic-only features. I agree that the momentum is still with Elastic but it is not the case that Opensearch is a dead fork. I know of several people that quit their jobs at Elastic because of the license change and that are now working on Opensearch or on Opensearch plugins. It will be interesting to see how the two forks evolve.
IMHO, the license change is long term a mistake for Elastic. They've cut themselves off from open source contributors who can still contribute to making Opensearch, Lucene, or Solr better. So, that's long term not going to be helpful. Elastic has always leaned on opensource heavily for contributions and hiring. A lot of their early hires came from the Lucene community and the many researchers contributing to that and later from people contributing to Elasticsearch. Additionally, a lot of value gets added to the ecosystem by OSS plugins. The authors of those now have to choose between Opensearch and Elastic. That's long term not good for Elastic as a lot of cutting edge stuff usually starts in the plugin community.
I don't necessarily like what Amazon did but I do think they did it the right way from the point of view of creating a credible alternative. And I think that the way Elastic responded to that is throwing out the baby out with the bathwater.
Perhaps the most striking statistic is that the total number of PRs for both combined declined: everybody loses. IMHO, they should just roll back the license change and grow their community rather than fragmenting and shrinking it. Amazon is going to get some of their customers either way. I actually think the fork is working out better for Amazon than it is for Elastic currently. It was a mistake. Amazon is not the most likely steward of open source. But stranger things have happened. Take MS in recent years for example. It's a sound business strategy to not piss off OSS developers. Elastic would do well to take note of that.
It seems like the contribution for Elasticsearch hasn't drastically changed that much (if any really), but what is interesting is contributions for OpenSearch has been increasing.
It's written in asynchronous Rust with native application speed albeit using much lower memory usage than ElasticSearch, and comes with even more features than ES, so it's feature-rich, blazing fast and can still benefit on multithreaded CPUs. Downside is that it does not have distributed indexing mode yet, but it is scheduled on this year (presumably Q4 2022 I guess)
I'm sure it is fine and has interesting features. But at first glance it's objectively a tiny subset of the features that either Elasticsearch or Opensearch has. If that solves your problem; great. But if you need a good all round solution for search I would not start with this.
The low end of the market is well covered with solutions that don't really offer a lot of features, are a bit challenged on the scalability front, lack usability, etc. Some of those have more merit than others of course than others. Things are not automatically better when they are half implemented in Rust. Lucene is an amazing piece of technology that over the years has resisted multiple attempts for people to do better in other languages. Contrary to the popular belief, it's actually pretty good with memory. Most of it is off heap memory or operating system file caches these days: it relies on memory mapped files for a lot of things. It's also pretty good with doing things concurrently. E.g. updating index files with 32 CPU cores while also serving queries is not an easy problem to solve. I've indexed documents at a rate of 500M/second on a 30 node cluster once. That's pretty amazing to see happening. I was basically saturating IO and CPUs. Indexed over a billion documents in about 1 hour.
Lucene has many issues; scaling isn't one of them.
This announcement is confusing, because it makes no mention of OpenSearch. The announcement implies that OpenSearch should no longer be available on AWS, but (of course) it is.
Honesty sounds like AWS is giving up on providing hosted elastic search. I hated using it, it always was failing and I spent more time administering it than anything else.
I recently used Elastic Cloud, and as much as I hate the company, their product is actually really good. I’d always recommend Elastic.
> Honesty sounds like AWS is giving up on providing hosted elastic search.
Yes, Elastic’s press release is carefully crafted to give that impression, but, AFAICT, all AWS is doing is using the name of the open fork (“OpenSearch” [0]) for their service, which is now labelled “Amazon OpenSearch Service” and subheaded “(successor to Amazon Elasticsearch Service)” [1], and not using the ElasticSearch name (except as a historical reference.)
Reminder that Amazon is the good guy and Elastic is the bad guy here. Elastic changed their license from a FOSS one to a proprietary one, and Amazon's fork is a FOSS continuation of the last version before the change. Yes, Amazon is evil in a lot of ways, but there's nothing even remotely evil about what they did here. And yes, Elastic had a legitimate problem that their old license was open to abuse, but the right way to fix that is the AGPL, not going full proprietary.
AWS has gobs of open source software. I think they open source everything that runs client side (all the agents and integrations for EC2 for instance).
It's pretty disingenuous to say they don't open source.
Based off experience with AWS and GCP, AWS has significantly more widely available. On the other hand, Google the company has contributed a lot to OSS (like cgroups in the Linux kernel)
I’ve been tossing up moving our workloads to Elastic Cloud anyway, because AWS ES Service is a source of constant headaches for us. Feels like at least once a week a server ends up in a state where we can’t fix it, and AWS engineers have to manually fix their internal state.
Their standard response is “add more nodes”; well, we did that, and it is costing us an arm and a leg, and it didn’t fix the problems. (Plus, now we have new problems where networking blips appear to be causing quorum problems and sending the cluster into a death spiral.)
Purely off the customer experience, it feels like Elastic Cloud has to be better; the whole licensing debacle has definitely turned me off Elastic though.