There’s an assumption that an open source project should “belong” to a single company (Mongo, Elastic, Confluent, etc). I think that model may be more dangerous to OSS than large cloud providers.
If it’s truly open source, one business shouldn’t dominate the project’s development. That’s tantamount to the project becoming shared source that accepts community contributions, not community driven open source backed by many orgs. It reinforces a monopoly in that community. And these companies themselves are not tiny victims (many IPOd at multi billions). They also tend to compete with their customers and community in the same way as AWS, so I’m not sure they can cry foul when AWS does the same.
It's hard to drive non-incremental innovation without a benevolent dictator model.
For example, the Hadoop ecosystem moves very slowly, and most of the innovation happens in new projects that are added, not in core infrastructure. An example of moving slowly is the fact that even to this day, the HDFS high availability story for NameNodes is pretty wonky.
In a community model, if someone wants to do a large scale refactoring or rework parts of the architecture, these projects die of a thousand cuts with objections or change requests from many stakeholders. The decision-making then becomes "design by committee," instead of pushing towards a long term vision.
With a single benevolent dictator, since decision making is more centralized, it's significantly easier to do large reworks of the architecture to improve the product in non-incremental ways.
> It's hard to drive non-incremental innovation without a benevolent dictator model.
Hmm, dunno that this is a universal truth. Most of the biggest improvements to Django came after Adrian and I stepped down and we moved to a more democratic leadership structure.
The key word is “improvement”, which implies that there was something already there to improve.
So, a democratic structure can improve Django once it’s already well-defined and widely used. But could it have created Django and made it successful in the first place? I think the answer is no, and that’s what the gp meant by “non-incremental innovation”.
Most of these projects reached that stage before or early in the life of these companies. So by that logic the companies were not necessary to support the projects.
I think “bdfl” and “exclusive corporate sponsor” are 2 distinct issues. I was addressing “bdfl vs. democratic leadership”. I personally don’t think it matters where the bdfl is employed in the early days of a project, there are examples of wildly successful projects with and without an exclusive corporate sponsor.
Good point, they are probably indeed distinct. In some cases the BDFL is employed by the corporate sponsor (eg. Confluent) which makes it hard to separate the influence of each.
I wonder if programming languages are different from infrastructure components, though. Programming languages tend to be harder to sell, while infrastructure components can be resold by providing them as a service. Think of Python as a service vs Jupyter as a service.
I was wondering this too. Though it used to be you could monetize programming languages more readily. I wonder as databases approach a commoditization point if the DB market becomes more like the programming language market.
Right now turnkey hosting is the main way of making money. But if open source push button distributed database in an arbitrary cloud happened (via easier containerized orchestration than currently exists) we wouldn’t really have much of a hosted database market. Then database open source might start to look more like programming languages and less like money making entities unto themselves. Honestly this is probably the bigger threat to the Mongos, Confluents etc than anything else. And why AWS pushed proprietary tech like lambda.
Distributed databases will always require some kind of specialist knowledge to maintain. The physical deployment of code is actually the easiest part, the big problem is how to keep things running and fix issues once things invariably start breaking.
The value proposition for hosted services is "we'll take care of everything, you don't have to worry about operations" while the value proposition for the development companies is "we'll help you run things and we'll reinvest the proceeds in maintaining the platform."
The incentive for AWS is to extract maximum profit from open source projects (with no real regards for sustainability), while the incentive for the development companies are more aligned with having a sustainable long term community and product.
You should check out hopsfs. It solves the namenode problem using distributed metadata and stateless namenodes. Kind of validates your point. It's a fork.
> If it’s truly open source, one business shouldn’t dominate the project’s development. That’s tantamount to the project becoming shared source that accepts community contributions.
Open Source has never meant anything of the sort. You always have a tyranny of the maintainer. Whether that's because it's one guy's hobby project and he's not interested in your patch because he doesn't personally need it, or because it's some giant corporation and it doesn't align with their interest, or anything in between.
In either case, Open Source has never been about other people being obligated to do work for you, which is what accepting a patch amounts to. Most contributors don't stick around, so integrating something means carrying technical debt going forward.
It's about you being able to say the maintainer is wrong, and invest your own time in maintaining a fork or a local patch. Anything beyond that (like upstream accepting your patch) is just gravy.
You’re right, I was overzealous with that statement.
I think my opinion would be the “free” in FOSS is not supported when a business monopolizes a community. That building a company to effectively control the project is at odds with a healthy open source community.
I hate defending Amazon. But if we think of Amazon as “in the Mongo community” then what Mongo, etc are doing is picking winners and losers in their community. That gets ugly, and creates a lot of incentives in that community for everyone to say “all hail Mongo Inc”. Effectively the community around the project becomes indistinguishable from the user community around a proprietary software. Everyone takes from the mothership, few give back, nobody questions or debates centralized decisions made, learned helplessness ensues...
I personally feel this is very counter to what FOSS is and should be. And I dont think it’s in the long term interests of the project, community and user base around the project
I don't necessarily agree with Rich Hickey, but he put a contrary view very clearly and elegantly in a recent post:
> Open source is a licensing and delivery mechanism, period. It means you get the source for software and the right to use and modify it. All social impositions associated with it, including the idea of 'community-driven-development' are part of a recently-invented mythology with little basis in how things actually work, a mythology that embodies, cult-like, both a lack of support for diversity in the ways things can work and a pervasive sense of communal entitlement.
That's because all of the open source definitions you've seen have attempted to be prescriptive rather than descriptive or idealist. OP is simply using a different kind of ontology regarding what open source is.
If it’s truly open source, one business shouldn’t dominate the project’s development. That’s tantamount to the project becoming shared source that accepts community contributions, not community driven open source backed by many orgs. It reinforces a monopoly in that community. And these companies themselves are not tiny victims (many IPOd at multi billions). They also tend to compete with their customers and community in the same way as AWS, so I’m not sure they can cry foul when AWS does the same.