Service Fabric is different from Kubernetes. It's not a container orchestration platform but a full-stack distributed state and processing system with components that can serve as the foundation for building something like Kubernetes itself.
The naming is confusing and there is a Service Mesh offering from Azure that lets you run your own applications easily if they're using the Service Fabric framework already, but SF can run containerized code in any language, and it has bindings for C# and Java (for now) if you want to use the primitives directly.
SF has been opensourced very late though [1], to get some attention and trying to survive. Legitimate but very different from being an open source project. The cross-platform support is very weak [2][3], far from K8S, Mesos and Swarm developer friendliness.
Trying to survive? It already runs all of Microsoft's cloud services and does just fine. Open-sourcing was more of a good-will move so that the community can also use it, similar to how Apple released FoundationDB.
The links you point to are about developing with SF, which doesn't natively run on Mac OSX. It can run on Linux but has fewer features for now, but this is being worked on. It's not the same as K8S/Mesos/Swarm though so if container orchestration is all you need then you shouldn't use SF.
MS is going where developers are, and outside MS there is no community around SF. Hired one of K8S founders, built new products around k8s, advocates docker left and right at conferences, these are clear signals that the tech doesn’t have a future
SF can also manage docker containers. They are doing lots of work with k8s, but this system has a huge amount of internal usage and has provably been used to build global scale services. Its strengths are different than k8s and the distributed system space has lots of options.
You (and others) seem to be enjoying some schadenfruede about SF not "winning" some competition. I don't think the world works that way. Lots of people will use it for big applications. More people will probably use something else. That's about it.
The type of comment you made doesn't get marked by the moderators for some reason even though it doesn't actually provide any insight. The article had detailed information about the internals of how it worked. That's the interesting thing here. The Ziff Davis style dev-adoption-horse race is pretty pointless.
I have to say, I'm evaluating some of the .net technologies for building distributed systems, namely Akka.NET, Orleans and SF, and using any of them requires quite a lot of investment and therefore time to learn to use them right.
The last thing I want to experience is that after a year or two, the tech I pick is going to be abandoned or at least put into maintenance mode.
Therefore I'd appreciate all insights (if possible data-driven) on the future plans about SF and how does it stack against k8s investments by MS. It would be great if MS themselves made an official announcement.
Seeing that SF doesn build on Windows for several months, makes me doubt whether the momentum behind open sourcing SF is real.
Theres a good presentation from the Build conference about the future.
In summary they will add the hosted “mesh” variant. And move the SDK towards polyglot libs for use in docker containers instead of a C# framework. The reliable collections part of the SDK will be just a lib providing storage for abitrarily shaped data needing replication.
They also mean to hav it all work similarity for your own clusters on-perm or on azure.
(And some one asked, yes managed service principles and secrets is coming up in 6.3 probably)
P.s There is also a regular q/a session over Skype they record and put on YouTube if you want more details or ask questions directly
(I'm Matt, from SF@MSFT, the same guy from up here [1])
So about those presentations and announcements! This[2] is the main presentation where we gave a bunch of updates and roadmap, including the upcoming changes to the SDK and the announcement of Service Fabric Mesh. Here's a short video that shows a demo of what that's really like[3].
So about the new programming frameworks and the changes/history here: You have a lot of choices for frameworks. Roll your own, Spring/Steeltoe, Orleans, Akka.NET, and the ones that SF provides, etc. For most of those though, they are just frameworks. They help you develop services, but don't have as much to do with how or where you run them.
...Except for the ones that came with SF, which were of course special in an unexpected and not entirely desirable way.
Previously the SF provided frameworks (Reliable Services and Reliable Actors) were different. They were coupled to the Service Fabric runtime environment. You had to run them in a SF cluster, you couldn't run them anywhere else.
And managing a cluster is hard work. You had to manage all the infrastructure, figure out the networking, figure out how you wanted to configure the drives and storage, manage certificates, all that stuff. Managing a cluster was real work and most of that work contributed basically 0 to how well the service actually worked. You had to patch and secure the OS. Some of this stuff was easier in Azure than doing it all on your own for sure. But easier still isn't easy in a lot of situations, and it's all boilerplate.
Just as an example, since you managed the hardware, at some level, you had to deal with scaling decisions yourself. SF could help you scale within the cluster, but because SF tries really hard not to get too wedded to a particular infrastructure, when you ran out of hardware, you were on your own to figure out how to add more.
With the new SF SDKs and roadmap a lot of this is changing. Particularly for the stuff SF provides out of the box like service discovery and routing, lifecycle management, and the reliable collections, we've separated these things out from the runtime. This means one of the following a) you just don't have to deal with them anymore b) we do the common thing automatically for you, or c) they're optional libraries you can include and that work anywhere you choose to run the service, even outside of SF.
As part of this, we're also introducing a bunch of different resources like routes and volumes that take a bunch of stuff that was previously much more implicit and making it explicit and declarative. The goal of the new SDK and resource model is that they're really going to help more types of workloads run on top of SF, especially ones that don't want to (or can't) use our SDK, while also making things easier to manage overall.
The goal of Service Fabric Mesh is to be an environment where Microsoft manages the cluster for you. This takes all that infrastructure goop and hides it, giving you a serverless, fully managed environment where you can run the stuff defined in that resources model. So now you get the power of the platform and some of the neat features that SF provides, and you don't have to do all that boilerplate work. This lets you focus on whatever your app/service was. And since the resources model is a core part of the product, you'll get a fairly seamless transition between building things on your dev box, running them in say some local test cluster, and running them up in Mesh in Azure.
I think it's really exciting and should help with adoption of SF since now a lot of the stuff that people had problems with is gone, and since the resources model and the changes to the APIs make it much easier to run arbitrary workloads than before.
Thanks. Is Service Fabric Mesh going to be easy to roll out without Azure? Are you going to provide a set of tools to set the cluster up and operate it, for those who need a self hosted version?
Another thing - how is the Windows build tools for SF going? Is it easy to build right now from the open source version?
Mesh is specifically the serverless environment that MS runs for you in Azure, so there's no real concept today of running Mesh somewhere else (probably except for Azure Stack) - you have to manage the servers then, so it's not serverless anymore :)
But the thing to keep in mind is that all the capabilities that you're thinking about are in the core product. So you could take SF and run it standalone and tell some people "hey you run the cluster" and another set "hey you write the services" and at least with the resource based model you have a pretty good separation of concerns. It's not Mesh because it's not serverless with MS running the cluster for you, but you can do almost all the same stuff.
Re: the Windows build, yeah what Mikkel said. Slow and steady.
(I'm Matt, from SF@MSFT, the same guy from up here [1])
To me, it makes sense that the Kubernetes services at Microsoft would be built on top of Kubernetes and expose Kubernetes to customers. That's a good way to get experience with running a service at scale, eat the dogfood, and deliver the experience people who want Kubernetes are expecting.
Just because some services pick K8s as the backend (when that's an appropriate choice!) and that Microsoft hired an expert to work on them is in no way proof that a different competing technology is dead. Plus: would you have rather we hired an amateur? :) I'm glad Brendan's here leading the charge.
The notion that promoting Docker indicates that SF doesn't have a future is also misleading - that's a container format vs. an orchestrator. Those are different layers of the stack with different jobs. Since SF has supported Docker containers for some time, we're fine with the rest of the company promoting Docker as a means of isolating workloads. Plus, before containers, isolating workloads on Windows was really tough. SF arguably made this worse by giving everyone an environment where you can run tons of different workloads SxS without giving people guidance on how to isolate things reasonably. SQL virtually had to invent their own way of doing it[2], and nobody really likes programming against Windows Job Objects (just like most people wouldn't prefer to deal with cgroups directly either).
Docker (or other) containers are a godsend for taking old leaky code and getting it running in a shared cloud environment without having to rewrite stuff. SF <3 Containers, but we also work with bare processes because, well, like Mesos, that's how we started, and also sometimes you just don't have the problems that containers solve or can't take the hit to switch to a containerized workflow at the moment.
Lots of new services get built on top of SF all the time (The video from build[3] shows the newer services built on top of SF in Azure). So in the same way, just because there are new services getting built on top of SF doesn't mean that the Kubernetes offerings at MSFT are dead either. It's a false choice.
The naming is confusing and there is a Service Mesh offering from Azure that lets you run your own applications easily if they're using the Service Fabric framework already, but SF can run containerized code in any language, and it has bindings for C# and Java (for now) if you want to use the primitives directly.
SF is also now open-source and you can run it all yourself in whatever environment you want: https://github.com/Microsoft/service-fabric