A few years ago, I worked on a project where we wanted to use Event-Driven architecture. But most team members were unfamiliar with it and uncomfortable working with asynchronous architecture. I asked myself: "Is it possible to make building an Event-Driven application as simple as building an HTTP API?" — and that's how Watermill was born.
We currently support 12 Pub/Subs, including Kafka, Redis, AMQP, and NATS. We also support MySQL and PostgreSQL-based Pub/Subs, so you can use Watermill even if you don't want to add extra infrastructure to your tech stack.
We also support poison queues and the outbox pattern out of the box and provide middleware that supports logging, retries, and circuit breaking.
Watermill is an independent project under the MIT license. We don't plan to get VC funding, and nobody from outside can force us to change the license to monetize the project.
We released Watermill v1.0 more than five years ago, and since then, we have kept backward compatibility.
We plan to release v1.4 soon, including updated poison queue support and sending delayed messages.
I'd love to hear your thoughts and feedback. If you have any questions, I'm happy to answer!
SlashID | Principal Backend Engineer | REMOTE (EU compatible time-zones) | Full Time
We’re an early stage startup building an identity platform. Our goal? Bring authentication for users, machines and APIs under the same (secure) umbrella (aka one identity platform to rule them all).
We’re looking for people who can design, build, and maintain distributed systems at scale. You’re also driven, product-oriented and enjoy getting your hands dirty; extra points for experience in security.
Stack: Go, GCP, Redis, PostgreSQL, Docker, Terraform. You don't need experience with all these but should be willing to learn).
With my friend Miłosz, we've been working on a way for experienced developers to grasp Go quickly. After long evenings and weekends, we're ready to share our idea that we think is unique.
Many Go courses cover lots of details irrelevant to most developers. The difficulty level is chosen for developers with all levels of experience. Often, they consist of videos, which makes them long. Or they make you write code in the browser. We believe that it’s not the most optimal way of learning. We decided to approach this topic differently.
While learning and teaching new technologies, we found that implementing real-life projects incrementally is the best way to learn. We decided to recreate this method of learning, but in a more structured and automated way.
We created a Go training for experienced developers, containing the most relevant knowledge. It quickly makes you comfortable writing Go code.
The training is made of 80 exercises based on real-life projects and examples. You write all the code in your local editor or IDE. Our CLI provides the boilerplate code for the exercise and fast feedback about your solution.
The first four modules are free after registration. The entire training is free for students (please check the FAQ[1] for details).
We are not aware of any similar training out there. It would be great to hear your feedback!
With my friend Miłosz, we've been working on a way for experienced developers to grasp Go quickly. After long evenings and weekends, we're ready to share our idea that we think is unique.
Many Go courses cover lots of details irrelevant to most developers. The difficulty level is chosen for developers with all levels of experience. Often, they consist of videos, which makes them long. Or they make you write code in the browser. We believe that it’s not the most optimal way of learning. We decided to approach this topic differently.
While learning and teaching new technologies, we found that implementing real-life projects incrementally is the best way to learn. We decided to recreate this method of learning, but in a more structured and automated way.
We created a Go training for experienced developers, containing the most relevant knowledge. It quickly makes you comfortable writing Go code.
The training is made of 80 exercises based on real-life projects and examples. You write all the code in your local editor or IDE. Our CLI provides the boilerplate code for the exercise and fast feedback about your solution.
The first four modules are free after registration. The entire training is free for students (please check the FAQ[1] for details).
We are not aware of any similar training out there. It would be great to hear your feedback!
If new hardware or technology is totally changing your solution - it's a bad sign.
It depends a lot on the domain on which you are working on. In places where I worked, such big changes could be encapsulated and separated from the domain logic (by using Clean Architecture, for example).
This is where modelling techniques are useful - with proper exploration you can create proper boundaries that will save you from such big changes. The only thing that is constant is change. It's all about being prepared for that.
Take for example hard drives. A filesystem optimized for a spin drive will take into account that the needle has to move a physical distance and arrange files to minimize the movement of the needle.
All that goes out the door when you have an ssd which has a O(1) access to any part of the dardrive
You are right - I'm using it as as shorthand. It's referring to the "typical" monolith architecture perception.
I'm actually a big fan of modular, properly implemented monoliths. In the first blog article I was even showing that it can be actually an implementation detail if an application is monolith or microservice: https://threedots.tech/post/microservices-or-monolith-its-de...
The good news is that Go have very little entry point. Thanks to that you can hire people, who are working in other technologies and are interested in learning something new. From my experience they can be productive within a week. It's totally not possible with Java I guess ;-)
It's probably a bigger problem to justify it on the company level. It's always some risk to go with a language that doesn't have such good position on the market ("Who will maintain it?", "Where you will find people who will fix it?", "Wouldn't this language disappear in 2 years?").
Fortunately, a lot changed in recent years. Thanks to Kubernetes, Prometheus, Docker and all other infrastructure Go is already used in the most of the companies. That's giving people much more confidence about Go.
You're missing one key question in your list: "How much (time/money) will it cost and what will be the gain?" Two key aspects to that question: whether/when/how to re-write existing systems in Go and how re-written or new implementations integrate with the existing ones. That's probably the most important question, actually.
> You're missing one key question in your list: "How much (time/money) will it cost and what will be the gain?"
I guess it depends on the situation of the company. But I guess that before doing such movement, company should do a pilot to verify if introducing Go can solve currently existing issues (with development velocity, bugs, performance etc.). After that answer should be simpler :)
> Two key aspects to that question: whether/when/how to re-write existing systems in Go and how re-written or new implementations integrate with the existing ones.
It also really depends, but from my experience companies that were switching to Go were keeping legacy part and people who were able to maintain it. In the meantime they re-written what was worth to be re-written. Without touching old part too much.
In that case yo can stay with a situation where you have some developers of old technology and some of the new one. But AFAIK it was not a major issue.
> While protocols in Go are great (one of the very many breaths of fresh air that Go provided), operations on data structures without generics does not seem like a good time. Does everyone get around the lack of generics by making most generic-looking operations slightly-specialized structs and some interface composition?
When it comes to domain code - probably when you are starting to think about generics that's a bad sign. Probably you are trying to make it overcomplicated. Domain code should stay simple and interfaces should be enough to handle it.
But when it comes to libraries the situation is totally different. One of the example is https://watermill.io library (that we are authors of BTW). Generics could give some nice simplifications.
It's also a case for event-sourcing library that we are using in the company where we are working on now. But You can still do a lot with interface{} and reflection. But it's not perfect and generate a lot of boilerplate.
The third example is a decorator pattern. You can't create generic decorator without code generation. It is of course some sort of the solution, but it's not trivial to implement.
A few years ago, I worked on a project where we wanted to use Event-Driven architecture. But most team members were unfamiliar with it and uncomfortable working with asynchronous architecture. I asked myself: "Is it possible to make building an Event-Driven application as simple as building an HTTP API?" — and that's how Watermill was born.
We currently support 12 Pub/Subs, including Kafka, Redis, AMQP, and NATS. We also support MySQL and PostgreSQL-based Pub/Subs, so you can use Watermill even if you don't want to add extra infrastructure to your tech stack.
It's handy for: 1. Building event-driven services 2. Implementing CQRS architecture 3. Creating complex message processing pipelines
We also support poison queues and the outbox pattern out of the box and provide middleware that supports logging, retries, and circuit breaking.
Watermill is an independent project under the MIT license. We don't plan to get VC funding, and nobody from outside can force us to change the license to monetize the project.
We released Watermill v1.0 more than five years ago, and since then, we have kept backward compatibility.
We plan to release v1.4 soon, including updated poison queue support and sending delayed messages.
I'd love to hear your thoughts and feedback. If you have any questions, I'm happy to answer!
Links: GitHub: https://github.com/ThreeDotsLabs/watermill Documentation: https://watermill.io/ Examples: https://github.com/ThreeDotsLabs/watermill/tree/master/_exam...