Read the ZeroMQ manual. That's the most important part of ZMQ, as it's the clearest and best guide to your options for partitioning a task and arranging communication between software processes.
After that, you can implement your stuff in 0MQ, AMQP, nanomsg, or whatever else. I fully expect nanomsg and 0MQ to compete with each other on Github over the next 5 years or so, and I personally won't take sides.
For me, I use the manual to design programs I then write in Rust, a lot of it with no explicit message passing at all. (REQ-REP and PUSH PULL become simple function calls, sometimes mediated through thread spawning, and sometimes not, for example.)
Hmm, I'm not sure I understand exactly what you're saying.
The 0MQ manual is (indeed) excellent, but it seems like a design whitepaper. It essentially describes what are known as the scalability protocols. As you say, it's independent of the implementation, i.e.: the library.
My confusion largely stems from a presentation I watched [0] in which it was implied that 0MQ suffered from a few design issues, and had therefore been forked. It seems like the library was rewritten in the meantime, however, so perhaps these points are no longer relevant.
I suppose I'm looking for a bit of clarity as to why the fork occurred in the first place.
``For ZeroMQ, our stated mission was "Fastest. Messaging. Ever." This is a nice, and nearly impossible answer to a problem we could all agree on: namely, the slow, bloated technology available at that time. However, my co-founder Martin and I had conflicting goals. He wanted to build the best software possible, while I wanted to build the largest community possible. As the user base grew, his dramatic changes, which broke existing applications, caused increasing pain.
``In that case, we were able to make everyone happy (Martin went off to build a new library called "Nano").''
Martin Sustrik split because of differences, did a completely new implementation called nanomsg. He is no longer maintaining nanomsg. That task has fallen to someone else, but it doesn't seem to be too active at the moment.
Right, but my point was rather about the differences you mention. I was under the impression that 0MQ was superceded by nanomsg for technical reasons (UNIX-iness, 0MQ's threading model, etc).
Whence comes the impression that nanomsg is inactive?
Yes you are confused. The 0MQ[0] have never been superceded by nanomsg. The deal as you are pointing out very clearly is that the two projects are alive and well.
On a related note, using jupyter[1], salt[2], circus[3] and building fruits have been a real pleasure too.
And lastly, a bit of info on how 0MQ and nanomsg differ[4].
I've been seeing the occasional post from his blog about the choice to die but didn't connect them to ZMQ until I saw this.
ZMQ, as a library, is a work of art and Code Connected is by and far one of the best programming books I have had the pleasure to read. That, coupled with the deep and interesting posts on his blog show we have lost a truly great mind.
If you're just learning about ZeroMQ (or just reading this post) - I suggest taking some time to read or listen to some of Pieter Hintjens blogs/book/talks.
It's a wonderful experience and he seemed like a great/genuine dude.
How would you review PC3 versus C4? It seems like C4 (https://rfc.zeromq.org/spec:42/C4) is the latest evolution of this design. It seems to embrace optimistic merging even more than PC3 and does away with reviewers.
That's a good point. I was basing it on dates I could find and what I've been reading in Social Architecture. Perhaps the latest C4 is later and PC3 is smack in the middle
I haven't used zeromq in forever, but did they ever fix the problem with request/reply sockets where the server socket could get into an indeterminate state after a client socket drops at just the wrong time?
Nope. The reality is that ZeroMQ is useful for a variety of tasks but doesn't really excel at the tasks for its specific socket types anymore. He offers a heart-beating pattern to get around this issue for Req/Rep sockets though.
For pub/sub Aeron is now much better (way more throughput and doesn't crash at multi-gigabit rates like OpenPGM). For REQ/REP HTTP/2 and other QUIC-based approaches are reigning supreme (if you need high performance across a WAN then you can repurpose something like FIXT 1.1 from the FIX protocol).
Looks like socket heartbeating has been added in this release of ZMQ. From what I can gather from the docs this should address the issue the parent post presents, but does anyone know definitively? See new ZMQ_HEARTBEAT_* options here [0] and Connection Heartbeating section here [1].
> For REQ/REP HTTP/2 and other QUIC-based approaches are reigning supreme
Oh? I implemented something recently using req/rep using pyzmq and then ported it to grpc. grpc was an order of magnitude slower. Then I updated the zeromq code to do pipelining via router/dealer and that was even faster.. by sending pipelined batches of 100 items it can do 160k lookups/second. grpc+batching I think maxed out around 20k.
Could have been protobuf that was the cause of the performance hit though.
gRPC is and certainly will never be the fastest protocol for small request/reply messages. The reason is the stream multiplexing layer that is required for it. You almost certainly need to copy data from the connections receive buffer into a streams receive buffer into the application and the opposite for the sending side.
If you don't have the stream multiplexing and just write complete request or response packets to a connection (similar to Thrift) you save quite a lot of overhead.
However this multiplexing feature is also the biggest upside and achievement of gRPC, since it enables you to stream big requests or responses and not only small packets. And it enables multiple big streams (file uploads, etc.) in parallel over a single connection without one blocking another. And of course it enables flow-controlled bidirectional streaming IPC, which can not be found in other systems.
Well the underlying thing I am doing is small request/reply messages - I'm doing metadata lookup for ip addresses. The way I sped things up with zeromq was first by batching requests. Essentially, if I have 10k lookups to do, instead of sending 1 at a time, I group them into blocks of 100 and send
' '.join(block)
Then I do all the lookups on the server and send a block of responses back. This turns what would be 10k queries into only 100 rpc calls.
That got me to about 60k lookups a second locally, but over a wan link that dropped down to 10k. I fixed that by implementing pipelining using a method similar to the described under http://zguide.zeromq.org/page%3Aall#Transferring-Files where I keep the socket buffers busy by having 10 chunks in flight all the time.
That got things to 160k/s locally and 100k+/sec even over a slow link.
I'll have to mess with grpc a bit more. Looking at my grpc branch it looks like I tried using the request_iterator method first, then I tried a regular function that used batching, but I didn't try using request_iterator with batching. I think the biggest difference would be if request_iterator uses a pipeline, or if it still only does one req/reply behind the scenes.
Yeah.. I figured as much.. zeromq in python is not slow though :-)
I could probably port the service to c++ or go, it's really just some string parsing and a hash table lookup of sorts.. but when my PoC python version does 160k lookups a second, I don't feel the need to spend the time :-)
"On python" can mean a few different things. It can mean a straight port, running in the python interpreter, or it can mean Cython (or similar) with all of the tight loops running as auto-generated compiled C code.
Numpy is a great example of this; all of the numerical operations are running on very fast compiled code, and being good at writing fast numpy involves knowing the ins and outs of how to minimize passing information between the slow python interpreter and the fast numerical engines. You want to just do all of the computation 'inside' of numpy, and then get the result at the end.
Yeah, I'm not sure how optimized the python protocol buffer stuff is. Years ago I benchmarked the pure python protobuf lib and it was terribly slow.
grpc was nice to work with though. I generated the stubs and stuck my logic in there and had a working client/server in about 20 minutes. The streaming request/reply stuff was crazy easy to use, though I don't know if it does pipelining.
On a related note, using nnpy [1] and mangos [0] have been a real pleasure.
And lastly, a bit of info on how 0MQ and nanomsg differ [3].
[0] http://nanomsg.org/ [1] https://github.com/nanomsg/nnpy [2] https://github.com/go-mangos/mangos [3] http://nanomsg.org/documentation-zeromq.html