Hacker News new | past | comments | ask | show | jobs | submit login

My experience is across several years writing high performance http clients, writing in pure C.



This changed recently so both of you are right in different ways.


Do you have a link to either an announcement of this change, or C code that demonstrates it? If it has changed, I might start doing some things differently...



I dealt with all of this in 2015, and spent three months trying to get epoll to perform similarly to kqueue; namely, to avoid returning on any and every packet received and, preferably, to abide by some sort of minimum bytes received (or connection closing) before returning.

Unless I'm missing something (and I could very well be missing something), I'm not seeing how that is any different than anything I tried...and I went through every socket option I could find, even if it seemed irrelevant.

Have you written code that actually performs this way? Or are you speculating? I really do wish I could do this with epoll, but I (and several others) just never were able to figure it out. And kqueue simply performed at least an order of magnitude better of any variation I could hack together with epoll.


I had neither written code to test it nor was speculating - I was reading the kernel source, which is pretty clear (if you know in general how file polling is implemented in the Linux kernel).

However, because of what you have observed I wrote up a quick test, which confirms to my satisfaction that epoll_wait() respects SO_RCVLOWAT on TCP sockets: https://gist.github.com/keaston/d2473b8b996a34a5860b0744684f...

It seems likely that you must have been dealing with an old "enterprise" kernel in 2015 that hadn't had this fix backported to it?


Wow. When I saw your code, I thought to myself "yeah, but does it work when the socket is for a client?" Turns out, it does.

My only guess is that I was botching something else when I was digging into this a few years ago.

I can't tell you how much I appreciate you putting that together. This seriously changes a lot of what I work on. Thank you.

The client code I wrote: https://gist.github.com/benwills/95ee844853d3b18588fb3df7d56...


Good example.

The errors are just because you need to check for EAGAIN or EWOULDBLOCK when read() returns -1, and not bail out when that happens.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: