All your kind words are literally bringing tears to my eyes--thank you!
My current plan is to keep writing freely-available guides for as long as I can reach the keyboard. And maybe even longer with what will undoubtedly be awesome futuristic speech recognition--or mind-reading! That's not scary at all!
I'm leaning on bash/zsh scripting one I finish the utterly gigantic C guide I'm on now... hopefully later this year. But I'm always open to suggestion for topics... :)
Your page was what bring me into "magical" programming as a teen 20 years ago. Being able to communicate with a remote program on a free shell and actually understand how it works was the best feeling.
This indirectly shaped my life in many ways. Thank you!
Your guide single-handedly got me through network programming in C at university. This was back in the early 2000s. Thank you! And I'm so happy to see your guide still getting updated.
I feel so nostalgic when I read this Beej's Guide. It's a resource that my computer science professor highly recommended back in the early 2000s. The writing is so clear and concise that it's a pleasure to read (as opposed to some dry and technical materials). It's more useful than some textbook that cost hundreds of dollars. It saved me so much time and sleepless nights to figure out how to complete my network programming assignment.
Let me give you a virtual elbow bump! You really made an impact in my academic and professional career!
When was the first edition? I feel like I remember using it in the early 90s ... maybe 1991? The mods have put the dates "1994-2020" in the title, but I feel like I remember it from earlier than that.
Edit:
OK, on 2003/07/06 15:11:16 the wayback machine captured Version 1.5.5 (13-Jan-1999). At the end it says:
For the brief amount of time I was your student at Lambda, you made a huge impression. Beyond this tutorial, you're a gifted teacher and inspirational person. Much respect and love!
I'll read your future-brainwave-transmission tutorials, anytime, even if they run the risk of causing minor insanity!
When I was first learning about network programming, I used to pour over your guide (hosted locally on a Chico State account at the time, if I remember correctly), absorbing every detail I could fit into my young mind. That was... more years ago than I want to think about, but I'm glad to see the guide continuing to get updates!
And I'm also glad for the opportunity to directly thank you for the work you've put into this over the years :)
You're Beej?! I learned all about socket programming from your guide, back in high school. In college, during a group project, none of us had taken a networking class yet, but I was armed with your guide and I pushed us to make a networked game. It was a great experience, the game worked well, and the networking teacher was brought in to take a look (he was impressed!).
Thank you so much for your time. I hope you realize how useful it's been.
> I'm leaning on bash/zsh scripting one I finish the utterly gigantic C guide I'm on now... hopefully later this year. But I'm always open to suggestion for topics... :)
That would be amazing, I'd love to read a guide over scripting. The only request I would have is a section over sh as well if you do write such a guide.
Hi, thank you for the great resource, I love the clarity and simplicity without the useless "filler" material to artificially increase a book's number of pages.
I looked up the section on classful networking and CIDR and would like to make one addition that is often overlooked: in classful networking, not only are the network sizes fixed, but also where they reside in the IP address room.
E.g., class A networks used 0.0.0.0 – 127.255.255.255 whereas class C networks used 192.0.0.0 – 223.255.255.255.
With CIDR notation, not only can you have networks of different size, but they can start at arbitrary addresses.
If people talk about a class C network today, they often falsely relate to a network with 256 addresses - but which is located in the address room of original class A/B/D/E networks, for example.
Your guide was what everyone else recommended when I was stumbling around learning networking a couple years back (reversing and reimplementing an iOS framework that used the BSD sockets API–not really the best environment for someone who couldn't tell connect(2) from accept(2)). With your help I was able to get through that and the understanding carried me through college, in some networking classes that were decidably less educational or useful than your writing. Not bad for a document bearing a copyright from years before I was born!
I discovered your guide for the first time today after decades of programming. Light bulbs going off constantly - thank you, not only for clearly elucidating the vital points, but also for the great humour which keeps me firmly grounded in the marvel and insanity of what's been built to connect us all together.
Such topics aren't dry and boring, they're fascinating and inspiring, and it's so valuable that you are able to present them that way.
I've pointed so many folks over the years to your guide.
Keep up the great work.
P.S. Each time I do refer folks over or see a thread like this gives me fond memories of the O'Connell labs and the opportunities we had to enjoy networking in the early days.
The fun of the HP bug of the week and running a unix farm with shell accounts for all of us students.
Amazing! This guide got me into network programming back when I was still on rural dial-up. It helped me through various programming courses in high school and university, and I still occasionally refer to it from time to time. Thanks for your contributions, they really improved my quality of life!
We were both at Chico State in comp sci together. I remember you although we didn't know each other. Dude, you do great work! Every time I see your guide mentioned somewhere it always makes remember that time, and puts a smile on my face.
I'll add another big thank you. Your guide was the sole reason I know anything about network programming, and was very valuable even in Python where documentation was(maybe still is) sparse.
This guide was instrumental in letting me understand sockets and network programming. I've probably recommended it to at least a hundred other people over the years.
I just came to say thank you, Beej! Your guide helped me learn networking and network programming. I love learning by example. Your guide was instrumental in my learning.
your guide literally helped me hack through a networking class i took in college a decade ago. dont work anywhere close to that stack but this brought back memories.
Back in 2010 I had my first embedded programming job. I needed to do some network programming in C (controlling a PLC over ethernet). My boss at the time told me "just read Beej's guide and you'll be fine". He was right. That system is running 24/7 since then.
Good to see that the guide is still being updated.
Heck, I passed my uni network programming courses thanks to Beej. It also helped tremendously that our professor was a seasoned UNIX programmer and during lectures he actually typed in his own code and ran it.
I had a few maths lectures who were highly regarded for turning up and proving things from memory in front of the blackboard. I hadn’t really considered that it might be done in CS.
The best lecturer I had in maths would screw up the proofs every time from memory, but the way he screwed them up taught me a lot more than if he'd gotten them right.
I remember a lecturer of mine having to state and prove a theorem due to Heawood[1]. I think he did it first because the statement was quite unpleasant and not easy to memorise. I found the point of proving things from memory is that to memorise a proof, one must distill it to key ideas with obvious steps in between. This was particularly useful if you might need to reproduce a proof in an exam.
My experience is that they get about 45 minutes into the lecture, realize that there was an error in minute 10 and spend the rest of the period trying to fix it before telling you to check the book for the correct proof.
One professor provided scans of his written notes. For one of the theorems in his notes, he’d stated it, attempted to prove it, failed, crossed out the proof and wrote ‘QED’. At least he was honest enough to include it!
You may be able to get it from your local library via an app like Overdrive[0][1]. Some library programs even have a way for you to get free access to Safari Books Online[2] (I'd say most if not all, in the US at least)
My experience has been that there is rarely a backlog on technical materials like this.
Libraries are like the secret weapon of a learner. I don't know why our industry has such a lack of knowledge about this one! I only recently found out about being able to do this myself
(In some ways, we benefit, since there is little pressure for the large publishers not to offer these kinds of things for free via the library system)
Ancedota (and unrelated to the topic of this thread, but interesting none the less I think):
In California, if you go to a state university (but if I recall correctly this doesn't apply to the University of California system, but the CSU system, if I got my facts straight) they must provide a minimum number of copies of each textbook / reading material required for a class be available via the campus library.
I hardly paid for books once I found this out. I just coordinated with others to make sure it was available when needed. Worked well for me.
I could be hazy on the regulation on this though, it might have just been dumb luck that the two places I went to school had this as a policy, but I remember it being explained as I did above.
Yes, I used the guide in 2006 I think or it at least looks similar. I had used it to write a little webserver in C to talk between the Flash UI and the internals of the device. Worked well :)
Socket programming never clicked for me before I learned about the protocols themselves first. My brain just rejected using an API for something I didn't understand. So if you're like me, I highly recommend reading Comer's "Internetworking with TCP/IP" first, and sitting down to socket programming later.
Volume 1 of TCP/IP Illustrated by W. Richard Stevens is a good one. If you're very pressed for time, start with the picture of the TCP state machine and work outwards.
Maybe, but in C land that high level socket API never materialized in the standard library. And you are right that it is awful. I've been doing network programming for over 20 years now and I still have to look up the manual when making a TCP/UDP connection in C.
I've made my own wrapper a few times but often don't get to use it for various reasons.
int tcp_connect(char* remote_host, short int port, options* options, static char** error_string);
int tcp_listen(short int port, options* options, static char** error_string);
int udp_listen(short int port, options* options, static char** error_string);
Does any necessary lookups for you and connects sets up the socket. The options structure can be NULL if you want the sane defaults, or you can zero it out and set only the options you care about like the connect timeout or flags like SO_REUSEADDR or O_NONBLOCK. Returns the socket number or -1 if an error occurs. In the event of an error the error_string is also set. The _listen variants spit out a socket ready for an accept() call.
If you need something weird then you're back to the low level BSD socket stuff, but this covers about 95% of my use cases. While it's only saving a handful of system calls, it ends up being quite a bit of boilerplate and error handling. Especially if you specify a connect timeout, since that is kind of awkward to do with the BSD socket API.
I had the most wonderful déjà vu the other day. I needed to do some serious-ish network programming for the first time in 20 years. I googled around a bit for a refresher. What do I find? Beej's guide – the same Beej's guide that was my first introduction to the topic 20 years prior!
It was wonderfully poetic to me that the same document that taught me how to make a wildly dangerous IRC bot in my teenage room for the lulz also re-taught me the stuff I needed to deliver a low-latency ML offloading demo at my serious(-ish) job 20 years later. Gave me a real warm fuzzy feeling :-)
> It was wonderfully poetic to me that the same document that taught me how to make a wildly dangerous IRC bot in my teenage room for the lulz also re-taught me the stuff I needed to deliver a low-latency ML offloading demo at my serious(-ish) job 20 years later. Gave me a real warm fuzzy feeling :-)
My experience with CTCP flooding large channels channels and causing netsplits in my youth provided a perfect base of understanding for when my production servers started flooding status updates to each other and couldn't stay connected.
PS sorry, undernet admins and users. Also, thanks to the admin on #quebec who asked me to flood his channel so he could tune his bot to block flooders.
I thought I'd add: I know that this is a very normal thing in some sense. Many textbooks remain relevant for decades and decades (I'm a mathematician, we know this perhaps better than most). But still, it's rarely the case for an online document – one about computer technology even more so!
I remember particularly enjoying the warm and jovial tone in his guide.
I first read through it some 20 years ago while being a student in an easter-european university, where the tone and style of professors was very "dictatorial", dry, and overly-technical.
Beej's style of exposure was a breath of fresh air, and brought a major realisation for the then young-me, that you can teach complex topics in a friendly manner.
> while being a student in an easter-european university, where the tone and style of professors was very "dictatorial"
I know it's just I typo, but now I have an image stuck in my head of a large rabbit, dressed up like Mussolini, lecturing about computer science.
Beej's guide was so amazingly helpful and welcoming when I found it over a decade ago.
Poorly written guides that try to be friendly or conversational are often worse than dry and terse guides of similar caliber, which really makes me appreciate the good and truly great ones.
I began writing a book for them in 2010 and quit when the goalposts kept being moved and had a different point of contact every week. Absolutely terrible experience.
Similar experience. In my case, they wanted a video course on a library that wasn't finalized or published, and was still in an obvious beta with frequent breaking changes. On top of that, I was in a 12hr timezone difference, and only wanted to communicate with me regularly at 11/12 at night on week days, which didn't work well. Oh, and they only wanted to use skype, an app I never use normally. I was promised promo codes and release on multiple websites, and I got a single weekend promo code, and only published on two websites. And the icing on the cake is now 100% of the people I worked with don't work at Packt any more, so I don't even have a person I can contact any more to work on promotions or updates.
Production failures due to bugs in both TCP-based servers and clients happen all the time, long time ago I invested a little time to learn basic socket programming in C and the ROI on this has really been spectacular. "Unix Network Programming" by Stevens is a great book on this topic, in particular it carefully discusses all the different ways things can go wrong. Wish there was something more up to date that could be recommended, but I don't know anything of comparable quality, so I think you still have to read it and later work out all the Linux-specific details, more modern APIs like epoll etc.
To anyone writing tutorials or guides, this is the gold standard - at least for me. After many years I still remember writing my first tcp client following this tut. I wish more content would be as focused and concise as this.
One if the best guides/books on programming I've ever read. This taught me the foundations of network programming and the fact it's available for free is incredible. I just took a second and ordered the paperback right now because of how fantastic it is.
Aside from the book being really in-depth and understandable, Beej adds in quite a bit of humor and that is something I love in a good software book.
Strangely no mention of High Performance Browser Networking
https://hpbn.co
I also would mention Distributed systems for fun and profit
http://book.mixu.net/distsys/
while being not exactly about network programming, it is likely a next step for anyone who's building apps that is scattered across several backends/services connected by the network
I had the privilege of working with Beej for a couple years at my last job. He was one of my favorite coworkers I've ever had. Just great to be around and work with. Incredibly sharp, but also humble, easygoing, and eager to just solve problems.
This "Beej" fellow is a legend. This is an example of very clear documentation that scales well with your level. When I was learning C/C++ as a teenager, I found the guides to be very accessible. They were recommended all the time to people asking questions on cprogramming.com forums. As I've referred back to them as a professional I still find them comprehensive and helpful. This is an excellent model and level of quality for other technical writers to aim for.
Wait... the Lambda School that's constantly on HN for its disgruntled students, questionable success rates, and murky financials? What possessed beej to associate his name with them?
And here's some reporting that aggregates questionable practices and student complaints: "The High Cost of a Free Coding Bootcamp"[1]
And here's some paywalled reporting from The Information with even more student complaints: "Lambda School's Growing Pains: Big Buzz, Student Complaints"[2]
Please, tell me more about the view from the inside.
A couple of decades ago, I spent a few years teaching an introduction to network programming using C on an embedded board running uCLinux for Network Engineering students. While my primary text was Stevens, Beej's guides were an invaluable secondary reference for my students. There were many, many occasions where students who were were having difficulty understanding my explanations could recover using Beej's approachable, application-oriented notes. Eventually, I found myself frequently abandoning my own notes and just lecturing from Beej's work. I miss the days when I still worked
with students that close to the metal, helping them build their own network devices. Thanks for the reminder of a time I still have fond memories of.
My first job out of university reqiured networking a small Linux device to an FPGA running a UDP stack, with nothing else. I'd never done anything with networking in C before, and this guide just made everything click
Haha, that was the exact class where I used it! Loved that class, though. By far my favorite in their MS program. It awakened a love for operating systems (and the general applicability of the concepts). AOS was also good and made me appreciate the point of cloud computing more.
Can confirm this is a nice introduction to network programming; it's how I got started. Once you know the basics you can switch to the man pages for more details.
Despite of its age, Beej's Guide is still one of the best introductions to network programming I've seen so far. I love how apart from the basic and concise explanation of TCP and UDP (and generally sockets in UNIX-like environments), there's also a separate, short "Advanced Techniques" section that can be very useful if you want to explore the subject even further.
I read this for the first time probably 15 years ago, and to this day I can't think about IPv6 without thinking about this passage from the guide:
What does this suggest to you?
That we need a lot more addresses. That we need not just twice as many addresses,
not a billion times as many, not a thousand trillion times as many, but 79 MILLION
BILLION TRILLION times as many possible addresses! That’ll show ’em!
You’re saying, “Beej, is that true? I have every reason to disbelieve large numbers.”
Well, the difference between 32 bits and 128 bits might not sound like a lot; it’s
only 96 more bits, right? But remember, we’re talking powers here: 32 bits represents
some 4 billion numbers (232), while 128 bits represents about 340 trillion trillion
trillion numbers (for real, 2128). That’s like a million IPv4 Internets for every
single star in the Universe.
“Some of you might want to do things the Pure Windows Way. That’s very gutsy of you, and this is what you have to do: run out and get Unix immediately!”
He's just joking of course. At first glance this seems like a great read. And he's funny too! :)
I do wonder how much of this is applicable to Rust though.
Sadly the Windows asynchronous (or something, I never quite got it) networking API seems very powerful. But I never found a comprehensible manual/tutorial for it.
I remember chatting in the fringes when nodeJS was doing their proper win32 porting and they did have some issues getting things stable due to lack of documentation but iirc got some help from MS so the libuv code should be usable as a reference for behaviours (As well as being fairly battle tested by now).
I’ve been trying to figure out the entire windows networking stack recently, particularly some more obscure parts and the docs have been pretty bad unfortunately. A lot of the higher level stuff fees really superficial and the lower level stuff it’s it own mess (all sorts of broken links and links to outdated versions in the NDIS docs).
A couple of observations regarding select(): its use is not limited to sockets; and, quoting Wikipedia, "with the c10k problem, both select and poll have been superseded by the likes of kqueue, epoll, /dev/poll and I/O completion ports."
Georgia Tech's Graduate Introduction to Operating systems OMSCS class rings a bell. Socket programming was fun in that class. And by fun I mean hours of hair pulling while debugging.
My latest immersion into Beej's guide was through the GIOS course too! It also opened my eyes to the amount of familiarity/expertise my older co-workers had with systems programming (they helped me when when they heard me whining about the socket programming assignment). It taught me not to dismiss someone just because they don't know the latest TensorFlow API/Neurips paper.
Beej has had great guides for many things on the internet for decades. I fondly remember learning C from Beej's guide to C [1] upon being recommended by WinterMute (from the Nintendo homebrewing community). Best introduction to programming I've seen by far!
What a classic! Back in 1999 I wrote the networking infrastructure for a distributed, evolutionary, fractal art system for Steven Rooke (maybe you remember the fractal absolut vodka bottle on the back of Wired magazine) and I learned everything from Beej's guide. This brought back great memories. What a serious internet classic for so many software devs. Thanks Beej!
Beej's guide is great and still very relevant today (at least if you're programming in C). Are there any equivalent documentation/tutorials for other domains? An equivalent for machine learning would be great, although part of what makes Beej's guide timeless is that a lot of the underlying concepts and syscalls have been static for decades.
I've always loved networking and recently implemented an IP stack just for fun. It doesn't take that long to be able to write ping yourself and ping Internet hosts. I was going to go as far as TCP but haven't gone there yet.
I think it's now time to read this and find out all the stuff I got wrong and cement the stuff I got right.
Networking is probably one of the parts of computers I’ve been consistently interested in over the years. I was recently looking at the man page for the addresss_families supported and there’s all sorts of crazy stuff that’s been adapted to the socket API in there.
I’ve been working on developing myself a custom ISDN stack recently. So far it’s just been me trying to design it at a high level without falling to feature creep.
I think what I love about it is the Internet is the biggest machine in the world! I can literally flip bits on the other side of the world by pressing some of these buttons here on my desk. After becoming pretty much accustomed to living behind an IPv4 NAT my interest was rekindled when my ISP deployed IPv6 and all of a sudden each device in my house has a real public IP address.
I'll definitely finish mine one day then. I was more interested in the IP layer when I did mine and learning about routing. TCP seemed a bit too complicated and not that interesting, but it would be cool to have at least once gone from a raw socket to HTTP.
I know exactly what you mean as well. When using technology there is a comfort in knowing that you could conceivably implement it yourself if you absolutely had to.
I read your guide when i was 14 (translated to hebrew), until now, i have managed to write a functional usermode tcp/ipv4 stack, and now in the middle of development of a usermode-based wireless networking device in an international company.
You are a big part of it. Thanks.
Ironically, with “they” becoming a more common specific-individual-identity-pronoun preference, it is at risk of losing its inclusivity, and of becoming as problematic for unknown individual preference as she and he.
But that's a problem for the future, right now “they” is usually the best inclusive pronoun available.
I've also personally never understood the slash syntax. Since one word requires the "s" and the other doesn't, shouldn't we use parentheses like when words may or may not be plural? e.g. "(s)he" like you might say "socket(s)" or "Unix(es)"?
Moot point if it doesn't solve the issue of it still being binary, I guess.
If you're serving webpages, look for a networking library that handles HTTP.
If you're sending arbitrary chunks of data (like for a game), use a socket library. I've used WebSockets in the past, which handles the low-level stuff under the hood. There are libraries that implement it for Java, Go, C++, JS, etc. Check it out at https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_...
It's not totally headache-free though - some issues I ran into were:
Your question is insufficiently specific ... it could mean nearly anything. You need to be clearer about what equipment you have, and what you're trying to accomplish.
As it stands, I don't think anyone can answer except it a completely generic way:
Almost every system has libraries to help, but it depends on what you're running on, what you're running over, and what you're trying to do.
After reading Beej Guide I think I find "The Linux Programming Interface" a good book in general to learn about sockets and other linuxy thing etc. Sadly it hasn't been updated since 2010.
Oh wow, thank you Beej for this work and content! Your resource has been a tremendous reference to many of us myself included and still stands as one of the best resources to network programming today!
When I TA'ed a course that taught networking fundamentals, I always pointed folks at this guide when they were struggling with assignments. Can't believe that was 8 years ago.
I credit beej to my first serious job. I worked through the network and concurrency guides and landed the job that started me off on my career. Thanks Beej! Please keep it up :D
Among other things, Beej has been a leader in putting together events and connecting software people here in Bend, Oregon. Well, when getting together was a thing, at least.
As soon as I read the title, I felt joy! I had followed this guide when I had began programming early on and had a great experience. I recognize it by its name alone!
I’ve always felt like the hard part of networking isn’t the APIs but the threading problems that you will have very soon after you try to write anything serious.
Even though some of the API specifics change, the BSD sockets API has been widely copied. It's a pretty decent introduction to the concepts in general.
My current plan is to keep writing freely-available guides for as long as I can reach the keyboard. And maybe even longer with what will undoubtedly be awesome futuristic speech recognition--or mind-reading! That's not scary at all!
I'm leaning on bash/zsh scripting one I finish the utterly gigantic C guide I'm on now... hopefully later this year. But I'm always open to suggestion for topics... :)