Hacker News new | past | comments | ask | show | jobs | submit login
Matching Binary Patterns with Erlang (dev.to)
51 points by StreamBright on May 1, 2020 | hide | past | favorite | 14 comments



It's even more powerful when you can have a length in your header and match the binary length to the value inside the binary

The major advantage to all this is that by being declarative you aren't going to mess up with binary codecs by getting lost in piles if statements to validate your binary lengths and prevent strange packets from messing up your system.


And optimization can happen.


true but for 98% of internet-protocolley things the network's gonna be the bottleneck, at least, IMO.


Wow, that's my NTP code they're pointing to! https://github.com/mnp/erlang-ntp

AMA?

Erlang seems optimized for several problems and network packet mangling is one of them.


Yeah, thanks for your code, it helped me greatly.

>>Erlang seems optimized for several problems and network packet mangling is one of them.

The only question is why other languages (claim to be low-level) did not adopt this.


Good to hear! And the blog post is quality hands-on learning material, like the sibling comment is asking about.

We know why Erlang does it well: its original narrow focus of switching packets and insane uptime. This meant it didn't prioritize other niceties, like strings for example. So it made sense to hard wire bit vector dissection and construction.

I think most other platforms have richer types and would rather use the generic type stack. Also, most general-purpose languages hide all the networking inside libraries so most users can just do foo = get($url) and not worry about it.


Yeah, Joe got that right. And a whole lot more.


I want to get some hands on experience with networking and am looking for a side project. Do you have any suggestions or know of any projects that I can use for inspiration?


It doesn't take much to get started.

A good start for reading is the concepts around IP, especially how protocols can encapsulate each other, also known as layers. Skim [1] maybe. The definitive reference used to be the W. Richard Stevens books cited there.

Next pick a fun, simple protocol you're interested in. Consider ICMP, Echo, DNS, or if you like NTP, see OP's fine writeup to repeat the experiments. Skim the IETF RFC [2] for your protocol, especially the bitmap diagram.

Install Wireshark and play around dissecting packets on your network, noting how it shows you the layers and bitfields as above.

The most important step is to write some toy code for a client, server, or peer for your fun protocol. As above Erlang makes it simple to take apart packets directly referring to the bitfield diagrams, but most stacks should have a packet inspection library you can mess with. Wireshark will help watch your work.

Another good thing to learn is routing. If you have a BSD or Linux box or a Raspberry Pi, learn some IPTables or BPF [3]. Play around with writing your own firewall rules on your box. Play with how the OS sees networking, ie netstat, arp, and route outputs.

1. https://en.wikipedia.org/wiki/Internet_protocol_suite

2. https://ietf.org/standards/rfcs

3. http://www.brendangregg.com/ebpf.html


The next protocol is DNS! :) It is a bit tricky but I have the working parser for the header. Looking into how I could parse qname with that octet thing going on.

https://github.com/n0hup/elafi/blob/master/apps/dnscache/lib...

It will be running on raspberry nodes (kind of replacing Pihole).


Nice. How do you feel about Elixir, seeing as how you're working in both? I'm happy with straight erl but always curious about the other grass.


Since I learned Erlang first and I really like it I have no issues with its syntax. I do not even understand people complaining about it. Elixir is just Ruby(ish) syntax on the top, it does not add anything (or takes away anything) at least to me. I write a lot of F# so I think I probably like it because Elixir took some of the features from F# (or ML languages). It is nice to have a pipe operator. One piece of functionality that is nice to have is fmt. You can format your code before committing to the repo, this takes away some of the bikeshed arguments. Other than that I am not using much from Elixir. Jose is a really smart Erlang guy though. He used to hang out on IRC and StackOverflow answering Erlang questions. Straight Erlang is fine.


hi @zest, I am working on a DNS filter in Elixir if you are interested.


Here’s a toy bloom filter I wrote a decade ago with this syntax: https://github.com/senderista/erlbloom




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: