I know HAproxy will aggressively use kernel features to reduce the amount of actual packet handling it does. For example, it does zero-copy forwarding using the splice system call. How this compares to what seesaw does is over my head, though. I just wanted to point out that HAproxy isn't strictly Layer 7.
Anycast VIPs and Direct Server Return were a couple key parts of VDN "secret sauce" in early to mid-2000s.
Cool to see this engine out in open source.
To build a VPN edge you also need cooperative tiered caches with some very counterintuitive cache admission and eviction algorithms, unicast front end with p2p (for vod) or multicast (for live) back end, multi-datacenter event aggregation and correlation, cookieless/db-less sessions, and a few other goodies, some of which you can now even find as Nginx plugins.
Assembling what you need is much easier now with HLS, DASH, etc., than it was in the MMS/RTSP/RTMP days.
As of last couple years, you could almost assemble a viable global VDN with off the shelf open source parts. If the open compute project keeps up its good work, the physical kit gets affordable too.
I've worked for CDNs and configured many VPNs but I have never heard of a VDN. What is a VDN? I don't recall hearing this term in the early 2000's either. Is that a Google internal name? Could you elaborate? Thanks.
LVS load-balancing is amazing. You can use a tiny instance to load balance huge amounts of traffic since the instance itself doesn't need to process and pass the traffic.
the part that makes it difficult to implement is that the hosts have to be on the same network. I didn't dive too much into the code but it looks like you also assign an additional interface to use as a VIP. I believe you'll need to be in your own network and not the cloud to implement this.
The reason I ask, I worked with one hosting provider who leveraged LVS for well over billions of impressions per month and was rock solid for all their client base.
I'm pretty interested in this, but not yet in a position to roll out a dedicated infrastructure.
The third link that you posted is for keepalived. keepalived uses LVS as the actual load balancer (which is what seesaw uses as well). So as long as you can setup seesaw to run a few scripts to inform DisitalOcean which droplet currently has the ip, then it should work fine. There is an example for keepalived in the link that you posted.
It actually is a bit different in that it advertises VIPs via BGP, so no floating IPs are needed, but you need to be able to configure your upstream routers, and its configuration is much more flexible.
Is this just some tooling around Quagga and LVS then? I mean if you had Quagga/BIRD and LVS on your load balancers(not an uncommon configuation) isn't that pretty much all the heavy lifting? What else does Seesaw provide, SSL termination, config management?
Given that it's built on top of LVS, this presumably supports UDP load balancing. If so, that's great, as the available solutions for UDP load balancing are, to my knowledge, quite limited.
OT: I'm a Go noob and I was wondering if there's something like requirements.txt for Go projects? Or is this usually the way to go?
go get -u golang.org/x/crypto/ssh
go get -u github.com/dlintw/goconf
go get -u github.com/golang/glog
go get -u github.com/golang/protobuf/{proto,protoc-gen-go}
go get -u github.com/miekg/dns
This looks incredibly brittle. What if one of those dependencies releases a new major version on its master branch that breaks compatibility? It was really surprising to see this done this way. I know go's package management is pretty weak but a serious project like needs reproducible builds. Since all the dependencies are on git they could just use git submodules in a vendor directory or something.
This looks incredibly brittle. What if one of those dependencies releases a new major version on its master branch that breaks compatibility?
You are not supposed to do this in Go. If you make changes that are API-incompatible, you need to create a new import path. From the Go FAQ:
Packages intended for public use should try to maintain backwards compatibility as they evolve. The Go 1 compatibility guidelines are a good reference here: don't remove exported names, encourage tagged composite literals, and so on. If different functionality is required, add a new name instead of changing an old one. If a complete break is required, create a new package with a new import path.
Whether this is a good approach is the question, but pushing API compatibilities to an existing import path is considered to be bad.
(Since creating a new repository for each API-breaking version is annoying, there are sites such as http://gopkg.in/ that allow you to expose version branches as a different import paths.)
...where v3 is a branch or tag named v3, v3.N or v3.N.M.
What Go needs is a real package manager that uses the new 1.5 vendor directory to manage dependencies. There have been some attempts (godep, gb), but they aren't very good, especially if you look at how existing languages have solved this (Ruby's Bundler, Rust's Cargo).
This is the biggest problem in golang ecosystem. Yes there are many third party solution, but no official one. So different repo use different solution make all solutions invalid.
Unless there's something weird about that repo, go usually figures out what it needs to pull from the imports, you don't need to manually specify them.
Doing "go install github.com/google/seesaw" should do everything for you.
> It’s worth noting that while this project comes out of Google, the open-source version is not an official Google product. So don’t expect the company to provide any official support.
The distinction is that this project is Google written/used not google supported.
I didn't downvote, but your statement in no way refutes parent argument. Typically =/= always. It's similar to someone declaring "HN was down", and you reply by saying "HN is typically up"
It was more like someone saying, "Foo is not bar," someone else saying, "How is foo not bar? Bar is bar," and then me saying, "Because bar has a different meaning, colloquially, at certain companies."
At least 2 years ago it was already not always the case and things were moving towards unification rather than the other way around. Unless the general direction has drastically changed since I left I can only assume your statement is even less accurate now than it used to be.
I don't want to digress too much from the main discussion, but what are the main reasons people use Go?
What are the main use cases for Go?
Just looking for a new programming language to learn for some of the next web apps I'll be building.
Excellent concurrency, blazingly fast and memory-safe.
The first is difficult for a lot of the commonly used languages, the second is hard for Python, Ruby, etc, the third is hard for C, C++, etc.
One downside (although some consider it an upside by design) is that it can be a bit more verbose than other languages since there is rarely any "magic". You can usually read any Go code and be pretty confident at what it does, because it doesn't allow you to hide complexity. This does tend to make the code more verbode.
I highly recommend taking the interactive Go Tour at https://tour.golang.org/welcome/1 because it will show you everything you need to get a high-level overview of Go's power.
Well, there is `EngineCon` interface [1]. And two types (`engineIPC` [2] and `engineRPC` [3]) that implement the interface. That's why there is some repetition. So, what?
Guilty - my apologies. Except rather than sell it I'm simply looking for feedback. While the culture of using similar tools (reek, rubocop, flay, flog etc.) exists in the Ruby ecosystem and it's easier to calibrate algorithms and expectations Go is an uncharted territory. This is something I'm really after and again sincere apologies for my previous misdemeanor.
It's good etiquette to include a disclaimer, since this code analysis is a tool/company you are affiliated with.
Personally I've never seen any of these code analysis tools produce anything remotely useful. De-duplicating code might be nice in some cases but it certainly has nearly nothing to do with the over-all software quality.
As much as I love Go and use it extensively for various production-grade projects, I'm wondering how suitable it really is for something like a load balancer.
My understanding is that Go's TLS performance is dramatically less than other C-based implementations like OpenSSL. I remember reading something about some licensing collisions between the Go project and some TLS stuff such that optimized encryption code couldn't be incorporated into the Go project directly. Cloudflare got around it and has much better performance for TLS.
Apart from that, GC is still a big deal and the current net/http standard library produces a lot of garbage.
Go's TLS support should be greatly improved in Go 1.6 thanks to the performance work you referenced by Cloudflare. The licensing issue was resolved and the work was merged 11/10/2016[1]
While GC will always carry a cost, great improvements have been made over the last year and continues to be an area of focus for each Go release. Rick Hudson gave a nice update on Go's GC improves at GopherCon 2015 [2][3]
I have not looked deeply at the implementation, at all, but given that the actual site refers to this as an "algorithm" for LVS, I kinda suspect all the heavy lifting is happening at the kernel level within LVS, and the go code is merely acting as a control program that tells the backend how to behave. Many high performance systems have lower performance scripting languages or similar built in or bolted on, in order to provide easier means of development without impeding performance significantly. LVS is no exception.
The LVS-based systems I've built and used in the past often had a number of non-C components (mostly Perl, as it was 10+ years ago), for health checks, data gathering, making balancing decisions, etc. It is entirely possible this is the kind of work Go is doing (again, I haven't looked deeply into the code...but, I don't immediately see anything indicating Go is doing the actual load balancing, but it is obviously doing health checks and providing management access).
Finally, Erlang is a garbage collected language, and yet it has been used for a couple of decades for this kind of workload. So, evidence strongly suggests it is possible for a GC language to do things like this (though, again, I don't think Go is doing the network layer work here, since LVS is in the picture).
That's because Erlang's GC is managed on a per-lightweight-process basis, which is possible because of the share-nothing nature of the concurrency model Erlang uses (actor/message-passing).
Garbage collection in Go can't be implemented the same way, because Go allows for shared mutable state.
That said, the work done to the GC in Go v1.5 seems quite phenomenal, but just because the two runtimes share a piece of terminology called "garbage collector", they're very, very different in terms of the semantics and effects exposed.
This is a Layer 4 load balancer. It doesn't look at the contents of packets; it only decides where they should go. So Go's TLS performance isn't relevant here.
Premature worry (aka FUD). We run Go services that terminate TLS and handle thousands of req/sec per server and it's never been even a slight concern. CPU usage for TLS has not been measurable. GC has also not been an issue, at least wrt net/http.
This component basically acts as a control plane for LVS which is implemented in the kernel. The data plane is not Go. Basically this load balancer is really the Linux kernel (and it's blazing fast).
Note that this is a network-level load balancer that is tightly coupled with LVS, not a Layer 7 load balancer like HAProxy.