I'm not sure about how Tumblr might do it, but one can use a combination of ECMP and ipvs (with a consistent hash) to do the lower-layer L4 load balancing. This means that even if one of the L4 load balancers go down and the connections originally going to that L4 load balancer by the switch/router get moved to another one, the consistent hash to the L7 load balancer handling the request means the connections will not be reset (except in some interesting and less-frequent cases).

This two-stage process also allows for good health checking from the much-simpler Linux ipvs L4 load balancer servers to the more-complicated L7 load balancer servers.

This was described in this Velocity 2013 talk - http://velocityconf.com/velocity2013/public/schedule/detail/...