I used to have a regular retail network connection at the office (local DSL carrier, not some industrial-grade expensive fiber thing) that allowed spoofing. The office was multi-homed (we had a second connection from a different provider for redundancy -- it turns out that our major local incumbent provider had maybe one nine of availability), and the results were hilarious. It turns out that, because Linux considers IP addresses to belong to a computer and not to an interface, it's fairly easy to (mis-)configure things so that traffic with a source address belonging to one interface goes out the other interface. This, coupled with spoofing being allowed, meant that we sometimes accidentally spoofed ourselves, and it worked! We went for surprisingly long times without noticing that some of our TCP connections went out one pipe and back in the other.
If I remember right, we would only notice it when we were confused by packet captures, when we had strange performance problems, or when we'd discover that we suddenly had two single points of failure. :)
If I remember right, we would only notice it when we were confused by packet captures, when we had strange performance problems, or when we'd discover that we suddenly had two single points of failure. :)