Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

TCP over TCP is a great idea for end user VPNs because 1) networks are generally pretty reliable and 2) NAT is everywhere. I regularly use an SSL VPN over crappy coffee shop wifi and tethered 3G connections. For the most part it works fine.


NAT is an argument against TCP, not for it, because when your NAT'ed IP address changes, TCP connections break. UDP will drift between NATs without skipping a beat.

And I'm very surprised you find crappy coffee shop WIFI and 3G to be pretty reliable - those are exactly the kinds of networks which have occasional sustained periods of high packet loss which wreck havoc with TCP.

Edit: the only downside of UDP VPNs is that stateful firewalls can have extremely short timeouts of UDP "connections" (e.g. 30 seconds!), which necessitates the VPN to constantly send keepalives, which kills battery life on mobile devices. TCP connections tend to be given much longer timeouts.


Completely agree with your righteous rage about TCP over bad connections. I allowed myself a bitter chuckle to the GP's "networks are generally pretty reliable" thing.

Minor disagreement though:

"Edit: the only downside of UDP VPNs is that stateful firewalls can have extremely short timeouts of UDP "connections" (e.g. 30 seconds!)"

Plenty of mobile networks will timeout inactive TCP connections in less than 30 seconds. TCP keepalives are an absolute requirement on long running mobile connections.... e.g. Google maintained GCM connections on Android for notifications... a simple packet capture will show you the frequency of keepalives there... and it's almost always more frequent than per 30 seconds.


> Plenty of mobile networks will timeout inactive TCP connections in less than 30 seconds.

Good grief - that is awful, but sadly believable. Do you happen to know who does that? A couple years ago I tested AT&T's 3G and found that the TCP timeout was 30 minutes, versus 30 seconds for UDP. I'd love to know numbers for other carriers.

Edit: Found an interesting paper from 2011 that tested 73 cellular carriers worldwide and found only 4 with TCP timeouts less than 5 minutes. The majority had timeouts greater than 30 minutes, and 21 had a timeout in the 5-30 minute range. Some of my faith in humanity has been restored. http://www.cs.ucr.edu/~zhiyunq/pub/sigcomm11_netpiculet.pdf (see page 8, table 5)

Edit: A paper from 2012 which measured Verizon and Sprint at 30 minutes, and AT&T at 3 minutes (my tests are more recent, so perhaps AT&T wised up?) http://www.cs.umass.edu/~yungchih/publication/12_mtcp_4g_tec... (see page 11, table VI)


Timeouts that short violate RFCs. Established TCP connections can't be abandoned unless idle for two hours four minutes (RFC5382 REQ-5) and even UDP timeouts normally have to be at least two minutes (RFC4787 REQ-5).

If people are violating the RFCs then applications that detect it should probably start notifying users exactly why their battery life is suffering.


I see a lot of "should"s. When push comes to shove, though, a notification is not going to change the behavior of the network.

There's a lot of complaints I have about how some networks (especially mobile ones) deviate from RFCs and break specifications. A 30 second timeout is not nearly at the top.


TCP over TCP for VPN is pretty useful in places where deep packet inspection blocks UDP by default, eg. GFW in China.


Do you have any idea why the GFW blocks UDP by default? I can imagine that for corporate networks, but as far as I know, uninspectable UDP streams only recently came into existence with Google's QUIC (which assumes pre-negotiated encryption keys are still valid)

Why block all streams if you can inspect them (or at least their handshake)?


Probably because they determined that most of it was VPN or other encrypted traffic, and that blocking it was easier than trying to inspect it.


While practically no official information exists publicly, this appears to be the reason. My gut tells me that the lack of structure in UDP makes it a little harder to inspect too.


Do you have any information you can share about this? A few years ago, I could reliably use OpenVPN over UDP, as long as I switched ports out frequently. Some time ago (I don't remember when), this ceased to be the case, and I switched to PPTP and, more recently, Shadowsocks.

What has been your experience with UDP over GFW?


UDP used to work (~3 years ago) but currently it's blocked wholesale. OpenVPN over TCP gets throttled and blocked thanks to DPI too, an obfuscation layer is required because OpenVPN traffic is identifiable due to a fairly unique encryption fingerprint.

http://blog.strongvpn.asia/china-blocking-udp-ports/


Thanks. This is totally consistent with my experience, and it's good to know that it's not just me :)


I'm just saying in a previous life I used to spend a ton of time fighting with IPSec NAT traversal issues. With TCP encapsulation (e.g. SSL VPN), you don't have that problem. Most NAT firewalls do a good job dealing with TCP. Other protocols are more questionable.

When I'm using wifi at a coffee shop and start getting a bunch of packet loss, I will switch to a tethered 3G connection. When my SSL VPN reconnects, the VPN server hands me back the same IP address I had before. In some cases, my SSH sessions don't even drop.


IPSec is indeed hell with NATs, and an SSL VPN would be much better. But UDP is even better - most NATs do a good job with UDP too, and if done right, it's possible to switch Internet connections without the VPN having to reconnect.


The hell are you fellas smoking? IPsec NAT traversal has been a non-issue since it was standardized about 10 years ago.


DTLS is a standard protocol for TLS over UDP. It is used by existing commercial products, such as Cisco Anyconnect.


In my experience, many NATs just drop UDP packets altogether, but still allow TCP through.

Similarly, I find the Internet at my local Starbucks to be some of the fastest, most-reliable networks around; even faster than some local ISPs hooking up to my home.


Note that any NAT that drops UDP packets altogether will basically disable DNS and VOIP type applications or at least degrade the experience in serious ways. I haven't come across many such NATs recently.


Me neither. I have come across networks that filter everything but a few TCP ports (like 80 and 443), but that's a matter of draconian firewall policy rather than a NAT limitation.


I have come across networks that filter everything but a few TCP ports (like 80 and 443)

Another point scored for TCP in TCP with an SSL VPN. :-)


Only if your TCP VPN is listening on one of the allowed ports ;-)

(I do keep a TCP VPN running on port 443 ready to go for these situations, but UDP is always my first choice.)


Google is apparently paying for fiber connections to Starbucks, since AT&T DSL wasn't getting it done.


Did you read the linked article? (Or did anybody else who is replying?)

It's not about how TCP-over-TCP is somehow aesthetically displeasing or about how people should feel bad about doing it. It's about how TCP-over-TCP is a technically bad idea because stacked TCPs interact poorly. It's never a good idea. TCP-over-TCP is still a profoundly flawed protocol even if you don't happen to tickle its problematic cases.


> It's never a good idea.

I disagree with that profoundly:

I read the article. Yes, TCP over TCP include unnecessary performance-harming features in certain circumstances, because you have two unrelated collision avoidance systems running at the same time. Even so, in many scenarios TCP over TCP is an excellent idea: it can provide you with many benefits, in practical terms works great, and has no readily-available alternative which is better.

It doesn't solve the problem as theoretically neatly as possible, and carries some cruft. But hypothetical me at a starbucks about to open an ssh tunnel to a trusted connection, hypothetical you telling me that that's never a good idea. Okay then, what should I do instead?


TCP over UDP.


So sitting at starbucks with my laptop -- what do i do? I am not aware of an option to have SSH run over UDP, although I do know that some VPNs allow you to use UDP instead of TCP.

Unless there is a relatively simple way of getting an encrypted tunnel for my HTTP traffic using tools like ssh and netcat and other things I'm likely to already have installed, I disagree with the notion that it's never a good idea.


When you are running a SOCKS proxy through ssh, you are not doing TCP over TCP. We are talking about things like OpenVPN which can do TCP over TCP, but that is generally a bad idea. It's default mode is TCP over UDP, as it should be.


> When you are running a SOCKS proxy through ssh, you are not doing TCP over TCP

Are you sure? SSH uses TCP, and encapsulates the web traffic which also uses TCP... in what way is that not TCP over TCP?


Here, read this: https://github.com/apenwarr/sshuttle#theory-of-operation

"TCP-over-TCP" and "TCP carried on something that happen to be TCP" turn out to be two very different things.


Thank you for the link :)


Ok, so how can we fix this?

Perhaps we could disable retransmission on either the upper or lower layer?


> For the most part it works fine.

Only for apps that don't try to utilize maximum throughput. Skype, YouTube, most web browsing, most mail use.

But those that do - like large file transfers over ftp/sftp or a very large email, for example - will cause the meltdown described in this article.

There are some TCP stacks that use RTT rather than packet loss as their congestion metric; Those fair well under a TCP-over-TCP regime (but have other problems)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: