>To the GFW, these 8 ApplicationData records could look like 4 pairs of HTTP requests and responses in a keep-alive connection. However as research has shown [5] [6], side-channel leaks in TLS can be exploited, for example by looking at packet sizes. Doing so, we can see that they indeed match the expected sizes of the messages exchanged during a CONNECT request and a TLS handshake:
As someone who crack (or at least block) VPN protocols for a living, I can indeed confirm this is 1 of the tricks used by all deep-inspection firewalls to detect VPNs.
In fact, a very popular VPN software for Chinese citizens uses TLS-within-TLS (sometimes fake TLS) to hide its data. From the author's description, the traffics are similar enough IMO for the GFW to detects.
(No, I don't work on GFW, but we block similar software. Sometimes we do comparison between various products to see how others block them.)
Which leads to absurd situation like a big Chinese company flying people from Europe to their Chinese offices to provide on-site support, and then having corporate firewalls more restrictive than the GFW itself, which renders said workers barely able to work at all...
Helping people get around the Great Firewall was one of the main reasons I started working on Streisand[1]. OpenVPN (wrapped in stunnel), Shadowsocks, and Tor (with obfsproxy) are all still highly effective. The setup process is completely automated, and other connection options are available too.
1. Never use Google Maps in China; Google Maps in China are just hilariously wrong and out of date and will get you lost. Even if Google Maps wasn't blocked you wouldn't want use it in China. Apple Maps is the best English language solution, it's accurate and up to date; that or Baidu Maps if you read Chinese.
2. The OP is missing the point of the GFW. It's not really about censorship. It's mainly about providing a market for local tech startups and about keeping the lower classes from organizing and causing trouble. The party doesn't care is middle or upper class people access the outside internet. They know network effects will keep them using local tech and middle or upper class people are too bought into the system to rock the boat -- after all these people travel outside of China and have a decent knowledge of the world outside. Just assume anything you put through a VPN that advertises as going through the GFW is being monitored. Just like in the elsewhere you'd just assume anything you'd send through Google is being monitored. On sensitive days suddenly all these VPNs are blocked too so the GFW definitely does have the power to block them at will.
Finally I find it a bit odd that someone so concerned with privacy uses so many Google Apps. Isn't that rather counter productive?
Also, only web based gmail is blocked. Access from Mail.app or iPhone's Mail is not blocked. Sadly translate has been blocked for about a year now... all the more reason to never turn the VPN on your computer or phone off.
Well, yes, the GFW provides censorship but that's a part of the government's greater goal of pacification of the population. The post's point is pretty clear as it elaborates that the upper classes in society are not likely to care since they have a better life, and know how things really are.
> 1. Never use Google Maps in China; Google Maps in China are just hilariously wrong and out of date and will get you lost. Even if Google Maps wasn't blocked you wouldn't want use it in China. Apple Maps is the best English language solution, it's accurate and up to date; that or Baidu Maps if you read Chinese.
I'm an Android user. I just spent 10 weeks in Shanghai from early October to mid December. I can read some amount of Chinese but simply not enough to use Baidu Maps for all my navigation needs.
I found Google Maps (via Express VPN) to be perfectly fine in Shanghai. The public transportation suggestions and times were completely off, but locating a given address (my main use case in Shanghai) and using walking directions was never an issue.
In Shanghai you use Uber/Didi Kuaidi/Taxi or the Metro as opposed to driving yourself. As long as you can use Google Maps to look up the Pinyin spelling of the next major intersection so that you can pronounce that to a Taxi driver you are good to go. :)
Interestingly enough when roaming for free on T-Mobile (at 2G speeds) I was able to access Google Maps and most foreign internet services without the need of a VPN.
I'm an Apple user so I'm not very familiar with the offerings on Android. I don't drive. Apple Maps has all the public transport info so that's key to me for any map. Plus, in Suzhou where I live, I've found much of Google Maps to ridiculously out of date. It's gotten me lost many times before I switch to Apple Maps. Now when I go for a trip abroad I switch back to Google Maps and when I get home I switch back to Apple Maps.
> Interestingly enough when roaming for free on T-Mobile (at 2G speeds) I was able to access Google Maps and most foreign internet services without the need of a VPN.
Yup, if you are roaming on another a foreign provider you will rare have any troubles. Not sure why, but that's always been the case.
Never made it to Suzhou so I can't tell you how I would have fared there. However in Yunnan province (Dali, Lijiang, Kunming) I didn't have too many problems with Google Maps either.
I would have loved to use Baidu Maps all the time if there were a Pinyin mode. When I first installed the app I was shocked by the permissions it requested, but on stock Android I didn't really have a choice.
I do think international iPhone users generally have it easier in China.
Let's see what happens this year (or next) with Google resuming some operations in China (saw some job openings focused on Ads in Shanghai and Beijing).
> Finally I find it a bit odd that someone so concerned with privacy uses so many Google Apps. Isn't that rather counter productive?
There's another angle to it that I think is more important to many - not having access to Google apps you're used to is fucking annoying, to say the least. Bing is nowhere near good enough or convenient enough to replace Google for search and translations (and DDG is blocked). If, like me, you use GMail and Facebook as primary means of communications, then GFW basically cuts you out from family and friends.
> Also, only web based gmail is blocked. Access from Mail.app or iPhone's Mail is not blocked. Sadly translate has been blocked for about a year now... all the more reason to never turn the VPN on your computer or phone off.
GMail for Android (and Inbox by Google) are somewhat blocked. So is the Facebook app and Facebook Messenger app. I had periods late at night when my phone would suddenly download a few e-mails or receive FB notifications, but the "connection" lasted something like 30 seconds.
RE censorship, GFW is already effective enough, even if it's a side effect. See:
> There's another angle to it that I think is more important to many - not having access to Google apps you're used to is fucking annoying, to say the least.
For some Google Services there are annoying but functional workarounds, for others.. not so much:
My Galaxy S6 fingerprint scanner acted up and locked me out of my phone. I didn't really remember the backup password (I was too clever by making it very unique - somehow forgetting to put it into Lastpass). I thought - no problem, I can use the Google Account password reset mechanism that I used before, right? Wrong! When you are locked out you can't change WiFi networks either, so connected to an Astrill router or corporate network that is on a VPN wouldn't have been an option. So the only option was a factory reset. Turns out that a US Galaxy S6 immediately tries to phone home to some servers blocked by the GFW during the reset process. You can't skip that step! So this is where I had to find some folks with an Astrill VPN router and connect to their private WiFi network that was tunneled to the outside world.
Lesson learned: Know your backup passwords and don't rely on fingerprint scanner functionality.
> Finally I find it a bit odd that someone so concerned with privacy uses so many Google Apps. Isn't that rather counter productive?
There's a fairly big difference between having a company consolidate your personal information and a country willing to throw you in a hole for thinking certain thoughts.
Any good map solution for Android? I used Baidu Maps a bit but it was painful because I don't read Chinese. Everything else I tried (Bing Maps, Yahoo Maps) was terrible; searches couldn't find some of my hotels; directions couldn't give me subway instructions. I had to revert to using paper/PDF maps most of the time which was annoying.
"The OP is missing the point of the GFW"
Perhaps I did not explain myself well. What confuses me is why does the GFW goes to great lengths to very effectively block some VPN solutions (such as an SSH SOCKS tunnel), while it lets other VPNs operate (such as ExpressVPN). Or maybe this is a question I answered already at the end of the post: they can monitor ExpressVPN but not SSH, so they block the latter.
"Finally I find it a bit odd that someone so concerned with privacy uses so many Google Apps."
Simple: I am an ex-Google employee (Security & Privacy Engineering). I saw from the inside how serious and obsessed they are about privacy and protecting people's data, and this made me trust Google more than ever.
Due to the map issues. Uber drivers always call you to ask where you are. Which is a huge issue if you don't speak Chinese. Uber is powered by Baidu map data in China.
No, access to any form of gmail is blocked.
I just came back from Shanghai, my personal gmail on my phone was never able to retrieve mail, and even my work's corp. gmail was also unable to connect. However your mileage may vary. This was in Shanghai connecting to various wifis and using my sim's (China Telecom) data plan.
However you can use this to bypass the GFW.
free vpn app: fastcat
Initially gives you 50MB, I think if you register and check-in every so often you can get more free data to use.
I live in Suzhou. And use gmail via Mail.app and iOS Mail daily. Other than around certain anniversaries, it's never been blocked for me. I do visit SH regularly -- it's right beside me -- and haven't had trouble. But YMMV. The GFW might just like me.
I wonder if it might be because I'm using Google's Apps for Business (i.e., our own domain). In my experience the GFW is much more lenient if it thinks what you're doing is work related.
> It's mainly about providing a market for local tech startups and about keeping the lower classes from organizing and causing trouble.
Actually it's the other way around. It's built by Chinese companies as well as Cisco, Nokia-Siemens making a huge profit from Chinese security funding. GFW's funding is >$10B USD. The protectionism is the side product.
1. If you don't mind Chinese maps, Amap.com is also a good solution, listed on the Google Play store if you plan ahead to download. You are in China, use some Chinese maps! Translate / dictionary is helpful. Amap has many place names in English as well (Banks, airports).
Using Baidu Maps worked fine for me for public transportation directions / walking directions. Often I could search for a location in English or Pinyin as well.
It only gets tricky when you want to take a taxi. Sure - you could show the address to a driver, but I found that to be very awkward and it didn't always result in success.
So my major problem with using Baidu Maps is the fact that I cannot view addresses or locations in Pinyin. If I knew the characters I could pronounce it :)
FYI google maps in china is wrong because the chinese government wont let them fix it. You need special licenses and permission to display updated and corrected maps which the chinese government refuses to grant to google because google refuses to censor. If google updates the maps without permission the chinese government will just block their service entirely and issue fines to them. All maps are considered state secrets by the chinese government even simple road maps
A very interested and detailed writeup! I'm forwarding it to friends who just flew to China.
One thing I was surprised to learn is the GPS shift problem - particularly that China has managed to figure out how to encrypt map coordinates[0], so that they still look right with a proper map. It looks like something trivial to solve, so I guess the only reason a solution isn't widely deployed is because it would be illegal to use in China.
Another thing, slightly off-topic:
> I learned through this experience that the GFW is unmistakably able to exploit side-channel leaks in TLS, such as packet sizes in order to detect the "TLS within TLS" characteristic of secure web proxies. This really surprised me. I had no idea the GFW had reached this level of sophistication.
Come on. This is not sophistication. This is something a 10-year-old child could figure out if you gave him a set of connection logs and told to split it into two subsets. Looking at side channels is the obvious thing to do if you can't determine something from just reading it.
I'll grant that implementing machine learning for this task may be sophisticated, but just noticing an obvious side-channel (and writing a simple check for it)? And this is not the first time I've seen people calling side-channels or looking at metadata "sophisticated". Is it because 99% of programming consists of writing absolutely trivial spec-following code that doing anything that's not in the standards is "sophisticated" nowadays?
It's plainly not encryption. The Chinese maps use a different reference datum (ellipsoid surface representing the shape of the Earth), and translation between coordinates in the two reference schemes (GCJ-02 in China and WGS-84 for GPS) is straightforward.
Why Google and other mapping services don't apply this transform has more to do with business decisions and the hoops they have to jump through to be able to work in China at all.
There is an article[1] about finding out what location at google maps user is looking at, based on observing TLS packets and nothing else.
If task at hand is to match encrypted connection to a smaller number of alternative (say identify accesses to popular place, like gmail/facebook login page, etc), same technique of observing packet size distribution over time should probably work quite reliably, and cgiproxy will do nothing to hide it.
You don't necessarily want random padding; you really want every single packet exactly the same size, potentially with dummy data transmitted to mask data rate as well.
The best tools to evade GFW are `shadowsocks` and `openconnect` at this point.
The first one is a proxy, but unlike TLS proxy, it does not have a handshake phase. Rather, it encrypts all data with a preshared key, and the server rejects the connection if decryption yields invalid data. The stateless nature makes it much harder to detect.
The second one is a VPN over DTLS or TLS. Its security is even better than OpenVPN, as it supports ECDHE and AES-GCM from very early. No stunnel or obfuscation is needed at this point.
I travel frequently to China; the best solution that I have found, that works out of the box on handset and laptop is shadowsocks, pointing to a secure [ie; your own] server.
I recently used this in Iran with success as well.
won't that fall prey to some of the machine learning techniques China is using?
The next day, the packet loss returned. But if I simply
used a different port number for the proxy, everything
would continue to work fine for another day or so. I
think this time the GFW was not blocking me based on
side-channel leaks, but based on network metrics. 100% of
the network traffic to/from my server crossing the
Chinese border was to my public IP in China, so the GFW
probably learned my TCP endpoint was likely used as a
private VPN, as opposed to being a public HTTPS site
accessed by many client IPs.
As the sub-poster mentioned, shadow socks is designed to counter it.
What is interesting is that the author (mainland Chinese) had a 'visit' from the 'police', who 'asked' him to abandon the project. He changed the default branch of his GH project to an empty branch, but the code still remains.
Further, although his code is known, the GFW doesn't block its techniques.
When I was in China, I used ExpressVPN. But even with a fast VPN like that, it's still so much slower in mainland China than in Hong Kong or elsewhere. It really isn't a problem if you're Chinese and mainly use WeChat, QQ, etc. But us westerners who use Facebook, Google, and Instagram are SOL.
Great write-up, I had no idea that China was using machine learning to detect VPN setups...
I'd suggest the author make their Idea #4 random-padding code open source, but ironically if it became popular then surely the ML-based filters would eventually cripple it...
I went to China over the winter break. I'm a medical student and I used my school's VPN in China and it worked beautifully. Was able to access everything and the internet actually felt smoother/faster.
Addendum: I also pay for roaming with my T-mobile plan, and I got Edge (E) network through (I think) China Mobile while I was in China, and interestingly enough, the internet through my mobile data plan was not censored at all in China.
As someone who crack (or at least block) VPN protocols for a living, I can indeed confirm this is 1 of the tricks used by all deep-inspection firewalls to detect VPNs.
In fact, a very popular VPN software for Chinese citizens uses TLS-within-TLS (sometimes fake TLS) to hide its data. From the author's description, the traffics are similar enough IMO for the GFW to detects.
(No, I don't work on GFW, but we block similar software. Sometimes we do comparison between various products to see how others block them.)