For people wondering how the hell a user can audit the server is diskless or whatever, the goal appears to be using TPM to provide remote attestation for all code in the boot path. See https://www.system-transparency.org/.
Here are some additional details for those interested. We intend to make use of TPM for remote attestation of the current boot chain, reproducible builds to provide a strong link from source code to build artifacts, and a transparency log for a historical record of previously used boot chains, artifacts, WireGuard server keys, and related signatures.
As dtx1 mentioned elsewhere in this thread, diskless VPN infrastructure is currently in use by many other VPN providers. That is not a novel feature of course. What is novel is user-auditability of running VPN infrastructure. We were the first VPN provider to state our intention to make our infrastructure user-auditable AND provide a realistic roadmap with the specific technologies needed to do so. See the link above.
I believe the technologies we use in System Transparency will ultimately reshape the VPN provider industry into a highly competitive space focused on maximizing the transparency of VPN infrastructure. Or not, but at least OUR users will be able to audit us. :)
Either way we’re looking forward to the future. The opportunity for improvement is immense.
Niiice. I really love the concept of reversing the usual DRM use of remote attestation--forcing customers to prove they're running only software allowed by the megacorps. Instead of DRM, it's proving the corporation/server is trustworthy to the customer.
Check out tpm2-totp. I stumbled across it while looking for a way to store totp secrets in my tpm, and was really impressed with the clever use of totp to verify a boot chain.
If you're from Mullvad, let me tell you that I love your service.
I had one technical support question, and I got an immediate response from an actual person who knew this problem, gave me a simple workaround (toggle between wireless connection and not), and told me that a permanent fix was in the works. Likely that fix rolled out because I haven't seen the issue in months.
The idea of an account that doesn't need a password because there's no critical information saved is such a nice one, too.
Keep up the good work - I recommend you to everyone.
This all sounds very exciting. Where will you draw the line between public and private – at the moment your consumer-facing app is on github, but less "server side" stuff (in common with many other VPN providers). I understand that probably you want to keep the database of "active numbers" private, but if I understand you correctly, you want to move to a model where anyone can download your in-memory image, run it in a VM, and audit it independently. I would welcome this. I'm particularly interested in how you maintain access to your bare-metal machines (e.g. do you have ssh / a serial console enabled)
> Where will you draw the line between public and private – at the moment your consumer-facing app is on github, but less "server side" stuff (in common with many other VPN providers).
All source code for all software on our VPN servers must eventually be public, and all build artifacts must be reproducible by 3rd parties.
> I understand that probably you want to keep the database of "active numbers" private, but if I understand you correctly, you want to move to a model where anyone can download your in-memory image, run it in a VM, and audit it independently.
Exactly, but we will also have to measure each artifact in the boot chain into the platform TPM, and allow anyone to issue a challenge to the TPM to get a signed quote of the boot chain measurements.
> I would welcome this. I'm particularly interested in how you maintain access to your bare-metal machines (e.g. do you have ssh / a serial console enabled)
We’ll have to constrain our own ability to access the VPN servers. We cannot be allowed arbitrary root access as that would make the TPM measurements meaningless from an audit perspective. Well, you’d be able to conclude we have root access, so not totally meaningless.
> Isn't network monitoring and logging the bigger issue for a VPN service?
Great point. I have a hard time comparing the importance of the integrity of our VPN servers with monitoring and logging of network traffic happening upstream of them without a long discussion of the many nuances. Let’s leave it at; they are both very important issues.
> How can you provide transparency of the network?
That’s very hard. Individual AS’s upstream of the VPN server you’re connected to might change their routing at any moment, and suddenly your traffic makes its way through an unexpected AS or jurisdiction, which may change the monitoring situation completely.
Enabling our users to use multiple hops is one mitigation, another is vetting our data center providers. In collaboration with some of our Internet providers in Europe we have deployed protections on layers below IP. Come to think of it, I don’t think we’ve blogged about that. Thanks!
I hate to be this skeptical, but let's say this is 100% possible (I have my doubts, see previous attacks on things like TPMs and SGX, but I digress). You probably could get 90% of the logging capability by putting monitoring in front of and behind the server, and associating connections by traffic/time.
It just seems like this goal of using technology to prove they're trustworthy is unlikely to actually work for a VPN company due to the threat models.
You are correct that System Transparency is not a universal remedy for all threat models. Indeed the word "secure" is undefined until you have a threat model. Most threat models are implied and undocumented assumptions.
At some stage in an R&D project one should shift from exploration to threat model-driven development. Most people, myself included, tend to focus on technical solutions, and argue back and forth how "oh, but it can be broken using X".
System Transparency aims to provide remote auditability assuming (1) the server hardware specification is correct, (2) a correct cryptographic hash of the contents of the SPI flash containing the platform firmware, and (3) a keypair generated on and only accessible to the platform. This is very simplified of course.
An attacker aiming to tap incoming and outgoing network traffic from our servers, who has physical access to the VPN server's Ethernet port, or an upstream router, isn't in the scope of System Transparency to protect against. We need to use other means for that.
Traffic analysis and correlation analysis is indeed a powerful tool, and in general only communicating at a constant bandwidth between all nodes at all times is the only way to completely defeat it (which is what, I understand, some military systems do). That's inherently highly wasteful, however.
To get around this, Mullvad offer very transparent comprehensive multi-hop routing systems [1]; you can bounce your wireguard tunnels around in layered wireguard tunnels (a bit á-la tor) by just choosing a series of ports to tunnel on and to. My understanding is that each one of these adds non-deterministic latency to your connection and probably would help to make such attacks harder at the very least, because from the point of view of an "all seeing" adversary the fact that all of these servers talk to each other all the time makes it very much harder to know where any packet could have gone. Yes, you can see each individual link but the metadata is lost.
I signed up for Mullvad when the UK's Snooper's Charter came into force and the local health inspectors suddenly had the rights to see my DNS record. Since then, I've had it installed on my router and just route everything through a custom wireguard (originally openvpn) tunnel. I've had some issues with my ISP randomly bandwidth limiting traffic on the odd port to 1 MByte/s, but frankly that makes me more inclined to put everything behind an encrypted tunnel. I don't want my ISP to do traffic shaping and I do want them to just leave me alone and let me communicate in peace. I have absolutely nothing to hide, but now have to accept that I partly live in a country where everything is surveilled all the time, and warantless, unaccountable investigation of my (highly personal!) online habits may be happening. I think Mullvad's excellent product, sensible architecture and reasonable price is worth paying. I'm an academic, unlikely to be of interest to three-letter acronyms, and therefore it matches my needs very well.
System Transparency reduces your trust assumptions on us. As a VPN provider we are in an immense position of power over you. We aim to reduce your trust assumptions on us to a few things that we would need to explicitly lie about in order to betray you.
As an example, let's say that we offered any of our users to at any time during the year show up at our office and inspect our VPN hardware, without warning us beforehand. In that situation, if we wanted to betray your trust and privacy, we would need to put in a lot more effort than if we said "We have secure servers. Trust us on that. No you can't see them.". Does that make sense?
> In that situation, if we wanted to betray your trust and privacy, we would need to put in a lot more effort than if we said "We have secure servers. Trust us on that. No you can't see them.". Does that make sense?
Have you ever thought about doing something like that with some big youtube personalities? Maybe have them hire some pen testers, randomly show up to one of your datacenters, and post recordings of what is done and attacks that could be possible. Since your software is open pen testers could prepare some things to try to attack days in advanced. I'd love to see something like this with Level1Techs or something.
Damn it, this is a really cool use case for TPMs! So far, every use case I've heard has made me wish they were never invented, but this made me reconsider...
