Nice work! Always fun to see something I wrote long ago reverse engineered. The packet format was indeed inspired by ESP over UDP, and I named it XSP. After system link shipped with the original launch of the console, I also worked on Xbox Live networking, including the client/server interactions and the design and implementation of the front-end Security Gateways that all Xboxes would talk to, first to authenticate themselves to the service, and then to maintain a heartbeat connection to the service (to keep NAT ports open during idle time), and to facilitate NAT traversal.
Nice! You did a great job on the protocol. Probably my only complaint on the XSP side of things is the fact that you have to do relatively complex parsing of the XSP packets before you can get to the point of verifying the signature of the packet. Seems like all of the corner cases were handled well in the implementation on the boxes, but as someone who does auth/cryptography in my day job, it kind of gives me the heebie-jeebies.
Do you know if the auth side was carried into deeper parts of the backend? So like, did the SG decorate incoming connections with the auth information as they made their way to the different services? There seemed to be more auth information than I expected in headers on some of those HTTP calls into services like matchmaking.
That's a valid point about complex parsing. I remember being very concerned about adding unnecessary overhead to each packet during encapsulation.
As for the SG, it primarily authenticated the Xbox machine account using Kerberos and then maintained a security association, accepted heartbeats, authenticated and decrypted incoming ESP-UDP packets into IP packets that it forwarded to the backend servers. Responses from the backend would be encrypted, authenticated, and encapsulated before sending back to the Xbox. I don't think the SG had any knowledge of higher level connections running through it, such as TCP or HTTP, so it would not have manipulated HTTP headers as they passed through.
Ok, cool. That's about what I figured at this point. Originally while REing the protocol I thought that it was holistically handling auth at that XSP layer, but then was surprised when a box would the identify it's XID to matchmaking as well, which should have been stored in the krb ticket to bootstrap that connection.
Thanks so much, I really appreciate your candor here!
The SG had to do a few TCP-level things for NAT purposes like rewriting checksums, and it would sometimes synthesize a RST. No layer 7 processing at all
There was a low level protocol allowing backends to get some extra metadata about a connection
So that's basically most of the lowest layers of the Xbox Live protocol as well. Simply game discovery happens over local broadcast instead of through the Live servers. There's some other specifics that change, such as logging into a host xbox doesn't diffie-hellman anymore but instead key sets are distributed by the server backends with the session information. Additionally the auth side is basically non existent on system link.
> So that's basically most of the lowest layers of the Xbox Live protocol as well. Simply game discovery happens over local broadcast instead of through the Live servers. There's some other specifics that change, such as logging into a host xbox doesn't diffie-hellman anymore but instead key sets are distributed by the server backends with the session information. Additionally the auth side is basically non existent on system link.
Ah, cool, there has been some similar work done before! Is there any documented resources or write-ups of the Xbox Live protocol somewhere?
> Great RE work!
Thanks!
> Disclaimer: I created a proof of concept implementation of the Xbox Live server infrastructure here: https://github.com/xombieonline
That is a really cool project! How does one make sure the console connects to the server instead of the (now shutdown) Xbox Live servers? Does it e.g. have a hardcoded domain so one can simply add a DNS entry?
> Ah, cool, there has been some similar work done before! Is there any documented resources or write-ups of the Xbox Live protocol somewhere?
I talked at defcon a bit last year. https://www.youtube.com/watch?v=HLyZfZMu-5E Otherwise I've been a "source is documentation" kind of guy so far. Could definitely use a high level walkthrough as good as you've done here; I should probably get on that.
> That is a really cool project! How does one make sure the console connects to the server instead of the (now shutdown) Xbox Live servers? Does it e.g. have a hardcoded domain so one can simply add a DNS entry?
Yeah, they bootstrap off of a set of hardcoded domains. The config screens on the boxes let you specify a DNS server, so the project runs it's own DNS server that's basically when obi-wan says "of course I know him, he's me". Then the relative lack of any public/private crypto lets us take over as long as we know the preshared key in the individual xbox's eeprom.
> "age in a bot" might be a truncated form of "message in a bottle."
Yeah, that makes sense, it is a very odd sentence otherwise. Truncating on "bot" might be a play on words for game AI/NPC.
> The PRNG exponentiation scheme is essentially Diffie-Hellman.
Ah, of course, yes, I was thinking it reminded of public-key cryptography.
> Including, e.g., plasma pistol?
Yeah, it seems so, that was the first weapon I was thinking it to be used for. I tried e.g. setting the duration to zero and charging the plasma pistol. It still shoots like normal and does not seem to affect visual effects, audio or damage for neither host nor guest. It is still possible that it is used for something else that I have not noticed. Modifying the host's fire duration does not cause a desync so it might not affect the game state.
People have determined what the player structs are on the xbox-memory-state side of things, so my guess is "age_in_a_bot" might be the time elapsed for a given player aka how long they've been alive:
Huge fan of Halo 1, and for me, the original (via LAN, XLAN or Xlink Kai) is still the preferred way to play compared to Halo MCC (RIP XBconnect). I dabble in map mods so this is mostly over my head, but interesting read.
Don't know if would help your research: but are you aware of a mod/mappack called Halo 1: NHE (No Host Edition)? It's a hacky way of using a third box to host the system link game.
4 year old account, and this is your first comment? :D You might be interested in super custom xISO with NMP, NMPv2, CXE, etc. I mentioned it in another comment. I'm going to upload to Internet Archive soon.
This is officially my favorite thread on HN, ever. Just waiting for rothgar to show up, sorry @rothgar, I should've hit you up for some Halo while I was in Seattle (and the k8s space).
NHE looks interesting, they seem to have modified all clients to make the host player teleport above the map. With the current protocol, it seems like it is impossible to achieve this without modifying the guest clients as well. And by modifying all clients they are also able to add extra maps and make gameplay changes.
Thanks, I was initially surprised that they were encrypted but I can speculate on a few potential reasons:
- The Xbox was designed to be able to play online via Xbox Live (e.g. with Halo 2 that came out later in 2004), and they might have simply reused the network stack for System Link over LAN. I looked a little bit at Halo 2 system link, it uses the same system calls from the kernel but the protocol seems to be more complicated (e.g. the IV is never sent in plaintext, the two consoles derive it somehow). I haven't looked at Xbox Live, but Halo 2 could potentially use the same for System Link and Xbox Live.
- The Xbox also really tried to lock down its security in order to prevent game piracy and homebrew games. This might have been another attempt to reduce the attack surface.
- They also might have just wanted to keep their game protocols secret for trade secrets or simply avoiding scrutiny. In this case, we were able to create a kind of cheat, which they might have also wanted to avoid. Even though it is rarely an issue at LAN parties, it might have simply looked bad for the brand.
From what I've heard from microsofties over beers, it was really the second one. They didn't expect developers to write network code in ~2001 that didn't have memory safety vulnerabilities and wanted a barrier there between simple overflows and system security.
The child in me from 20 years ago dreams of a software tool downloaded from a questionable website that jailbroke an xbox just by putting in the IP address.
I mean, minus the USB->controller wire "hardware mod", it has been perfected to "download a memory card gamesave", "have a non-GOTY edition of a game" and you were a few minutes away from being soft-modded, ready to hack your cached Halo maps to swap the needler texture in for the wall textures, and change character textures to brighter versions (gummibear hack, maybe?).
Makes me want to wipe the XBMC xbox in my parents basement and break out my Splinter Cell and memory card just to do it all again.
Thanks for the answer. Like 5 years ago I tried snooping around halo system link packets because I just assumed there was no encryption and was disappointed. Wasn't this multiplayer rushed out in a month? Why on earth is a LAN protocol secure in 2001?! lol
Makes total sense, especially recalling the zeitgeist of the era. IIRC, this was during the height of worms which propagated via vulnerabilities in code that handles network traffic.
Probably for the reason the author speculates about in the article:
> For example, would it not be cool if there was a game with a stack buffer overflow bug that allowed us to run arbitrary code remotely? That could potentially enable us to softmod an Xbox over the network, without the need of any special hardware.
As someone who's written replacement backend Xbox Live server infra, I can confirm that system link is very nearly the same base protocol set as is used by Live. They must have had that nailed down pretty close to launch, and were mainly working on support services that live needs but system link doesn't up until Live actually launched.
The game was actually close to ship without multiplayer, it was an afterthought. Some devs have talked about this (iirc, Marty O’Donnell was one of them).
Goldeneye is one of the best case studies for why companies should give their engineers room to experiment. If I remember correctly, the multiplayer mode was originally a side project of one of the engineers and was added near the end of development.
This isn't to say the campaign mode isn't good, but the ability for boys to play a shooter game against each other on the same console is what made that game.
It's a shame it's proprietary, but there's a neat service[1] that's been around for over a decade that reverse engineered System Link and similar protocols to make them Internet-enabled.
They don't really need to reverse engineer the underlying protocol for that service. It's enough to simply bridge the networks in a way that local broadcast still gets through.
I haven't tried this out, I wonder how well the Halo 1 net protocol works over a less reliable connection. I encounter issues as soon as I momentarily minimize the emulator and it does not recover without starting a new game.
I wonder if it is enough for XLink to simply send the packets through a networked tunnel or if it actually needs to modify the packet payload somehow. The consoles might be able to handle everything as long as they are able to communicate with each other?
I played competitive HaloCE through 3 at a professional level. XBConnect is what people playing competitive CE used prior to halo 2 and xbox live. Game was playable up to about 60ms, with <40 greatly preferred. Hosts had a large advantage, but off host was very predictable, and required leading the targets.
The same advantage existed on LAN, with competitive tournaments setup to trade host advantage for team color advantage. The last in person CE lan I attended, we were playing on modded Xboxes with a patched version of the game that allowed a third box as a neutral host, and added an on screen timer. Happy to expand on the details if anyone is interested.
It's just branded network tunneling software. They went through the effort of making friends lists and integrating IRC channel into it, but if any two people want to play each other there are a myriad of ways to tunnel local network traffic over IP. Here's one by the guy who created xemu:
Tangential point: the weirdest part about the Xbox was the system name. As far as I'm aware that was never configurable by users, but certain games would set your system name as an easter egg. Any reason why it was built that way?
Halo 1 + system link + XBConnect is responsible for a huge swath of my career, life, friendships. I remember sitting in my parents basement as a [very young person] marveling at XBC and wondering how it worked. I can still remember the layout of the webpage I found diving into the protocol, how discovery worked, broadcasting discovery packets to 0.0.0.1. I remember opening Ethereal and seeing the packets. I remember, a bit later, hacking together a Java app, failing at a C++ app, many years later trying one in Go and then... pre-drop-@-gc Rust. Wonder if instead of encapsulating packets with src/dst mac I could send a map back and forth and index into it to save the whole... 6 bytes?
So many hours of my life spent playing that game. Even recently I re-discovered an old Halo (x)ISO I mastered in high school containing a multitude of map packs that the community had made for it (NMP, NMPv2, CXE, +??). I even hacked them to change their internal map IDs to prevent cache conflicts when switching packs. My friend dug it out of his collection, copied the ISO and I fired it up in XEMU. Wild to see some of those maps, that some random people made with hacked together tools, and wild to read this now, and the comment from /u/dinartem. Even wilder that it's playable emulatable now. Especially given the way MCC massacred Halo1 with the horrendous Halo 1 PC port back to Xbox, and then later to PC again.
I'm horrified to see someone comment that multiplayer almost didn't launch with Halo 1. My life would be so unimaginably different.
So many memories, this comment doesn't mean much, but what a thing to see on HN.
lol, I'm almost tempted to drop my XBConnect Forum name here. I remember when I thought Todd was an absolute god among humans. Oh man, thank you HN for the dose of nostalgia. If anyone remembers a huge block-letter forum signature that was briefly animated ;). The era of sprawling PHP file upload sites. Wow. The internet before it became truly cursed.
edit: shout out if anyone knows what I mean by "clear walls". Oh man, what a world.
editN: oh wow, "cross over cables" is a phrase I haven't thought of in a long time.
editLast: there was a glitch that was supposedly reproducible that caused a tertiary console's player to override the inputs of another console player. Afaik it was never widely discussed, despite repeated claims that it was reproducible on demand. If anyone has any details, you'd make this a truly magical thread for me. <3.
For your last edit, I've only seen that glitch twice: once in a youtube video [1], and once in one of my own games on XBConnect. I heard similar unsubstantiated claims of intentionally reproducing the glitch, but didn't dig too far into it myself.
>Especially given the way MCC massacred Halo1 with the horrendous Halo 1 PC port back to Xbox, and then later to PC again.
MCC launched in a terrible state, but it's had a crazy amount of deep improvements over the years and I think all of the big known issues of the original Halo PC port have been addressed since a few years ago.
No, it really hasn't. https://halobugs.com/ I have hundreds of hours in Halo 1 and play MCC regularly. Halo 1 PC players don't understand because these bugs originated in the original Xbox->PC port, and they've never known what it should actually feel like.
I mean hell, when they first launched MCC, fall damage jumping off the base in BG was wrong. You can feel the effect of the Gearbox port to this day. I mean, did they even fix the pistol spread issue? Not that that would be important or anything.
You're speaking my language friend, and honestly half of this reads like I could have written it myself. I was obsessed with video games and knew I wanted to be a programmer before CE, but H1 on XBC, combined with LANs in high school and thousands of hours of split screen in college truly laid the passion for "how does all this work" and my current career.
I don't play CE as much, my doubles partner and I still hop on MCC every few months for a little nostalgia hit, but we're admittedly not the types that felt that Halo completely died after CE (although nothing ever felt as good after), so these days we generally play BTB Reach/H3, or 4s or Squad Battle on Infinite.
You just put the absolute biggest grin on my face. HN was all worth it, somehow. And wow, am I not surprised to hear Squad Battle is your favorite. Somehow I'm sure we've played some games over the years.
I recall the game recording feature feeling pretty cutting edge at the time, as you could pause it and rotate/fly round the scene, it felt really fluid and responsive. I guess now it would be common place but back then it definitely seemed a little magical.
Are you thinking of Theater mode that first appeared in Halo 3? Unfortunately it was slimmed down in later Halo games and it is rarely seen in other games at all, especially not console games.
This was common in PC games at the time, but not on console games. Essentially you're just dumping game state memory somewhere and then replaying it. Half-life games had it.
Loved it, thanks for being so thorough and document the train of thought.
I want to ask about one thing I could not understand completely on the final section: If there was a client that sent arbitrary values for selected weapon, forward, left, etc; would the host count them as valid? (I understood this is essentially what the MITM allowed to do)
Also, a little feedback, my immersion broke when the video did not show Howard and Ghost anymore. Something like Howard1 and Ghost1 would've helped understand a little bit more.
> Loved it, thanks for being so thorough and document the train of thought.
Glad to hear! I was hoping to make the whole process easy to follow without any large skips or prior knowledge. It is more or less my process but with a lot of dead ends removed.
> I want to ask about one thing I could not understand completely on the final section: If there was a client that sent arbitrary values for selected weapon, forward, left, etc; would the host count them as valid? (I understood this is essentially what the MITM allowed to do)
Yes, it appears so. The host seems to accept more or less arbitrary values (at least for forward, left, actions) and also just re-broadcasts them to everyone. The guest client then also ignores what it sent and just uses what it receives from the host. This allows us to modify the inputs/speed of any guest client players without desyncing the clients.
> Also, a little feedback, my immersion broke when the video did not show Howard and Ghost anymore. Something like Howard1 and Ghost1 would've helped understand a little bit more.
Good idea, I could have made it clearer which perspective the video was taken from (the guest client). Might be able to re-record or simply mention it in the text.
