As someone who reverse engineers things for fun, and can read the C "pseudocode" generated via decompilation pretty easily, I am going to have to disagree with the assumptions made in this post.
First, there's no proof this is from Steam, I've poked around a few of the DLLs since I saw this and am unable to find anything even remotely close to what this does.
Second, this method does NOT send anything to Valve. This method grabs the DNS cache, yes. And it MD5s the entries, then it stores it. This method itself does nothing more with the hashes. For all we know VAC could be doing a LOCAL scan of the list, and comparing it to an internal list of "known" cheat subscription servers.
Until someone posts details of exactly where in Steam this is (What DLL is all that's required to verify), and the calling method that supposedly sends this information to Valve, I would take this with a very massive grain of salt.
So yeah, no proof it sends anything. It could be a local check.
>I've poked around a few of the DLLs since I saw this and am unable to find anything even remotely close to what this does.
This is part of the Valve anticheat system. I believe that the idea is that this sort of code is downloaded on the fly by the game client to make it harder to reverse engineer by game cheaters. What you have to do is to find some way of dumping the code that VAC pulls from the steam servers, and of course Valve tries to make that difficult for legit reasons (regardless of whether this particular snippet of code is legit).
VAC streams the DLLs from their server at runtime, so they never actually sit on the disk. There's also a number of DLLs which are loaded and unloaded as necessary, to make it harder for attackers to dump the whole thing. I suspect that there are blackhat forums where people have discussed how to dump parts of VAC.
I suppose it would just be the matter of intercepting the network traffic going between the PC running steam and the internet and see what goes through the pipe?
IMHO the fact that they seem to store the hashes in a lookup table hints that it's meant for local use, what's the point of doing that if you just want to upload it on the net? I guess they could do both though.
btw, help me understand the use of the DnsFree variable... why is it often exor'd with 0x23dc67e8? is that the addr of a routine and exor is faster than adding? thank you.
That's just in-memory obfuscation. The value is xored with 0x23dc67e8 when set and the variable xored again whenever is used. Its first use is to check that the symbol was resolved correctly, that implies testing for NULL/0. The compiling-decompiling process made it so that it is actually just comparing the variable to the mask.
"
Yes, with some simple wireshark analysis you can see it is being sent back.
Use wireshark, join a local TF2 server, try and isolate the VAC IP address (they are not static, but use rDNS & whois the IP). Go by process of elimination. Happy to give you pointers if anyone is interested.
Use wireshark and monitor the SSL communications of VAC for the first minute. Record the total size of outgoing packets (for me, I got 1.94 MB and 1.88 MB on my two tries -- the first time you join a VAC server and when modules update it's likely to be higher as it downloads it's modules).
note: Keep everything else constant - like what windows you have open, what processes you have running, etc.
Bloat your DNS cache. (What I did was edit my hosts file, used a script to add over 20k hosts [careful actually crashed notepad when I tried to read it])
Repeat step 1 and 2. I got 2.47 MB and 2.58 MB on two tries (first min of outgoing packets). This increase seems to be twice the amount 20k of MD5 hashes would take. Maybe a bug is causing it to be sent twice?
Clear your hosts file, flush dns cache. Repeat step 1 and 2 again. I got 1.99 MB."
Wouldn't it be possible to MitM the SSL traffic by using a self signed certificate? Does Steam use its own CA list or does it use the one from Windows?
I'm pretty sure this is done to combat cheating. Seems pretty clear cut to me. They likely don't want to do the domain check locally, because then the cheaters would know the hosts that are banned.
The intent doesn't matter. It's still reporting essentially your browsing history to them. Especially bad because the cheaters have caught on almost immediately (even worse that its the only reason its come to light that it is happening).
Anti-cheat protections use exactly the same methods rootkits use to "monitor" your system, it's one of those things we grudgingly accept when playing multiplayer games.
It's not even what Valve and co are doing with the info, it's similar to the Sony rootkit case where the main worry would be that the rootkit could introduce security vulnerabilities.
VAC has never been particularly effective. This check isn't particularly useful as it doesn't actually prove that a player is or has ever cheated. It's also trivial to bypass (`ipconfig /flushdns`) if you are an actual cheater who frequents these websites.
To be banned because you visited a website is also an abusable medium. Similar to the memory and window name scanning VAC and other similar anticheats have done in the past (or continue to do). You could historically get people banned by just doing stupid things, like having an IRC title that shows up in mIRC's window name that's detected or sending someone a message via any chatting medium that has a detected substring.
Imagine you just send a tinyurl link to your opponents in an upcoming tournament. Or just embed a simple <img> tag in an otherwise harmless webpage (display: none, for example) so everyone who visits your website has that domain fresh in their cache. This kind of "evidence" quickly becomes useless because it can be used for nefarious purposes, which is why it should never be used at all.
This is not only ineffective, it's dangerous. And it's an invasion of privacy since you can simply create a rainbow table of domain names, as other people have pointed out.
Sounds like they're just checking if you've visited known cheat sites. Unfortunately the DNS cache is a stupid way to do this since it's open to abuse. Modern browsers will prefetch DNS, so all you'd need to do is put a URL to a known cheat site as a comment somewhere popular in order to get users flagged. Even easier for forums that allow image hotlinking.
You could pull the same trick with Battleye, too (it scans all processes' memory for a short list of signatures, too, and several months ago those were leaked).
If their goal is to see where it's customers are going, hashing the data with MD5 is a very strange way to go about it. Sure you can break the encryption, but unless their favorite activity to do is run expensive compute farms, they're not going to bother.
I assume if that's the objective they just maintain a list of domains they are looking for and match their hashes to the ones they fetch from the users.
I guess it's a way for them to pretend they're not actually invading the user's privacy, just looking for certain websites. That's pretty weak though.
Why can't the MD5 be to protect their own list? They have a bunch of urls they want to block. They don't want to share the list. They md5 each entry on the list to prevent trivial discovery of these urls.
That's a good point actually, if the check is done locally I'd be curious to know which domains they're looking for. If someone could get the list of hashes I'm sure it wouldn't take long before someone manages to bruteforce them with a rainbow table.
Assuming it's a list of cheating-related websites you wouldn't even need a rainbow table, you just post the list of hashes to a cheating forum, have forum users compare their DNS entries and post the hits.
Well, look at it this way, if you were a programmer for Valve looking to solve that problem, wouldn't you think MD5 is decent enough? A stronger algorithm + salt would be slower for no real benefit.
I meant it was a "weak" excuse, you're right that a stronger algorithm wouldn't change much.
In fact, after reading the code and the rest of the thread I'm starting to believe it might be for obfuscation rather than protecting the user's "privacy".
Unfortunately the space of all possible web addresses is much (MUCH!) smaller than the space of possible MD5 outputs (modulo the deep web). This means it is much easier to reverse. It just means iterating over the ~billions/trillions of incriminating pages you want to search for and collecting hits rather than brute forcing the MD5.
Or they have a hash dictionary of domains they're interested in whether or not you've visited. That is, maybe they don't care about you're midget porn habit, but do care if you visit a competitors web site.
I think Ubuntu doesn't do any DNS caching by default (unless you manually install something like dnsmasq), so they probably figured it's not really useful on that platform.
Can someone explain this to me like I'm an idiot? Is VAC only applicable to certain games? Is it running the entire time Steam is running? Does it watch what websites you visit outside of Steam or only with the Steam browser? Does it do so even when you aren't playing a multiplayer game? If you use private browsing in a normal browser is it still watching? Etc.
VAC hashes all DNS entries cached by your OS (obtainable via ipconfig /displaydns) and sends them to their servers.
This would affect any page you've loaded in your browser (as long as it's still in the DNS cache), as well as any other application resolving hostnames (or well, any application using getaddrinfo and the like).
Yes, this would also include sites opened while using private browsing, since the DNS cache doesn't get flushed afterwards (at least a Mozilla bug report said so.)
I believe most Linux distributions do not come with a local DNS cache by default, so I'd guess they didn't bother implementing something like that in VAC's Linux port.
OS X does use a local DNS cache AFAIK, so it's possible. The decompiled code is Win-specific, so I can't really answer that without the decompiled code for their OS X VAC port.
I hope this isn't true, because I like Valve... IANAL, but I can only imagine this is quite illegal in the EU. No terms or conditions could possibly waive your rights to this extent.
Maybe that's just my optimism speaking though. Every good company turns bad at some point.
I am not expert, but in most cases you need permission from data protection agency. I think it is pretty similar to LG TV uploading list of files on local disks.
If they're sending hashes of the data rather than the data itself, they could quite easily argue that it's not "personally identifiable information" (remember that data protection only covers certain things).
On the other hand, if they want to know if you have visited a certain site, they can just hash the domain name and compare it to the hashes that were sent over, so the privacy is just illusory.
Yeah, for my company I have to comply with some serious regulations over what I can do with customer data, what I can collect, etc... I can't imagine that ripping a customers entire web history is anywhere near compliant.
When I think about how much work I do on home machines with Steam installed, I can't even begin to imagine how much of a security breach this could be at larger companies.
If it's in the EULA? I didn't check, though it crossed my mind, fixing a tap [faucet] instead.
http://pastebin.com/PTG57bXb is the .deb and the online "Steam Agreement" - section 2A of the later is probably most relevant from a brief look through (unless they've hidden things under non-relevant headings). That requires you to give them rights to run "the software".
There are parts about their rights to your UGC, but that doesn't seem pertinent really.
Nothing seems to be a warranty from you as user to Valve giving them rights to snoop around your system and upload data they find.
I can't see any evidence that the hash is sent to any server. I think it is more likely the hash is used to look up the recent websites in a hash table or via a bloom filter.
Makes sense to me. For example if you play warcraft 3, and you visit Shadow French all the time. You are most likely a hacker. Obviously that doesn't seem to be the only check they do. But it's a very easy one. And they would only need to keep a list of common "game hacking" sites and check it.
So far there seems to be zero evidence that this is used to send data to valve.
I don't think this was made to capture people just visiting cheat sites. Elsewhere I've heard people mentioning that this functionality exists to detect a new and evolving set of cheats wherein you enter credentials into an innocuous-looking executable and start up a game. In the background, this executable connects to a cheat distribution server, authenticates you, and live-patches whatever game you're playing. The DNS hosts they're looking for are these endpoints.
Is this a reasonable way of detecting cheats? In my opinion, yes. They can't send the hostnames to the client (even in hashed format), because then the cheat authors could see if their hostname(s) are listed and subsequently change them, even if the list sent to the client is hashed (they'd just have to run their own hostnames through the same hashing function). Having my DNS cache sent to VALVe and used (likely ephemerally) is a small price to pay for multiplayer games that I enjoy to continue to be fun.
It's a terribly ineffective way of detecting cheats - all the executable has to do is carry out its own DNS lookups directly without going through the cache and Valve won't see anything amiss.
So, now you can be banned by Valve because of a thought crime? Shouldn't you actually be cheating to be kicked off the network, instead of just reading up on it? What if you visit a Warcraft 3 fan site that has href to images on Shadow French or other thought crime sites? Won't that show up in your DNS cache as well? (Answer: yes it will)
You wouldn't even have to consciously visit a cheating site to have it in your DNS cache. For example: someone could embed an image hosted on said cheating site in their signature on a popular gaming forum and you'd have it in your cache
Honestly, I'd probably believe this is just a method for validating sanity of DNS for debugging/troubleshooting, as there are far better ways of going about doing almost anything you could think of as a malicious use of this.
Why would they include something like that in VAC, instead of the Steam client? Besides, there are better ways of validating DNS sanity (just look up a known hostname controlled by Valve and compare the result to what you expect it to be). What if I just flushed my local DNS cache? Would VAC think my DNS is broken?
No, we can safely assume this is done to compare your DNS cache to a list of hosts known for their involvement in some kind cheating (be it a website distributing cheats or some kind of cheat connecting to a certain host.)
Even if Valve doesn't intend to use this for any other purpose (and I honestly don't think they are) doesn't mean that a) they won't change their mind later on and b) their infrastructure is perfect, meaning someone could steal this information.
The above will manually clear out the windows DNS cache. Note this doesn't block it, but rather simply limits what it can find out. I'm not sure it would be possible to block this functionality entirely without also breaking DNS, or otherwise causing the Anti-cheat tool to detect something odd.
By default, the Windows DNS cache doesn't flush itself, with entries lasting for 24 hours or so. As stated in the post, you can view currently cached domain names by entering
ipconfig /displaydns
Strangely, this doesn't seem to list some of the sites I've visited this morning within firefox
At which point the entry would still be cached by the OS, and since firefox apparently uses the OS API to resolve hostnames (if it didn't, I wouldn't have found the entry in my cache after opening the site, right?), it would still return the cached result (as long as it's not expired).
Besides, the question was how to prevent VAC from uploading your (hashed) DNS cache, and clearing Firefox cache doesn't flush those entries from your OS DNS cache.
First off, in that case I'm not sure what your experiment was supposed to demonstrate.
More importantly, Windows will cache DNS records for no longer than the TTL. Firefox will keep entries in its cache for hours if not days. That's how sites you have been visiting will not show up in the OS cache.
OP asks if it's possible to block VAC from leaking your DNS cache (specifically the OS DNS cache obtainable via ipconfig /displaydns).
blueskin_ mentions that Firefox uses its own DNS cache, and deleting the browser cache also deletes the DNS entries.
However Firefox still uses the Windows API for DNS resolution, and deleting the browser cache doesn't result in those entries being flushed from the OS DNS cache (which is demonstrated by my experiment.) In other words, simply deleting your browser cache in Firefox will not prevent VAC from uploading your OS DNS cache. As long as the entries aren't expired or flushed manually, they will remain in your OS cache and VAC has no problem getting them.
kamakazizuru asks if it's possible to block the leak
mdisraeli answers how, and remarks that many of the sites they're visiting are not in the cache
blueskin_ explains why those sites are not in the cache
pfg challenges blueskin_
Yes, 'some' of the sites will be cached by windows, this was explicitly stated in the original comment by mdisraeli. I don't know why you're acting like anyone is wrong. Did you skip midisraeli's comment accidentally, which lead you to think blueskin_ was offering a flawed solution to kamakazizuru? Blueskin_ was not offering any kind of VAC-related advice or information, just explaining Firefox's weirdness.
The fact that some entries aren't visible in the OS cache has nothing to do with Firefox having its own DNS cache though, it would happen with any browser. And clearing the browser cache has nothing to do with the OS cache either.
So I'm not sure why we're even talking about the DNS cache Firefox uses internally.
blueskin_'s response implies that because Firefox uses its own DNS cache and clearing the browser cache will clear those entries too that either a) opening a page in firefox will not put the corresponding hostname in the OS DNS cache or that b) clearing the browser cache will flush the entries from the OS cache as well. (His reply wouldn't make sense in any other way in this context.)
I'm not trying to prove anyone wrong, but the comment as it was implied you're safe if you use Firefox and clear your DNS cache, when in fact you're not.
>The fact that some entries aren't visible in the OS cache has nothing to do with Firefox having its own DNS cache though, it would happen with any browser.
Not true. Because Firefox has its own cache, news.ycombinator.com is not in my OS cache even though I'm actively opening and closing connections to it.
>blueskin_'s response implies that because Firefox uses its own DNS cache and clearing the browser cache will clear those entries too that either a) opening a page in firefox will not put the corresponding hostname in the OS DNS cache or that b) clearing the browser cache will flush the entries from the OS cache as well. (His reply wouldn't make sense in any other way in this context.)
Option a is the correct interpretation, more or less. Opening a page that is still in the Firefox DNS cache, which lasts hours to days, will not put the hostname into the OS cache.
>I'm not trying to prove anyone wrong, but the comment as it was implied you're safe if you use Firefox and clear your DNS cache, when in fact you're not.
I don't think it implied that, but it's okay if we disagree on this as long as the factual points are clarified.
Wait, are you saying Firefox' DNS cache doesn't honor TTLs? If it's behaving the same way the OS cache does (i.e. drop entries after TTL is reached), basically any entry in Firefox would have to be in the OS cache as well.
Of course if Firefox actually ignores TTLs, forget everything I said.
I'll now set my hair on fire while pondering why they would do that.
It does not honor TTL. The only reason I know about it is the pain it caused me in the past moving a site from one server to another. Everything else on my system went to the new server except firefox.
Proxy servers and/or added dodgy addresses into your [i]hosts[/i] file would circumvent it.
I'm not convinced I like other peoples suggestions of constantly flushing your DNS because you're having to hope that VAC doesn't recheck your cache before your automated job flushes it.
No... that just makes the DNS cache useless and it will require you to look up every domain every time you request something from it. The problem is Steam spying on its users.
I came across this a lot back working as a mac support engineer. It used to be a common issue with older routers that too many DNS requests would kill the inbuilt DNS server, effectively preventing people from using their internet connection. This was typically isolated to someone running a torrent, with their torrent client doing reverse lookups to the hundreds of connections that occurred. This crashed the router's DNS service forcing a reboot.
Workaround was to set the DHCP server on the router to give out the ISPs (or google's) DNS servers.
As someone who reverse engineers things for fun, and can read the C "pseudocode" generated via decompilation pretty easily, I am going to have to disagree with the assumptions made in this post.
First, there's no proof this is from Steam, I've poked around a few of the DLLs since I saw this and am unable to find anything even remotely close to what this does.
Second, this method does NOT send anything to Valve. This method grabs the DNS cache, yes. And it MD5s the entries, then it stores it. This method itself does nothing more with the hashes. For all we know VAC could be doing a LOCAL scan of the list, and comparing it to an internal list of "known" cheat subscription servers.
Until someone posts details of exactly where in Steam this is (What DLL is all that's required to verify), and the calling method that supposedly sends this information to Valve, I would take this with a very massive grain of salt.
So yeah, no proof it sends anything. It could be a local check.