For what it's worth, I just downloaded the latest D compiler distribution for Windows and scanned it with Windows Defender -- it didn't report any problems.
Yes, we are using this reporting tool frequently. Every single file we ever submitted to Microsoft was confirmed by them as a false positive. The definitions were updated, and it was fine. The AV behavior changed again, once they "tuned" their heuristics and the game repeated itself. The only difference was the type/name of the infection, which was still a confirmed false positive.
Brave browser has been flagged by Windows Defender for over a year, instead of blocking it Antimalware Service Executable goes into overdrive and chews up all cpu usage.
Many have reported the false positive, the browser is open source. Nothing has changed. So good luck with that.
I just downloaded the latest Brave browser and scanned it with Windows Defender -- no issues reported. Perhaps there have been issues in the past, but there don't appear to be any known ones at the moment.
Now, the separate tor browser component that's included, that's a different story. But the Brave browser itself is just fine.
Microsoft also noted that the best way to avoid false positives is to publish apps on the Microsoft Store as it scans for viruses or malware before publishing the app for the users.
The overwhelming desire to lock people into their walled garden is so blatant.
The first method to prevent a false positive is to ensure the software is digitally signed.
To someone looking to make money from malware, the cost of code signing is probably insignificant. To someone innocent and just wanting to share some software for free, it's not.
It shows that MS is clearly not interested in actually improving the detection accuracy of their AV much, but more interested in converting developers into a revenue stream.
With respect to this story, I have not seen any references to ken Thompson c compiler backdoor. Just because a compiler is compiled from source doesn’t mean it’s secure.
I've seen plenty of stories of "Hello World!" programs being detected, and on the other hand it's also pretty easy to modify malware so that it stops being detected. IMHO the fundamental aspect of how AVs work is broken because they're not behavioural but basically pattern-matching on binaries, and poorly matching at that. Combine that with their propensity to be used as censorware (detecting keygens, patchers, and other cracks which are otherwise non-malicious and functioning as intended), and you can understand why I don't trust nor use AV. Too bad Windows Defender is installed by default and a real pain to remove.
I ran into similar problems with Rust executables like ripgrep and xsv. I don't know if it is Windows Defender or some other "endpoint security" solution but I have seen both of them blocked in a corporate environment.
According to the linked issue, it's not just Windows Defender; 21% of scanners reported by VirusTotal also flag D. What could cause so many independent virus checkers to throw a false positive like this?
They may not be as independent as you would think. For instance. There could be straight-up signature sharing or outsourcing between vendors, and there is definitely an incentive to flag something if your competitors also flag it.
IIRC Microsoft does share malware detection information with other antivirus providers, so that sounds plausible.
(I think that's also part of the reason why Windows Defender appears less competitive in antivirus comparisons, since you'd have to be doing something wrong to perform worse than the baseline Microsoft gives you.)
Can't find the details now, but there were cases before of AVs using VirusTotal for getting their answers if they lacked any info / signatures for a binary. Could be what happened here, so they're not really independent.
It's pretty common for programs compiled by "uncommon" compilers, and since the D compiler is selfhosted, it would fall into that category. AV software isn't magic, they probably just look for stack/calling/allocation patterns that don't look "right" meaning none of the major compilers generate such code, and it "might" be bad somehow.
I've a few flags from npm with Kaspersky, because there's so many packages that include some little binary, and since this is npm, it's often "unusual", like go.
The AV might not be even looking at patterns in the executable code.
We've had an issue where we accidentally ran the msys strip tool over binaries compiled with the MSVC compiler. A quarter of the scanners on virustotal went crazy and "detected" our program.
The same unstripped program was fine with them. And by "the same", I literally mean the same: virustotal showed that all sections in the "malicious" stripped executable had identical hashes to those in the clean unstripped executable. The only difference was some header bits that were irrelevant to the runtime behavior (I think it was the "linker version" field, and maybe some others).
AVs throw these all the time, for all kinds of applications and tools. They're triggered by patterns and signatures in the executable that "may indicate malicious behavior", etc., heuristically. It's in the AV vendor's interest to be overzealous and have lots of false positives rather than to have a single accidental false negative.
all antivirus sw is blocking all the demoscene-releases(size coding 4k, 64k, etc.) and constantly deleting my own releases after windowsupdate turns the defender on again...
Exactly my experience. I downloaded a large archive of demoscenes. It lead to a lot of "fun" with Windows Defender.
Just my imagination: If a malicious attacker would like to "convince" a victim to turn off the antivirus software, I can imagine this might be a good starting point how to annoy the user sufficiently.
Not the same topic, but we recently faced malware that got distributed via Hadoop yarn - where what looked like crypto mining code would kick in and hog cpu once the scale of spark compute hit some point. It would kick in randomly and it installed itself as a cron job. It was crazy figuring out this was happening as it wasn't something we expected with java code and from such a distribution. We got rid of yarn to eliminate it at that point.
The problem is that it isn't really something the project can fix -- the problem is with the antivirus, whose vendors aren't very responsive. See for instance the PuTTY devs' todo list item about virus false-positives: https://www.chiark.greenend.org.uk/~sgtatham/putty/wishlist/... -- persistently every release gets falsely flagged by something or other, even if they do make a report and get it taken off the list it comes back for the next release, and at least one of the antivirus programs seems to be looking for the putty binary specifically!
Part of the problem with this is how some AVs rate new binaries. Previous job, we had issues with deployments of new versions. Most of our (only) corporate clients used Symantec. They would consistently flag our app as a possible virus because it's "reputation" was low. To Symantec, reputation basically meant how many times it'd seen the app before.
Symantec basically coerced us into buying a code-signing cert from them, and we still had to submit every build directly to them for this reputation issue to go away. Royal pain in the ass.
Sounds like a racket, to be honest. Can't say that I'm sad that their reputation, along with the reputation of traditional AV vendors as a whole, is in the dumpster these days.
It is most definitely a racket, and their "repuation" score severely affected our reputation with existing clients.
Same deployment procedure as had been for years (we'd previously been code-signing with a GoDaddy cert) with Click-Once deployment (was a .Net Office add-in), all of a sudden we start getting dinged.
The cost to the company (a relatively small company) was significant. Had to buy a Symantec code-signing cert (~$900), had to ditch ClickOnce and invest in InstallShield (don't recall the license cost) and engineer the installer to both sign the installed libs & the installer itself. All of this because Symantec hadn't seen the new libs enough because, well, we targeted a smaller corporate audience (for argument sake, we had around a 100 clients, of which, each would only have 1 or 2 installs of this particular app).
I get where they are coming from. Windows is far from the top OS used by D programers, it only effects people who use anti-virus, and I'm betting you can whitelist it or otherwise bypass it. All of those things may seem like sizable hurdles but we are talking about programmers here, I think they can handle it. I know back when I used Windows I would get false positives for all kinds of things because I was on the bleeding edge or always trying new stuff that they hadn't seen or looked malicious (but wasn't). If you are going to run "anti-virus" as a developer you need to know what side-effects you might run into (the same way I know that PrivacyBadger and uMatrix in Chrome can cause issues, I don't go running to the website that is broken just because my extension broke it).
> Anti-virus is bundled with windows. Everyone will have it.
Someone interested in programming in D should be able to disable it and programmers != Everyone
> I wouldn't know how easy it would be to bypass. And in a corporate environment you would not be able to.
I feel like this knowledge is just a google search away and for corporate... If you computers are so locked down that you can't disable the anti-virus they are probably locked down so that you can't install new programs in general. I, for one, would never choose to work for a company that locked down and in that case I feel like you just need to talk to IT to ask them to whitelist it.
All of that said I feel like there is a huge disconnect between the points you are making and the actual population that is trying to use D. Look, I have no clue on the actual install base/user base of D lang but I can't imagine we are talking a "real" number of people having this issue. D doesn't even rank on any of the "Top languages of 2018" lists I found online (that isn't a knock on D, it's just facts) so that coupled with the fact that the vast majority of D developers seem to use non-Windows OS's make this all seem like a lot to do about nothing.
> I, for one, would never choose to work for a company that locked down
Good luck finding a job in banking, government, healthcare, defense, retail, insurance, securities, food, pharmaceutical, or pretty much anything other than social media.
All of these industries are basically forced into “whitelisted software only” by some regulation or another.
I think the size of the company determines this far more than the industry does. I prefer to work for smaller-sized companies or ones that don't treat their employees like children with locked down computers and access. I work in the corrections industry (think ankle bracelet monitoring) and there are no such regulations. In fact I use my personal computer for all my work people it blows the standard equipment out of the water. Even so all our devs have full root on their boxes and install whatever they want.
Also I find it very hard to believe that everyone working at FAANG companies is using a locked down computer.
I for one, never did choose to work for such a company, and yet now I do anyway because my chosen small company was sold to a huge one, after I am 19 years in.
It's almost like that was easier to say than to do.
I'd like to clarify and point out I used the word "choose" very much so on purpose. There are a number of things that would effectively remove this choice from me. If tomorrow I was laid off and I couldn't find anything within a a couple of weeks I'd work at McDonalds to get by but for as long as I have the choice I'm not interested in locked down computers or locked down internet where I work. If I were in your position I'd have to weigh the pros and cons before deciding to look for a new job. You obviously decided the pros outweighed the cons and I don't hold it against you or anyone else.
> I'm betting you can whitelist it or otherwise bypass it
That you can easily bypass it is irrelevant - it's a bad sign to me, as a potential user, when the installer is flagged as a virus, and a far worse one when the D team seems utterly apathetic about fixing it. I see no evidence in the thread whether it's a false positive or not (though plenty of unbacked claims and some ad hominem: "you are the one using the snake oil software"), only developers asking the users to blindly report it to their AV vendor.
I also did not see any response to Mike Franklin's comment: "It's not the compiler that is reporting the virus, it's the installer. What utility are we using the generate the installer executable?"
This throws up an entire forest of red flags to me as a developer - I'll stick to languages where the contributors care enough about the language to report a false positive themselves.
Why is it on D's developers to fix a false positive in someone else's software? They redirected the reporter to the proper venue. I don't see what else they should be doing.
2. They are the only authority that can rightfully claim that their product is virus free. In time they might gain an understanding of what triggers it which might also be beneficial when reporting it. If it is a common occurrence they should probably have a template ready and a process for doing this. They also have interest in it being done correctly and that they have the ability follow up on any issues. In fact, they should prefer to do it themselves rather than random people doing it for them when they only indirectly can be informed of any progress.
3. Thank the reporter for bringing the issue to their attention.
4. Happy to answer questions related to the issue.
Doesn't matter who is to blame. The D community is taking the hit, they should act to try and avoid that.
If google erroneously blacklists your site, why should you care? You haven't done anything wrong? Some things just suck and you have to take care of it yourself.
1. these same false-positives have been a recurring issue for multiple years already
2. there is no fix for a tool which is not a mass-market-consumer-product
3. as a technologist you should know that all anti-virus is bullshit which does not work. false positives and false negatives abound. the anti-virus's job is to accurately identify viruses/malware because users can't. But the anti-virus can't do it either. It's true. Really. I figured this out on my own over 15 years ago, and it never changed.
I really hate the sentiment and consensus around anti-virus. It's like an "car-crash protection talisman". It just does not work. But if you tell some one not to use it, and they get in a car crash, it's your fault. If they use it, and they get in a car crash, oh well, what can you do. But it's truly bullshit.
> They are the only authority that can rightfully claim that their product is virus free.
This is a dumb idea. What's the point of anti-virus if we just trust the vendor.
So, the solution is to antagonize a user that is taking time and effort to inform you of the problem?
> This is a dumb idea. What's the point of anti-virus if we just trust the vendor.
The AV company will, hopefully, not... The vendor is the only one that can vet that the file on the site is the file that it is intended to be. They are also the only ones that can make changes to the their builds (which can help).
That's like asking why you should correct false data on your personal credit report. It's someone else's computer database, not yours! (Or maybe a better analogy would be "Why should I ask Google to stop marking my legit domain as spam and hiding emails to my users? It's their system!")
Software developers have to lobby on their own behalf so their software fits into larger ecosystems seamlessly. Or deal with angry users.
Angry users is not something you can opt out of. Malware reports are just another fact of life for many open source projects. I think the putty link (was posted elsewhere) is an awesome summary of the impossible fight you're asking developers off taking on.
Also, that's an insane comparison between AV reports and personal credit reports.
akavel said it best, you have to pick your battles. Who are we to tell the D programmers that this small issue affecting a tiny percentage of their users is a higher priority than any feature/bug/etc they have in their backlog?
All of this stinks of entitlement to me. "I want you to fix my problem cause by my software that only affects a small group of people".
I totally emphatically understand, that for DMD authors, reporting their binary to a truckload indifferent AV vendors is the worst kind of kafka-esque, soul-draining, burnout-inducing, and quixotic chore imaginable. While at the same time they're certainly trying to manage a lot of hard chores already, where at least they have a bit more agency over the eventual result. There's only so much they can do with limited resources. They must pick their battles, and it's fully and only their choice to make, no one can force them otherwise.
Then, they're actually still patient enough to be polite and trying to suggest to the user what he/she can do to help the project (or actually himself/herself), in order to resolve the issue he/she has. And in response, the user is only trying to deflect any responsibility and work/effort, and aggressively trying to push it back on them, using some weird, illogical, and generally absurd arguments (that they also are politely trying to explain as being invalid). Instead of actually trying to do what they suggested, and what could actually help resolve the issue.
That's how it would look like from the authors/maintainers' side of the fence.
There's probably a significant overlap between developers who use a non-mainstream language like D and developers who use non-mainstream OSs like Linux or macOS.
It happens with useful system utilities (windows sudo and nssm for instance). It's because they request elevated privileges and aren't part of an installer application that's popular.
Something similar happened to Racket a few years ago with AVG. IIRC I just send an email to one or two of the antivirus companies and they fixed the problem in a few days.
Anyone can send the report of the false positive to the AV companies. I don't understand why nobody took a few minutes to solve this.
It's one of those things where it's a few minutes per company per release. And then you make a new release, and have to do it again. And again. And again.
As a volunteer, it's hard to do that for software that seems to be financially motivated to report false positives, so the users can pretend the money they give antivirus companies is worthwhile.
In my case it was only one or two companies in only one release. It was not necessary to resubmit it in every release.
Also, I understand that this is not top priority and that is a nice small task to be delegated to one of the newest contributors, specially the one that have the problem and are using the obscure AV. But after a few months, it's time to say "whatever" and try to solve it. [Anyway, I tried an hour ago, it's not so easy. See my other comment.]
I decided to try to solve it myself. I don't get any warning with Windows Defender (Win10). With VirusTotal I got only one report https://www.virustotal.com/#/file/e02d6aff4b59b2f66ed21fb74e... by Tencent . I went to the webpage of Tencent but I can't find the correct way to report a false positive. Most of the contact page is in Chinese, but I don't speak Chinese (Hi from Argentina!) I'm in doubt if I will send the report to whoever looks more relevant, or just forget about this.
I just whitelisted the entire directory for windows defender, and I know a lot of other D devs do the same. (Without that, even my own programs I compile will have startup lag - it doesn't flag them, but it takes an extra half second to run the program, presumably scanning it first, which is infuriating for an edit/compile/test cycle.)
So for us who are already in, it is a "solved" problem. I recognize it doesn't help first timers though.
Devs learn pretty quick to turn off AV. It's nigh impossible to run a test suite with AV on, for example, because it tries to scan/flag every executable built during the tests.
After hearing "Wolf! Wolf!" a few thousand times, one is done with AV.
Is there a reason this happens with D, but not with Rust, C++, or C#?
I think if I had to turn Windows Defender off, it'd be better to just switch to *nix (which I assume doesn't have similar false positive problems for whatever reason).
Some AV vendors (notably Avast, AVG, Avira, McAffee, Norton and Windows Defender) use extremely poor heuristics and never actually tweak them, instead just whitelisting a given file. You think that if false positive in a compiler or runtime is reported they'll at least will get serious about it? Nope.
Consider Red, which gets hit by AV truck almost on a monthly basis [1]. Not only toolchain itself gets recognized as a "generic malware" †, but almost all compiled user applications get flagged too. Such false positives are alway duly reported, issue seems to be gone... and then cycle repeats, even though no changes in compiler output or runtime with REPL were made ‡. Just like that, sporadically. Some of them go nuts even over simple "hello world".
Last year, as a last resort, developers directly twitted one of the most nastiest vendors (Avira) about this issue [2]. Nothing changed. They don't care at all, since 2012, when the Red project started.
I know NirSoft [3] products suffer from that plague; network utilities (Wireshark, nmap) get hit, so as various packers (WinRar, also see [4]); re-reading the thread after the original post, I'm not even surprised that demoscene experiences problems with anti-virus software; but seeing this issue pertaining mainstream language such as D is, well, terrifying, to say the least ⁂.
It can be easily brushed off as an anomaly in such mature, widely adopted code bases, but with non-mainstream, yet-in-development languages it's a scourge that hinders the adoption and serves as a constant source of headache both for developers and users. And, honestly, I can't see how this can be resolved without raising public awareness and engaging other developers in a debate.
† We can only guess what causes the trigger, as vendors never give any feedback. One guess is that toolchain generates a file layout different from VisualStudio or GCC linkers, and uses a custom compression of DATA segment.
‡ I think this happens after each update of heuristic DBs or AV engines which vendors share. IIRC Windows started to use machine learning for malware detection some time ago. Total bonanza.
To make sure that our releases are clean we apply best practices like generating the Windows binaries on Linux servers running Wine. We test the binaries to the last detail with various AV packages. Malware analysis is also incorporated to get to the ground why some AV packages report the binaries as infected but to no avail. The heuristic engines of major AV vendors are tuned very severely, and there is no way of getting around it for us developers.
The other day we saw an article about someone embedding a .zip in a jpeg that survived twitter's thumbnailing. What about doing similar things with the eicar string or other false positives that make antivirus freak out? This seems like it'd be an interesting DoS attack.
For some reason Windows Defender started hogging my CPU today whenever I have my IDE open. In the multiple years since switching to Microsofts internal AV Solution i've never experienced this, does anyone else have a similar experience?
Maybe your IDE started reindexing source files in the background, accessing many files very quickly? Defender can cause lots of load if that happens and it wants to inspect everything.
I would imagine they encountered malware written in D, compared the executable to a bunch of others (not written in D) then decided the distinctive characteristics of D were indicative of malware.
When there is no cost to providing a false positive, there is little incentive to avoid it.
There is incentive to avoid it but as you might imagine there are not that many D programs in corporate environments so prob not a pain point for any major customer and it's low priority.
- open source, used by many before
- av companies frequently get false positives
- virus companies probably use common sigs and heuristics
- the installer might probe for higher privileges without actually being malicious
isavirus:
- not much
Was clamav tried? Is ESET listed (my favourite AV)
https://www.microsoft.com/en-us/wdsi/filesubmission
...to ensure things like this get investigated.
For what it's worth, I just downloaded the latest D compiler distribution for Windows and scanned it with Windows Defender -- it didn't report any problems.