The vulnerability means that there is an ASN.1 certificate format you can use that will validate against any unpatched GnuTLS-using application, regardless of how you configure ciphersuites and negotiation. The bug isn't cryptographic. It's much simpler and more basic than that. My guess is that it's also been embedded in GnuTLS for longer than the SecureTransport bug had been.
For connoisseurs of bogus conspiracy theories, it's worth noting that the coding flaw that produced the bug is much more subtle than "goto fail", but the bug is equally simplistic. Where "goto fail" required a duplicate line, GnuTLS required only a single switched token; not only that, but the bug comes from a mismatch between two common C error checking regimes (a zero-return error and a negative-return error).
I think what I like least about the bug is that it was found by the GnuTLS maintainer, long after the code shipped. That's a signal of how much attention GnuTLS gets.
That's a signal of how much attention GnuTLS gets.
It's a signal of how much attention the dark and dusty corners of all open source code gets. My "favourite" FreeBSD advisory was the 2011 telnetd "christmas present" -- remote root to any system running telnetd with the default options. The bug was in routines for setting up encrypted telnet sessions... who even knew that telnet had encryption? The bug was a blindingly obvious buffer overflow, but nobody goes digging through telnetd looking for encryption code.
About six months after that advisory, I heard from a major aircraft manufacturer that they had been using the BSD telnetd code to provide a channel for in-flight tuning of engine performance.
This is why I award bug bounties for punctuation errors in comments in the Tarsnap source code -- it encourages people to read all the code, not just the interesting bits.
I did. Most of the ftp, telnet, rsh, etc. utilities shipped with BSD and Linux had Kerberos and encryption support. I also suspect that these had at least wider use in larger environments than many people knew, because before OpenSSH had Kerberos support, you could integrate encryption and authentication in useful ways.
All of Howard Chu's analysis in that thread is great, but he didn't discover or report this bug. Reading that message should make it clear; he's talking about sloppy string handling and bad APIs, neither of which has anything to do with this bug, which was mixed-up return code semantics in an internal, static function. Look later in that thread for some other pretty sloppy code he tears apart, though.
The point is that the GnuTLS code quality was obviously crap, the developers were obviously incompetent, and only a fool would trust GnuTLS with their system security. I pointed out numerous problems - more in private emails to the GnuTLS authors and Debian maintainers, not just those published on the list - all related to the sloppy programming practices and inconsistent APIs. This mixed-up return code semantics is just another consequence of those sloppy programming practices. How many ways do you have to see that "this software sucks and the developers don't know WTF they're doing" before you realize you shouldn't be trusting them?
Linus's Law as described by Raymond is a claim about software development, named in honor of Linus Torvalds and formulated by Raymond in his essay and book "The Cathedral and the Bazaar" (1999). The law states that "given enough eyeballs, all bugs are shallow"; or more formally: "Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix will be obvious to someone." Presenting the code to multiple developers with the purpose of reaching consensus about its acceptance is a simple form of software reviewing. Researchers and practitioners have repeatedly shown the effectiveness of various types of reviewing process in finding bugs and security issues, and also that reviews may be more efficient than testing.
In Facts and Fallacies about Software Engineering, Robert Glass refers to the law as a "mantra" of the open source movement, but calls it a fallacy due to the lack of supporting evidence and because research has indicated that the rate at which additional bugs are uncovered does not scale linearly with the number of reviewers; rather, there is a small maximum number of useful reviewers, between two and four, and additional reviewers above this number uncover bugs at a much lower rate. While closed-source practitioners also promote stringent, independent code analysis during a software project's development, they focus on in-depth review by a few and not primarily the number of "eyeballs".
Raymond's quip may be somewhat hyperbolic for effect, but I think it's harder to argue with the assertion that "with many eyes, many bugs are shallow". True, there are some things that are esoteric and which only a few very specially trained persons would notice, but in many cases, it's not that way. The fact that a random person will show up on the mailing lists of open-source projects, even minor ones, and have a bugfix patch validates the theory that an open codebase contributes significantly to software quality, and particularly a greater degree of software buglessness.
I think that people who argue over Raymond's quip are just being pedants. It's true that not all bugs will be shallow, no matter how many eyes are on software, but I don't think the expression was meant to be taken literally.
My favorite part of the "many eyes" argument is how few bugs
were found by the two eyes of Eric (the originator of the
statement). All the many eyes are apparently attached to a
lot of hands that type lots of words about many eyes, and
never actually audit code.
I mean shit, I don't need to be an aeronautical engineer or work in the FAA to say that several people investigating an airliner crash will be more effective than one single dude sifting through a literal field of debris.
ESR's quote isn't wrong (perhaps hyperbolic, but in spirit it isn't wrong). It's just an uninteresting observation. What do you do when something has you stumped at work? You ask the guy across the hall to look over your shoulder for a second... It's just common sense.
People simultaneously revere and hate the quote only because ESR said it. If Joe Shmoe had said it all those years ago, it would have just been met with "no shit" and never given second thought.
I think there's a point here. The point being much more people talk about many eyes than actually audit existing code. The fact is, there are quite a number of bugs that stayed in the code for a long time despite being not that deep. Maybe that says we don't have enough eyeballs, but most of people quoting that sentence quietly assume we do, equating number of participants in FOSS communities with number of people actually auditing the code. Unfortunately, the former is much bigger than the latter. We pretty much have the same situation as we have in science publications where a lot of results are not reproducible and a lot nobody even bothers to check for reproducibility, despite reproducibility being the basic tenet of open publication and peer review and being necessary to validate the result. Unless there's a reason to - e.g. an identifiable bug - not many bother to go back to existing code and check it for correctness.
The quote is hyperbolic for the "all", and should probably use the word "shallower", but otherwise I don't see anything wrong with it.
"much more people talk about many eyes than actually audit existing code" is true of free software, but is also true of just about everything. More people think "I feel comfortable crossing this bridge because plenty of engineers have looked at it" than there are engineers actually looking at bridges. I haven't read the quote in it's original context for several years, but I don't remember it conflating software users with software developers.
The difference is that engineers actually know how to build safe bridges. I feel (and am actually) safe living in a house designed by a single competent architect, but I am not secure using a TLS implementation written by a single competent programmer. The only way we have found to end up with reasonable secure code is to have it reviewed by a lot of competent people.
There are a dozen points here... The shallow bugs statement is true when there is a community that is large and with incredibly deep understanding; linux and posix is a great example, there is staggering depth of knowledge on those interfaces and behavior. Unfortunately there isn't wide and deep knowledge of tls architecture and implementation, there isn't even a lot of commonality across interfaces.
There is a social axiom that you and I don't know crypto and we should leave it to the experts, yet they need help too.
There is conventional c style and this function, while documented, does the opposite. I had to look at the code a couple times to see the bug, a lot of reviews could have missed it.
There is conventional c style and the whole failure chain cleanup mess, look at that code again, they've got their own data representation and free function that uses it to detect if memory is allocated, the free is private but the initialize is done inline. That stuff happens everywhere by many projects, I'm not saying its wrong but it leaves it open for easy bugs, one gaffed variable declaration potentially screws everything and you need to know the data structure even though you don't directly use it.
And I don't want to pile on gnutls, I think their intentions are good and this is a bug, but this looks fucking prime for unit testing...
There are a lot of variables to determine and measure a projects health, the whole community needs to step up in the quality department, there are lots of ways to contribute and despite the adage, more people should look at crypto.
> I mean shit, I don't need to be an aeronautical engineer or work in the FAA to say that several people investigating an airliner crash will be more effective than one single dude sifting through a literal field of debris.
With zero people looking at airliner failures, nothing will be discovered, no matter how many inspectors the FAA has on its payroll. In the software world, there are almost no people doing code audits. Bugs and security holes are left in live code for years. Basically, Theo is saying "Shut up and audit."
How much open source code have you audited for bugs recently? How many subtle correctness issues have you found in projects you've looked at? For the sake of code quality, I certainly hope it's higher than the amount I've audited.
"Linus's Law" itself says little about whether or not people will actually look at code. It just says that more people looking is better than fewer people looking.
How much code I have audited, you have audited, ESR has audited, or hell, how much code Paul McCartney has audited, really has little to do with the obvious correctness and banality of the 'law'.
In other words, your interpretation of "Linus's Law" has no impact on code quality in the real world. Edit: However, since it's meant to imply that in open source, there are people looking at code, I think that the criticisms stand.
Yes, in the sense that Crito's Law (coined by me, now) "The bigger the ship, the more freight you can fit in it." has no impact on the real-world ship transport industry.
It is obviously true to the point of being banal. It is a pointless statement of uncontroversial fact that provides next to no utility to anybody. It's not even interesting for being a tautology.
If I were a particularly objectionable and self-promoting person, then perhaps people might object to Crito's Law whenever it were quoted on shipping forums, but that wouldn't make it incorrect. Nor would my shameless self-promotion make it profound.
(Is Crito's Law _precisely_ true? Well no, some large ships are not designed for freight after all... but the general principle is true.)
"Many eyes make bugs shallower" is indeed true and a tautology.
The way ESR meant it, it's merely BS.
He meant is as in: "because open source code is available for everybody to see, many people look at it, and so bugs are found more easily".
In the context it was said, it was meant as a factual observation about what GOES ON in OSS, not merely as a trite theoritical description about many eyes being better.
So, people are arguing against that, not against the idea that if more people ACTUALLY look, they will find more bugs.
The case is, very few people look at code. In some cases, even for extremely widely used software by millions of OSS users, even less people than the people paid to look at a particular proprietary software look at the code.
Heck, Gtk, the basis of most Linux desktops, had in the latest years like 1-developer really working at it (I know, because he complained publicly about the situation).
I don't know what happens in the Windows GUI toolkit or Cocoa, but I doubt there's one person doing all the work -- and only during his free time at that...
It is neither pointless nor content-less. It gets to the exact heart of the problems with ESR's claim. If Joe Shmoe had said it, people would not say "no shit", they would still say "no that is complete nonsense". Because it is in fact complete nonsense, for exactly the reasons Theo pointed out. All the eyes in the world don't amount to anything if they aren't being used to audit code. And to do that, they need to be attached to people capable of auditing code. ESR is simply being used as an example of his hypothesis being wrong since he is the most obvious example. It isn't attacking him to point out the flaw in his hypothesis.
"Eyes" obviously refers to "eyes of people auditing the code", not "eyes in skulls of people who are currently in the building", not "eyes sitting in jars in the Mutter Museum", not "every eyeball belonging to children currently living in Idaho". Furthermore, different eyeballs of course have different worth. Nobody in the world disputes that.
The law itself doesn't say anything about whether or not people will choose to examine the code. It just says that more people examining the code is better than fewer people examining the code. One audit is better than none. Two is better than one. Three is better than two. One shitty auditor is better than none. Two shitty auditors are better than one. One auditor twice as good as a shitty auditor is better than one shitty auditor, if you really want to get into tedious eyeball calculus. Etc.
Seems stupidly obvious; so obvious it isn't even worth stating? That's because it is.
Or to address your point another way:
If I said "You need fuel to make your car go." would you object that this is bullshit because not only do you need fuel, but it needs to be in your car, and the right sort of fuel? I don't think you would say, "But diesel fuel sitting in a puddle on the ground is useless and won't make your petrol car go anywhere!"
The point you seem to be deliberately missing is that the opportunity for something to happen and something happened are not equivalent. There is a natural assumption that because there is the opportunity for many people (however you decide to define "many") to review/audit/test code in open source projects, somebody must be doing so. Even when you base that assumption on the number of contributors to the project, non sequitur; feature addition/refinement does not imply bug fixing.
Another thought occurs to me. It isn't clear to me how much reading the source actually helps with finding most security vulnerabilities. It does seem like some of these recent ones in SSL have been found by detailed code review (after years of providing bypasses), but I mean in general.
It's remarkably hard to find a security problem reading the code unless that security problem is blatantly obvious. Part of the problem is that code also communicates intent, and security is in part counteracting that intent.
Most (more than 90%) of the security problems I found (almost all of which were in open source prjects) started off by observing misbehavior or wondering if I could induce misbehavior. This works because I would start off attacking an API, not the underlying implementation. Only after showing the problem would I look at the code. I think on one case, I was looking at code and thought "maybe that's a weak point." But in every other case, my testing was fully black-box to start (things later lead to a code audit in several cases).
> It's remarkably hard to find a security problem reading the code unless that security problem is blatantly obvious.
Then I guess I would consider most (C code) security problems rather obvious. The kind of stuff you see patches linked to in CVEs.. good old buffer overflows, off by ones, arithmetic screwups (mixing types or not checking for overflows), missing return value checks, the occasional swapped parameter, typo, simple logic whoopsie, etc.. These are incredibly common and rather easy to find once you get used to that sort of grunt work. But obviously you have to know what to look for.
The remarkably hard ones (for me anyway) are much more subtle, things like race conditions...
Using min where I mean max and max where I mean min is my favorite classic mistake that I usually just make most of the time and remember to check for and fix afterwards.
The same thing with misusing "this" in JavaScript closures: no matter how hard I try to consciously avoid doing it, it's always the first thing I check for when code doesn't work as intended, because I still make that same mistake all the time.
Another source of confusion in code that Ben Shneiderman pointed out, is that the meaning of "and" and "or" are opposite to programmers and normal human beings, so languages like SQL and logical expressions in most programming languages are fundamentally confusing to many people, and the inconsistency can be an un-obvious blind spot.
Normal humans mean to union when they say "this AND that AND those AND these", while programmers mean to intersect when they say "this AND that AND those AND these". So that's a terrible source of confusion.
In other words, programmers think adding "AND" clauses narrows down the selection like "((user.age >= 18) AND (user.gender = 'F'))", while humans think adding "AND" clauses augments the selection like "adults AND women".
I'm not advocating changing the meaning of "AND" and "OR", just pointing out that it's a source of confusion you should look out for, and be aware of when talking with non-programmers.
I saw Alvy Ray Smith give a talk about 3D graphics programming, and he confessed that he and his colleagues would just write some code and then flip the signs around until it did what they meant it to do. That made me feel a lot less embarrassed about doing that myself.
I try to read the code I need to use, but of course I never have the time to read as much as I should. It's really the best way to learn programming in general and to learn particular apis, as important as a musician listening to other musicians' music.
I feel much more confident using libraries whose source code I've already at least skimmed through. (Speed reading code and learning where to look for stuff later when you need it is a useful skill to develop.)
But reading static code isn't enough to trust it and be sure the comments and formatting aren't lying to you. Stepping through code in the debugger and looking at its runtime state and control flow is crucial to understanding what's really going on.
But the problem with reading code (especially code that you're not running in the debugger), is that you see what you think it's supposed to do, not what it's actually doing, especially when you're "skimming" over it as I like to do.
Occasionally I have the luxury of enough time to go into "study mode" and carefully read over code line by line (I've been reading the amazing npm packages in http://voxeljs.com recently, which is some amazing and beautiful JavaScript code that I recommend highly). But that is extremely tedious and exhausting, and uses so much energy and attention and blood sugar that I have to close my eyes and take little power naps to let my mind garbage collect.
And then I get these weird dreams where I'm thinking in terms of the new models and api's I've just learned, and sometimes wake up in a cold sweat screaming in the middle of the night. (I have sympathy for Theo and his neighbors he wakes up at night from nightmares about all the terrifying code he reads.) (So far no terrible nightmares about voxeljs, but a few claustrophobic underground minecraft flashbacks.)
Refactoring or rewriting or translating code to another language is a great way to force yourself to really understand some code. I've found some terrible bugs in my own code that way, that I totally overlooked before. And looking back, the reason the bugs were there was that I just saw what I intended to have written, instead of what I actually wrote.
And for those kinds of bugs, comments that describe the programmer's intent are actually very dangerous and misleading if they're not totally up to date and valid. Because the compiler does not check comments for errors!
I try to use lots of intermediate descriptive variable names (instead of complex nested expressions), lots of asserts and debug logs, and do things in small easy to understand and validate steps that you can single step through with the debugger. It's important to examine the runtime state of the program as well as the static source code. But that is hellishly hard to do with networking code in the kernel.
I also like to get away from the distractions of the keyboard and debugger, and slog my way through every line of the code, by printing it out on paper, going outside, sitting in the sun under a tree, and reading through every page one by one front to back, scribbling notes on the paper with a magic marker. That forces me to make my way all the way through the code before making any false assumptions, jumping around, and getting distracted. (ADHD Management Techniques 101!)
Using analogies to avoid dealing with the actual topic is not productive. ESR claimed that open source projects are inherently more secure because "many eyes make all bugs shallow". That is what we are talking about. Not cars, not airliners. His hypothesis relies on the (false) assumption that many eyes are looking at the code simply because it is open source. That is not the case, as obvious security holes like these demonstrate time and again. It takes dedicated, qualified people spending significant time auditing code to make it more secure, not being open source. A closed source project with 1 security expert auditing the code is more secure than an open source project with 0 security experts auditing the code and a million users downloading the source and compiling it.
I am with Theo on this one. The many eyes argument is a poor one for a large number of reasons. Bugs are one issue regarding security but software design is a much bigger one. It is good design that ensures that software is robust even when bugs surface and that's a far larger issue than the question of bugs.
The problem with ESR's view here is that it occurs in a more general essay on this matter come up with the idea that open source has the advantage because of eyeballs, distributed design and so forth. But you can only distribute design so far, and almost every successful open source project has a small design and engineering team.
This being said there are plenty of cases where I do in fact rely on many eyes. It's not that it makes the bugs shallower but that repetitive review and discussion helps to shake out both design flaws and software bugs. I tend to push a lot of security stuff to the underlying platform for this reason. But part of it is also trusting the design team.
Code auditing is tough work and I generally assume it doesn't get done. What is good however is that a lot of other people are depending on software that is professionally developed in a relatively transparent manner and so chances are somewhat better that people will audit the code at some point.
The comments you perceive to be nastily and unjustly bringing down ESR are presumably being made by people who do not agree with you about the value of his intellectual contributions. In fact, those comments are directly rebutting his best-known intellectual contribution. So your objection begs the question.
Indeed. A lot of people just see the criticisms and think that he is unfairly targeted, but you don't have to look far to find the obnoxious and arrogant behavior that earned him the derision he attracts today.
Theo de Raadt's argument (as presented here) isn't logically valid. A character attack is not an argument. So he's just hurting his cause. That's my point here.
As an aside, I'm not really begging the question because I defended ESR's "Linus' Law" thing in a different comment.
>Theo de Raadt's argument (as presented here) isn't logically valid
Yes it is.
>A character attack is not an argument
His argument is not a character attack. It is pointing out two problems with Eric's hypothesis. The reality that simply saying "lots of eyes" doesn't actually mean there are lots of eyes. And that the eyes have to be attached to people who actually know what bugs look like or they won't be found. ESR himself does not bother to look at code, thus providing counter evidence to his own claim that open source software is seen by many eyes.
The proof is the fact that we have just gone through two historically tragic, enormous, and terribly stupid security holes, each big enough to land the space shuttle on, either of which could have been easily caught by ESR or anyone else simply shutting their mouth and reading the code.
Um, I've had Theo in my house. I've known him for a long time, we've had technical conversations about the kernel that 99% of the people here couldn't understand (maybe it is me but I am so sick of "look at me I've figured this out" and it is stuff you should have learned as an undergrad. Go code more and talk less.)
My opinion is until you have done as much as Theo has done you should maybe not talk so much.
Theo has earned my respect and the right to be as much of an asshole as he wants. The important things he's so good at doing often require that of him, so it's not counter-productive or self defeating.
I also have a tremendous amount of respect for RMS, and forgive his personality quirks, although I'd never want to work for him. Unfortunately, a lot of his personality quirks and ways of communicating are self defeating. But more importantly, his beliefs are totally logical and consistent and well thought out, and he sticks by them. It's his priorities and his way of communicating them that people have problems with.
He's also got a brilliant sense of humor, that a lot of people just don't get, and take offense at, when he was just trying to make them think. But at the same time, he's incredibly easy to wind up by mentioning Open Source Software. But I think he's in on the joke and it's just a theatrical performance, like Saint IGNUcius.
My Emacs Hacker Boss from UniPress Software and I ran into him at a scifi con, and my "Evil Software Hoarder" colleague asked him "I heard a terrible rumor about your house burning down. Is it true?" He fired back without missing a beat, "Yes, but where you work, you probably heard about it in advance." We all had a laugh and no offense was taken: he's got a sharp sense of humor and he's quick on his feet!
Here he is being a total dick, by chastising someone for posting a baby announcement (who is now 21 years old) to a mailing list about having dinner on the other side of the continent as he was on. But he's fucking brilliant and hilarious and makes some excellent points that are totally consistent with his beliefs, even through he wound everyone up and was repeatedly told to fuck off in various ways, which he took in stride.
"You people just have no sense of humor. I thought the original message was pretty funny and made a few good points (if it didn't, nobody would have been offended). I guess it's a shock for smug self-righteous breeders to learn that not everybody in the world thinks babies are cute and special. -Wayne A. Christopher"
"Finally, someone read the message as it was intended to be read. -RMS"
"I'm somewhat surprised by the idea that a mere message from me could torpedo the happiness of parents. I'd think it wouldn't even come close to doing that. Not that I wanted to do that. The most I thought it could do was to discourage the posting birth announcements. -RMS"
RMS is like William Shatner, in that he's in on the joke, and can have a good laugh at himself, and at least he isn't the mean kind of narcissist. To extend that metaphor further than I should: RMS = Captain Kirk, Theo = Spock, ESR = Harvey Mudd, Microsoft = Klingons, and Free Open Source Software = Tribbles.
Since you've been so generous to offer you opinion of Theo, I think it's only fair and balanced for me to offer my opinion of Eric the Flute, in a way that is consistent with Eric's own name dropping protocol.
I believe that Eric the Flute was disrespectful to Linus by labeling ESR's "Many Eyes" theory "Linus's Law".
I believe that Eric the Flute was disrespectful to RMS by relabeling RMS's "Free Software" movement "Open Source".
I believe that Eric the Flute has made a career out of bogging down the FOSS world in internal doctrinal disputes, and that his "many eyes" argument gives people a false sense of security in open source software, and that kind of pap diverts attention and money away from supporting qualified eyeballs and assholes who do the incredibly difficult and tedious work of meticulously reviewing code and fixing bugs like Theo De Raadt does.
And I believe that Eric the Flute is being a narcissistic hypocrite when he writes stuff like this recent blog posting, with numbered instructions for where, when and how to drop and not drop his name. Specifically, number two, which gives me the right to drop his name in this context:
2. Do drop my name if by doing so you can achieve some
mission objective of which I would approve. Examples
that have come up: encouraging people to design in
accordance with the Unix philosophy, or settling a
dispute about hacker slang, or explaining why it's
important for everyone's freedom for the hacker
community to hang together and not get bogged down in
internal doctrinal disputes.
So it's important to "not get bogged down in internal doctrinal disputes",
huh?
My mission is to explain why it's important for people in the FOSS community not to base their careers on tearing other people down. Why can't we all just get along, huh?
I'd like to hear Eric the Flute explain how his goal of "not get bogged down in internal doctrinal disputes" squares with his decades-long ongoing feud with RMS about "free software" -vs- "open source software" on which he's based career?
And I'd like to ask him to please stop encouraging his followers to act as if there's some kind of war going on between Free Software and Open Source Software.
For example, Eric the Flute's friend and fellow right wing global warming denying libertarian gun nut internet celebrity "Tron Guy" Jay Maynard (who fawningly replied to that blog posting "FWIW, I apply my own fame in much the same way, and follow this set of rules both for myself and for my friendship with Eric. Like him, I didn’t set out to become famous.") has taken a stand on wikipedia and his Hercules emulator project about how there is a war going on, and he ideologically opposes Free Software but supports Open Source Software, and it's insulting to him for anyone to insinuate otherwise:
The Hercules development community generally objects
to the term "free software", and in several instances
contributes to Hercules specifically as a reaction to
the misuse of the term. As long as the portal and the
categories use this misleading term to apply to
software that is freely available and redistributable,
please do not add Hercules to them, since it implies
support for the "free software" side of the ongoing
political war that does not, in fact, exist. -- Jay
Maynard (talk) 08:57, 13 March 2009 (UTC)
Please do not ascribe to me a viewpoint I do not hold.
Hercules rejects the term "free software", and many of
its developers - including me - contribute to the
project on the explicit basis that it is not part of
that world. This has been hashed out at the
Talk:Hercules emulator page.
Calling it "free software" here ascribes to me a view
that I not only do not hold, but actively disagree
with. Please don't count me as a supporter of "free
software", the FSF, or Richard M. Stallman, and please
don't enlist me on your side of the "free
software"/open source war.
I believe calling it "free software" is argument by
redefinition, and fundamentally dishonest. It's also a
naked attempt to glorify a major restriction of
freedom for programmers by nevertheless calling it
"free", in the same vein as "War is peace". The
concept of freedom is far too valuable to demean it in
that manner.
As for "but it's free software anyway", the reverse
argument, that "free software" is all open source, is
just as valid - yet "free software" zealots reject it
out of hand and say "don't co-opt our work!" Well,
that sword cuts both ways.
I am not a member of the so-called "free software"
movement and never will be. Please don't insult me and
misrepresent my views by calling me one. -- Jay
Maynard (talk) 13:22, 16 August 2010 (UTC)
I wonder where "Tron Guy" got those ideas about this "ongoing political war" about "Free" -vs- "Open Source" software, and why he's getting so bogged down in internal doctrinal disputes?
Like Lubos Motl, his crankery is a counterpoint to his
area of brilliance, not a negation of it. And I
disagree with just about every political opinion ESR
has. (And have actually argued them with his good
friend Jay Maynard.) - David Gerard (talk) 20:46, 29
July 2010 (UTC)
It saddens me that Jay "Tron Guy" Maynard is one of
ESR's fans. Turns out the guy who made cosplay
respectable for grownups is a right-wing asshole --
wonder if he's a brony? (Anyway, it seems he's given
up lead maintainership of the Hercules mainframe
emulator, so, um... yay?) EVDebs (talk) 23:42, 10 July
2013 (UTC)
"But there is still one thing that America has taught the world. You have taught us all that giving second chances is not just generosity, but the wisdom that even the best of us sometimes make stupid mistakes that it would be grossly unfair to believe were one's true nature." -Eric Naggum
"I learned a lot from talking to Erik on matters technical and non-technical. But one thing I learned, not from what he said, but from the meta-discussion which was always there about whether to tolerate him, is that I think we as people are not all the same. We make rules of manners and good ways to be that are for typical people. But the really exceptional people among us are not typical. Often the people who achieve things in fact do so because of some idiosyncracy of them, some failing they have turned to a strength." -Kent Pitmann
"The purpose of human existence is to learn and to understand as much as we can of what came before us, so we can further the sum total of human knowledge in our life." —Erik Naggum
I think your analysis is irresponsible and fallacious.
ESR is making a completely valid point, and the underlying premise of his theory---that having software open to review can help---is only confirmed by this incident, not rejected.
Specifically: If GnuTLS were closed source, this problem would likely never be publicly discovered and disclosed.
So overall, ESR's theory is accurate and useful.
Note the word "almost" in the theory, which serves as a (completely valid) escape hatch (that you are mistakenly neglecting) for incidents like this---which fit the underlying premise, but are "corner cases" rather than "common cases."
If it is open source, as in a Debian release, an end user's recourse is to fix it themselves.
If it is commercial, be it closed source or not, an end user's recourse is to sue the supplier (or something similar).
The commercial supplier has a financial incentive to get it right, the open source developer has an intellectual and street cred incentive to get it right. I'm not sure which one actually works better, I know that the popular opinion is the ESR eyeballs claim but it's not clear to me which gets it more correct. Seems like they both fail at times.
> If it is commercial, be it closed source or not, an end user's recourse is to sue the supplier (or something similar).
Are there examples of doing this successfully? As far as I can tell, software manufacturers have largely been successful at avoiding traditional product liability for damages caused by malfunctioning software, through a mixture of EULAs and courts buying the "but software is different" argument. Here's an article series on that: http://www.newrepublic.com/article/115402/sad-state-software...
The commercial supplier has a financial incentive to get it right
Is this why Microsoft dominated the market for 15 years with the worst security model of all contemporary operating systems?
How many lawsuits were successfully pressed against Microsoft for losses due to their crappy security implementation? Forget about successfully, how many were even brought against them? Of those brought against them, how many were from companies not large enough to have their own legal departments?
>The commercial supplier has a financial incentive to get it right
Is this why Microsoft dominated the market for 15 years with the worst security model of all contemporary operating systems?
No, that's why Microsoft after XP tightened their security. Because they had an incentive to "get it right".
>I have better things to do than wait 15 years for a vendor to look at fixing a serious issue.
Perhaps, but that's just one aspect.
For most of those 15 years there wasn't a better supported, friendlier to the common user, with tons of desktop and business software and compatible with almost all hardware, OS available.
They had an even more incentive to get that right first, and they did.
> If it is commercial, be it closed source or not, an end user's recourse is to sue the supplier (or something similar).
That is assuming the supplier is still in business, which is probably a dubious proposition for a majority of commercial software that has ever shipped.
Something can be open source and used commercially. Apparently this bug was found via an audit by RedHat, which obviously is a commercial company that uses GnuTLS.
Don't mistake me as a zealot, though. There is a place for open source and there is a place for closed source. AFAIK, that is also ESR's point, and why he broke ranks with Stallman, who claims that closed source is evil.
There are also many reasonable ways for closed source software (or more accurately "non-free proprietary software") to make the sources available for review, but not give away all the rights. Like "Microsoft Shared Source Common Language Infrastructure", etc.
Of course it's better for software to be free / open source, but it's nonsense to imply that only open source software has the potential to be seen by "many eyes".
My eyes are still red and sore from staring at the MFC source code before the turn of the century.
Why wouldn't it be discovered? As I understand, it was discovered by audit of an interested party. That happens to closed-source software too. If GnuTLS were a proprietary product of RedHat, of sold by proprietary company to RH while allowing RH to audit, but not publish, the source, the result would be the same. Disclosure might not happen, but discovery still would.
What evidence is there that there actually were "many eyes" on this code? If anything this underscores the importance of license compatibility in order to maximize the utility of the "many eyes" resource. Honestly, GnuTLS seems to exist purely for ideological shim reasons, and it's not surprising that ideological shim reasons don't motivate "many eyes" to bother show up.
That's a No True Scotsman argument. The problem with the "many eyes bugs shallow" theory is that all eyes aren't created equally, and the valuable eyes aren't distributed uniformly across all software.
It would only be a No-True-Scotsman argument if the original statement of Linus' Law were, "In an open-source project, all bugs are shallow" and someone were now trying to claim that GnuTLS wasn't open-sourcey enough.
In reality, the law is about code that has many eyeballs on it, and it's a fair argument to point out that evidence suggests GnuTLS didn't have that many eyeballs on it.
Can you present some of that evidence about the lack of eyeballs on GnuTLS? Because my point is that the right kind of eyeballs were not on GnuTLS; my point isn't compatible with this supposed "law".
So are you saying that GnuTLS had "valuable eyes" and missed this or that they had "many eyes" and missed this? What exactly is your alternate hypothesis?
So do you mean that (until now) they just failed to find them, or that (until now) they were only found by people who did not have the incentive to report these bugs?
Because in the top post you call the bug "simple and basic".
But then in that same top post you imply (in your last line) that GnuTLS gets so very little attention, but now you say that lots of people have tried to find bugs in it.
I'm not trying to criticize you here, but trying to figure out what you're trying to say?
That's part of the fallacy. In the OSS world it's assumed that when code passes through many hands, is depended upon by many projects, and is critical to the functioning of many systems used by many users then naturally the code in use will be seen by many eyes.
But this is anything but true. The reality is that code review is far less common than code use, with many defects impacting many users as the logical consequence.
> That's part of the fallacy. In the OSS world it's assumed that when code passes through many hands, is depended upon by many projects, and is critical to the functioning of many systems used by many users then naturally the code in use will be seen by many eyes.
Right and as you point out that's not true. What gets less attention is why. I know in my own code there are dependencies I know very, very well. Some of them I have helped to author. Some of them I have helped with regarding maintenance programming later. But there are many which I don't.
There are a bunch of reasons for this:
1. Code has varying degrees of readability and is easily comprehensible to various extents. Code which reads like a book gets more time than that which I have to figure out its structure first.
2. Code that is well maintained provides less necessity to add someone else. I tend to be more likely to read poorly maintained code than well maintained code, for example because bugs don't get fixed is a good reason to try to fix them myself....
I think all that means is that "many users" != "many eyes".
I think ESR's ultimate point is that Open Source, if the security or bugs deeply matter to you, you can independently add your own eyes (or provide resources for the same). It's a very different matter with proprietary software.
Exactly. How can you trust something you don't have source code to? You can't, full stop. Any thinking otherwise is at best childishly naive ignorance. Given the recent revelations of the past few years, it should be glaringly obvious to anyone with half a clue that companies have been compromised, either willingly or otherwise, so that trusting closed source is a bad idea (as ESR, RMS and Bruce Schneier have been saying for decades!). The finding of this bug in GNUTLS is a good thing! Claiming that this bug would have been found at all, much less fixed, in proprietary software is galling, to put it lightly.
You can audit binaries. So definitely not "You can't full stop". Most people are in no position to audit all the source code they use, just like most technical people are in no position to audit binaries they use.
(And I'm putting aside the whole issue of backdoors in hardware, compilers, etc.)
Raymond wrote Cathedral and Bazaar almost 20 years ago. In the context of the time and the state of open software "all bugs are shallow" was a pretty damn accurate response to its critics. It still isn't a bad description in many cases.
There are always edge cases where beta testing and multiple eyes fall short of mathematical possibility [never mind sophisticated attacks]. That such a bug as this matters 17 years after Raymond made his remarks is a testimony to the robustness of the mechanism he described.
Im not an ESR fan, but let's be real here, the bug got found. How many bugs of similar impact are hiding in crufty old Windows code that nobody is looking at.
Unless you're talking about an IBM mainframe, validated at EAL 5, there are security bugs all over the place. With open source, you don't get a platoon of elves scanning the code, but you have a much better chance of someone happening across a defect or identifying the responsible party.
But how would you do know that those bugs in closed source systems haven't been found? The bug reporting systems are closed too, and the hotfixes and patches that get issued don't link to a list of bug reports. So you are none the wiser.
For all we know, bugs are found in closed sourced systems all of the time and are fixed frequently; the only difference is that they're not publicised.
No, actually, there IS a platoon of elves with millions of eyes, carefully scanning all of the open source the code for bugs, meticulously going over it line by line, building rigorous test harnesses, feeding it every input imaginable to probe its weaknesses, and writing up reports describing every quirk flaw they detect in it. And then not fixing it.
And your tax dollars are paying for all of that work. And you can be sure those elves have known about those bugs for years.
And those bugs have caused many unfortunate consequences. So it's just not wise to go around giving people a false sense of security in order to promote your brand.
> This is yet another nail in the coffin of Eric Raymond's irresponsible and fallacious "many eyes make all bugs shallow" theory.
What a facetious and unsupported assertion you have made.
You are seemingly purposefully disingenuous.
The simple fact of the matter is, ESR is still right. Perhaps because few people use a piece of s/w these things slip through. The seriousness of this flaw is limited and thus not subject to the fierce post humous questioning you give it.
Please define how many eyes saw or used or benefited from this code. I certainly live in the world of this code and don't depend on it.
> The simple fact of the matter is, ESR is still right.
ESR is not completely mistaken here, but the problem is that he's not exhaustively right with "Linus's Law".
It's more a description of why beta testing with access to source code is good, than a description of why open source is inherently good (and before you flip out, I've contributed to open source longer than I've done my day job).
Some types of code flaws will simply never show up to the kind of beta testing that introducing software to a wide population provides. E.g. proving that a X.509 certificate which is faulty in a certain way is actually caught by a software library; few people run into that in practice.
What these types of bugs require is code auditing (after the fact) or before-the-fact code review that prevents their entry into the source in the first place. But neither auditing nor pre-commit review are inherent to open-source, and in fact it could be argued that closed-source software companies are better able to ensure sure things happen.
The saving grace for open source is that these companies optimize for market success and not code quality (except to the bare extent needed for market success). Additionally you can pay to audit open source code much more easily than you can closed source (e.g. Google's audit teams that do exactly this).
But "bugs are shallow in the presence of sufficient eyeballs" is not unique to open source, either in theory or in practice.
It's clear that there aren't "enough (properly skilled, willing and available) eyeballs" for every project out there. Raymond's argument could be correct; still, the irresponsible thing would be to expect, just because of the openness of the code, that someone will ever care for no reason.
Straw man. The phrase "all bugs are shallow" doesn't mean all bugs will be found. It means that, once a bug is found, the correct fix will be obvious to someone.
Linus was directly aiming to maximize the number of person-hours thrown at debugging and development, even at the possible cost of instability in the code and user-base burnout if any serious bug proved intractable. Linus was behaving as though he believed something like this:
8. Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone.
Or, less formally, ``Given enough eyeballs, all bugs are shallow.'' I dub this: ``Linus's Law''.
My original formulation was that every problem ``will be transparent to somebody''. Linus demurred that the person who understands and fixes the problem is not necessarily or even usually the person who first characterizes it. ``Somebody finds the problem,'' he says, ``and somebody else understands it. And I'll go on record as saying that finding it is the bigger challenge.'' That correction is important; we'll see how in the next section, when we examine the practice of debugging in more detail. But the key point is that both parts of the process (finding and fixing) tend to happen rapidly.
In Linus's Law, I think, lies the core difference underlying the cathedral-builder and bazaar styles. In the cathedral-builder view of programming, bugs and development problems are tricky, insidious, deep phenomena. It takes months of scrutiny by a dedicated few to develop confidence that you've winkled them all out. Thus the long release intervals, and the inevitable disappointment when long-awaited releases are not perfect.
In the bazaar view, on the other hand, you assume that bugs are generally shallow phenomena—or, at least, that they turn shallow pretty quickly when exposed to a thousand eager co-developers pounding on every single new release. Accordingly you release often in order to get more corrections, and as a beneficial side effect you have less to lose if an occasional botch gets out the door.
And that's it. That's enough. If ``Linus's Law'' is false, then any system as complex as the Linux kernel, being hacked over by as many hands as the that kernel was, should at some point have collapsed under the weight of unforseen bad interactions and undiscovered ``deep'' bugs. If it's true, on the other hand, it is sufficient to explain Linux's relative lack of bugginess and its continuous uptimes spanning months or even years.
One can only conclude that either ESR is wrong (it wouldn't be the first time), or that ESR's beloved open source "bazaar" has become a cathedral.
Linus was directly aiming to maximize the number of person-hours thrown at debugging and development, even at the possible cost of instability in the code and user-base burnout if any serious bug proved intractable. Linus was behaving as though he believed something like this:
8. Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone.
Or, less formally, ``Given enough eyeballs, all bugs are shallow.'' I dub this: ``Linus's Law''.
My original formulation was that every problem ``will be transparent to somebody''. Linus demurred that the person who understands and fixes the problem is not necessarily or even usually the person who first characterizes it. ``Somebody finds the problem,'' he says, ``and somebody else understands it. And I'll go on record as saying that finding it is the bigger challenge.'' That correction is important; we'll see how in the next section, when we examine the practice of debugging in more detail. But the key point is that both parts of the process (finding and fixing) tend to happen rapidly.
In Linus's Law, I think, lies the core difference underlying the cathedral-builder and bazaar styles. In the cathedral-builder view of programming, bugs and development problems are tricky, insidious, deep phenomena. It takes months of scrutiny by a dedicated few to develop confidence that you've winkled them all out. Thus the long release intervals, and the inevitable disappointment when long-awaited releases are not perfect.
In the bazaar view, on the other hand, you assume that bugs are generally shallow phenomena—or, at least, that they turn shallow pretty quickly when exposed to a thousand eager co-developers pounding on every single new release. Accordingly you release often in order to get more corrections, and as a beneficial side effect you have less to lose if an occasional botch gets out the door.
And that's it. That's enough. If ``Linus's Law'' is false, then any system as complex as the Linux kernel, being hacked over by as many hands as the that kernel was, should at some point have collapsed under the weight of unforseen bad interactions and undiscovered ``deep'' bugs. If it's true, on the other hand, it is sufficient to explain Linux's relative lack of bugginess and its continuous uptimes spanning months or even years.
I think this largely shows why ESR is wrong here though. Let's start with two basic assumptions:
1. Some bugs are generally easy to resolve with one-line fixes (the Apple SSL bug being a good idea).
2. Some bugs are genuinely deep because they are design limitations of the software, or flawed assumptions on the part of the person who designed the software contract.
Now let's also point out that bugs of the second class may have an apparent shallow fix which in fact simply paper over deeper problems. A bug fix needs to resolve the issue, not just provide some cruft to make life immediately easier.
Certainly if you see Linux push back on patches, you see he is pretty heavily aware of that fact.
There are two things I have learned on this topic in my time as a programmer. The first is that the only really deep bugs are those which are design flaws. The second is that review in advance prevents problems, not eyes on the problems in retrospect.
You can't turn a deep bug into a shallow bug after it is already there. By the time you have beta testing going on it is too late. What you can do is have a few good people who design things well (and review eachothers' work there) and then deep bugs don't happen as frequently.
I don't have a link to it so I'll paraphrase what I remember:
A bunch of programmers were endlessly arguing back and forth about how to do such-and-such to emacs on the gnu-emacs mailing list.
RMS derailed the argument by pointing out that there were just a few people in the world who know the code well enough that they could actually just sit down and solve the problem themselves without discussing it with anyone else, and they were all very busy.
But all the bike shedding and social chatter about what to do, how to do it, what to call it, and what color to paint it, by the people who either can't or won't actually do it themselves, is just distracting and wasting the precious time of the few people who can just solve the problem themselves without any discussion.
In other literatures, there is a notion of diffusion of responsibility; I haven't read the works that you mentioned, but is such a risk risk mentioned in them?
I have never thought of that as the meaning. In fact, how does this wording apply at all to bug discovery when it seems to me discussing bug resolution, which feels like a very different thing? However, I have never read The Cathedral and the Bazaar, so I just took the meaning that made the most sense.
ESR is doing an "AMA" on /. today, and I posted asking about what he thought about this, specifically my theory is that everyone is ignoring the amount of code in proportion to the eyes that see it.
This not a case of GnuTLS not being audited, or audited by the right/wrong eyes. Audits were done; the Debian maintainers chose to ignore the conclusions.
Cryptography is uniquely difficult to audit precisely because it is encrypted. It requires more effort to tell what an application is doing if all its input/output are completely incompressible.
Maybe cryptography should be handled exclusively by the OS. Software would communicate through a protocol that can be easily audited (like normal http with special headers). Firewalls could be used to verify what is going on and normal tools could be used to inspect the data. That kind of checking is essential and doesn't require the talent needed for code review.
Correct me if I'm wrong, but I think most developers were "disappointed" with the "goto fail" because it was code structure issue which could and should have been caught at compile time. Hence the conspiracy theories because Apple should have these tools in place.
This one on the other hand, is a logical bug (interpretation of return codes) which is relatively harder to track especially when the use of said return codes is not consistent throughout the project
Say what you will about the half dozen mutually incomprehensible error code standards in Win32, at least 0 indicates success in all of them. Well, except for BOOLEAN.
I really don't understand why anyone would return an int and have 0 indicate failure. OpenSSL does this too, and it makes no sense to me. I don't know how far this goes back, but clearly we have a long tradition of programs returning 0 to the OS to indicate no error.
Using zero for errors means you can express error checking directly with C expressions that appear boolean. It's not as safe or rigorous, but is more readable.
Yes, you can, but that doesn't read very well to my eyes, given most function names. if(open()), if(read()), if (CreateWindowExW()), all suggest success to me.
if (!open(...)) goto fail;
if (!read(...)) goto fail;
if (!close(...)) goto fail;
reads a lot nicer to me. Although, the reality is that you need to assign to an error variable, which makes it not as nice and clean.
Is there any auditing technique that could realistically find out, after the fact, how often this long-outstanding bug has been exploited over the years it has been in place, like collecting a sample of presented certificates?
If we can't write perfect software, at least being able to quantify the damage would be a good start.
Well, I suppose it might be a good idea for clients in general to log all certificates as they are presented -- the amount of data should be negligible. For a more organized approach there's always:
I'm not sure if the data dumps contain the data needed to search for this bug in particular (but I suppose it should)?
I wonder if moving most logic from these libraries to lua or guile might be a good idea -- and keep only the bare minimum in C/assembler (as I understand it, the key crypto algorithms need to be in something like C in order to be able to guarantee (as much as possible) against side-channel attacks). But I don't know if there are any real reasons to keep everything else in C (other than possibly that making things easier to port/compile/etc?).
Yes, I agree: that is what conspiracy theorists do regardless of the facts on the ground. But that's tangential to my point. People on HN found the "goto fail" bug damning because of the specific form it took (among other reasons), and I'm happy to point out how damning that form turns out not to be.
The fact that GnuTLS is extraordinarily unlikely to host an NSA backdoor helps my point, which is why I brought it up.
However, I would agree with a hypothetical conspiracy theorist who would disagree with your assumption: Linux may be the platform for many high-value targets and companies, so it's not quantity, but..."quality" of a sort.
Is this an interesting observation? I doubt it a little. What security-conscious developer believes C is a good language in which to write complicated, security-critical code? You write code in C because you have to.
Actually people seem to need frequent reminding that we don't live in the best of all possible worlds, and that some of the difficulties in working with the best-available current tools aren't inescapable but rather shortcomings that can and should get fixed sometime. It's great if everyone now knows how suboptimal C-everywhere-forever is on security grounds, at least—but tbh 'security-conscious developer' sounds a little like 'true Scotsman' here. A hae me doots.
Personally I believe C is the best language to develop in. Anything you write in C can be easily used in any other language you care to use; this characteristic is not true for any other language you could name.
There is nothing wrong with developing security-critical code in C if you're an experienced C programmer and know what you're doing. The GnuTLS programmers were neither experienced C programmers nor knowledgeable about security; they should not have been writing security-critical code in the first place. And no one should have ever believed that their code was trustworthy.
Can we all finally accept the fact that what is most needed now is not a new whizbang but a comprehensive review of all of the core pieces of software that we depend on as a base for making new whizbangs?
How many critical and ancient bugs still remain undiscovered?
Everyday there is a new Telegram, Cryptocat, etc. all presumably being constructed on top of insecure libraries. What progress can we make with such shaky foundations?
There has been word that the Linux kernel devs are considering slowing new feature adoption for a time while focusing on bug discovery and elimination.
At a minimum, commonly used security packages should be reviewed and OS distribution maintainers commit to updating to latest versions of said packages regardless of the age of the OS.
For example, Ubuntu 12.04 LTS uses an older version of OpenSSH and OpenSSL. There should be no reason why Ubuntu (and others) can't commit to updating to the latest versions so that features in say OpenSSH 6.5p1 are avail. BTW, saying that you can compile and install this yourself is noted beforehand but honestly, how many people do that on a regular basis?
What I'm getting at is that security software can and should be held to a higher and current standard precisely because it affects so many other pieces of software in fundamental ways. It's not a big deal if the latest version of bc is not installed but it sure is if GnuTLS or OpenSSL is broken.
Disclaimer: My understanding of the goto fail bug is based on agl's writeup, and my understanding of the GnuTLS bug is limited to this article.
In both of these cases, it seems the code starts with the assumption that the cert is valid and then tries to find various conditions to invalidate it. Isn't that backwards? Why not start with an assumption that the cert is untrusted until you've proven otherwise?
That's not what happened here. The problem is that the logic of the validation predicate was broken. In fact, this bug happened after the code had determined the cert was invalid; it just signaled the error improperly.
This is an interesting question. I agree that invalid-by-default makes sense when reasoning about the higher-level system, but regarding the actual Boolean logic it seems that if a critical condition is broken the particular formulation will not make much of a difference.
Additionally, it seems that in certain situations having several plain-if conditionals set the valid state to `1` would be less robust than having several conditionals set it to `0`. In the first case if any one of the conditions have bugs the false valid state can happen, whereas in the second case the last conditional to execute would have to contain the bug.
Can anyone else comment on this? Are there relevant theories and/or best practices which have been discussed in the security community?
What about several conditionals that set various elements of a struct to 1, and the cert is valid iff all elements are 1. At least that way a skipped check (i.e. goto fail;) wouldn't result in a false positive...
So basically abstract the details of each sub-condition from the final check that all sub-conditions are met?
That does sound like a good idea as long as the meaning of `1` and `0` do not get confused, and as long as we're not talking about some niche context where extra overhead for the sake of code maintainability would be a problem.
Is it that they just now started actually testing these implementations, for the first time? Even if you wouldn't follow TDD practices, it's obvious that anything that implements a protocol of some sort (particularly a security protocol), should be tested to actually follow that protocol. Isn't that common sense? What is going on?
TLS stacks aren't built with TDD practices for the same reason that the kernel TCP/IP stack isn't built with TDD: the things that go wrong in TLS involve the state machines of a distributed protocol and somewhat elaborate mathematical operations.
Read Schneier and Ferguson describing in _Cryptographic Engineering_ the steps required to validate a bignum arithmetic library. That's one of the smaller problems in getting a TLS stack right, and it's complicated.
If you demand that your browser implement protocols the same way ActiveRecord implements SQL join generation, you should just stop using browsers; none of them are built the way you'd want them to be.
I have a hard time not being dismissive about TDD's role in improving systems software security. If you're going to propose a radical change in the way systems code is written (or, worse, hold systems developers to a nonexistent standard), be intellectually coherent about it: demand that security-critical code be implemented in a rigorous language, like idiomatic Haskell. Don't propose monumental changes that promise only that the resulting code will be asymptotically secure as Ruby on Rails.
With all due respect I must say that I find that you underestimate how much tests help.
If I implement a network protocol I will make sure to write automated test involving clients and servers. Setting this up on localhost or on a virtual network using TUN/TAP is not that hard. And this has made me find TONS of bug ahead of time.
I hear a lot of arguments as to why testing networked or otherwise distributed things isn't necessary. But just look at Aphyrs complete destruction of well-known distributed systems by using realistic testing.
And don't claim that the kernel network stack isn't tested. It's just tested by hand. Plus, there are projects like autotest. Without testing, I don't think there's any chance that the kernel devs could release any more versions.
Safe languages, formal verification, using proven methodologies and testing are all separate methods of improving code quality. But I don't believe any one precludes the other.
I'll end with a famous quote from Knuth: "Beware of bugs in the above code; I have only proved it correct, not tried it."
>If I implement a network protocol I will make sure to write automated test involving clients and servers. Setting this up on localhost or on a virtual network using TUN/TAP is not that hard. And this has made me find TONS of bug ahead of time.
I agree that it's helpful, but the problem is coverage. Your test code, most likely, doesn't cover EVERY single possible condition that can happen with a simple TCP/IP connection. Especially once you get out of localhost land, where you're dealing not only with your code, but all the hardware and software between the two systems
The fundamental problem is that even with a rigorous test suite, you're probably going to run into things you didn't even think were possible once the code is out in the wild. For example, we just ran into a scenario where we were seeing corruption through a TCP connection. Knowing the wire, it was impossible for the packets to be appearing in the way that they were(there was some packet level corruption). After going through multiple wireshark logs, we found that the culprit was a hardware firewall in between the server and client. Thankfully, our code didn't crash, but it's also something that we never tested for, because(in theory, at least), it should never be possible for that specific corruption to be sent in the first place.
Not sure I can agree with the statement about TCP stacks. Yes, kernel TCP stack aren't built with TDD. But it's not a fundamental property of the problem space, it's just another instance of kernel programming sucking in general.
A TCP stack can be written in a way that makes it simple to test, and getting full branch coverage isn't that onerous a task. Does that catch all bugs? Of course not. But it'll still catch many simple logic errors like this one. Such logic errors do creep into code quite often, can be hard to trigger or detect during manual testing, and will be hellish to debug if they manage to get all the way through to a live system.
(This isn't idle speculation. I'm the lead engineer for a high performance userspace TCP stack that's handled petabytes of data, probably for millions of endpoints. We have deterministic unit tests for a large proportion of TCP behavior. Getting into a position where we could do proper testing took a while since the initial version was not written with testability in mind, but that work was definitely worth it.
That's not what he said at all and your comment dips into "the elite few know more than you do" territory, particularly with the unnecessary dig at Web development. His only mention of TDD was "even if you don't," yet you jumped on the acronym TDD and took him to task for it.
What I read him as saying (and agree with) is that he's surprised that GnuTLS isn't tested against known-bad certificates in a simple integration test that doesn't require a Ph.D. to set up, as you imply.
What "known-bad" certificate? You appear to be suffering from hindsight bias. We know what the bad certs look like now, but they're idiosyncratically related to a very, very specific bug: having the test case that checks for this bug implies that you don't have the bug to begin with.
The fallacy that you could have easily written a test case for this bug appears neatly encapsulates the weakness of "TDD" as a mitigation for programming flaws.
My point wasn't the specificity of the argument, merely that you had misinterpreted the argument and gone after its originator misguidedly, to the conversation's detriment, something you're continuing to do with me by accusing me of hindsight bias.
Your résumé is well-known around here and you don't have to set the terms of the discussion about every security-related issue on Hacker News, particularly when it's this heavy-handed and you get recognition upvotes. All I'm saying.
I no longer have any idea what you're talking about. If you want to pose an argument about how TDD applies to building things like TLS stacks, I will read, consider, and respond. Debating TDD for TLS stacks isn't a waste of time, even if I do think the debate has a foregone conclusion. If you want to talk about me personally, I'm not interested.
Him: Can't you test for this somehow? Even without TDD...
You: TDD is bad because math and wall of text.
Go read a book and realize why you can't, Web hacker.
Me: You jumped on TDD unnecessarily, there.
You: Now I'm going to redirect the argument to you!
You're browbeating anybody that comments here, rather unnecessarily, almost as a display of expertise.
I'll address the sliver of this comment that is actually relevant to the thread: the distinction between "testing" and "TDD" is not important to the point I'm making.
Reading your original reply, I walk away with these four points from your four paragraphs, now with the knowledge that wherever you typed "TDD" you could mean "testing" as well.
- Testing distributed cryptography is difficult.
- Read a book.
- You're a web developer and don't know the first thing about
testing TLS stacks, clearly.
- Testing does not improve systems security, as evidenced by
Ruby on Rails: Rails uses testing and it's insecure.
Seriously, re-read your comment. More than half of it is just unnecessary and dilutes what little point you've made into something unrecognizable in the snark.
Testing does not improve systems security, as evidenced by Ruby on Rails: Rails uses testing and it's insecure.
This is an important and valuable point. He wasn't discouraging new ideas, but rather pointing out how hard it is to make new ideas practical in the security arena.
Well, lets assume gnutls had a test suite that tested for a few known-bad certs. Quite possibly the conniving bastard that tried his hardest to break tls by grinding out various inconceivable certs wasn't able to stumble on this particular case -- so the bug wouldn't have been found through TDD alone. Now that it is found, the cert could (trivially) go into the test suite, and (parts of) that test suite could be reused by anyone writing a TLS library from scratch.
It's hard to see how that test suite couldn't help improve the reliability (and therefore "reduce the insecurity") of said new implementation (say a library with only support for a subset of tls1.2 -- without any support for fallback).
>What I read him as saying (and agree with) is that he's surprised that GnuTLS isn't tested against known-bad certificates in a simple integration test that doesn't require a Ph.D. to set up, as you imply.
Define "known-bad" in a general enough way that a specific test can be created to cover the entire range of "bad" certs. That's quite difficult, and, probably isn't realistically possible to go through all the "known-bad" if you want your tests to run quickly.
Realistically, all you can do is have regression tests to make sure that the found bugs aren't repeated in future releases.
> Define "known-bad" in a general enough way that a specific test can be created to cover the entire range of "bad" certs.
Any known-bad cert at all would have been quite sufficient to catch this bug apparently. A simple ARE WE ACCEPTING BAD CERTIFICATES LOL sanity-check would have found it, which is the kind of unit test it should be possible to think of in advance rather than in response to a specific bug found earlier. A little can go a long way.
EDIT: Additionally, the difficulty of catching all bad certs is good reason to develop and continually update a torture-test of invalid certs (and valid ones) to test SSL clients against. The suite would be much too slow to check against once per recompile, but testing once before each point release should be useful enough...
My thoughts on this are more about the previous TLS disclosure, not this one, where an integration test that says "this invalid certificate should fail validation" probably would have caught it. But, this isn't my argument to make as I don't know enough about it, I just want to see the actual coherent argument made instead of a side tangent as we're seeing here.
What integration test? Writing that integration test implies that you know the bug is a possibility. Once you consider that possibility, a glance at the code tells you if it's there or not. It's not like this bug is likely to regress, either.
Again, I'm talking about the "goto fail" one, not this one, in which a simple test of the library against an invalid certificate would have caught at least one case that I recall. And I think it's pretty obvious that "you might erroneously validate a bad certificate" is a bug that's, you know, a given.
Feature: *Certificate Validation*
In order to *keep NSA from reading my emai*
As a *TLS X.509 validation library*
I want *to never erroneously validate a certificate*
What are the "Scenarios"?
Given: *???*
And: *???*
When: *???*
Then: *the certificate should be rejected*
Remember, if we're switching topics to the "goto fail" bug: that bug didn't affect every instance of certificate validation. You had to be in a particular set of ciphersuites.
I agree with this demonstration of why acceptance tests are not going to prevent bugs like this, but I'm not so sure about unit tests. I've seen good unit tests reveal a lot of similarly difficult-to-spot bugs. Granted, a really good type system like Haskell gives you more guarantees than a test suite in less rigorously typed languages ever can, but I think you're too quick to dismiss unit tests out of hand. It's a tool in the quiver, and the reason TLS stacks don't tend to have them is not because they are not valuable, but probably because the developers weren't any good at unit testing.
I spent a few minutes replying to your original comment, then you ninja edited in the closing paragraph and invalidated mine, so this is where I end the conversation. I can't reply to an ever-shifting comment with someone who reads what they want to read out of my comments and invokes the NSA for no reason whatsoever.
You'd be a happier person if you didn't assume people were editing their comments just to make you look bad. You can be assured that if I had seen you respond to something I'd written, I'd have noted where I added to my original comment.
But I agree that you & I are better off not discussing things.
I'm quite a happy person and thanks for the irrelevant concern trolling, based on a comment thread that's calling you out (do you always assume people that think you're wrong are unhappy or otherwise defective?). And no, I didn't assume those words you've placed in my fingers. I was, rather, annoyed that I invested a few minutes into a playing field that you'd already moved while I worked.
If you'd permit me a brief bit of my own concern trolling: I remember when I looked forward to reading your comments, several years ago. Now I see your nickname and say "bah, again?" What changed? Was it me or you?
I'm a firm believer in regression tests for complex systems. Sure, this specific case is extremely unlikely to come up again, but in general, it's always good to have that coverage, especially if a large refactor happens later on.
I'm definitely not arguing against regression testing, or really even against test suites for TLS (there should be more of them, and better ones). I'm suggesting that the discipline of aggressively testing code for coverage is unlikely to make as much of a dent as moving to a better language would, and since both are extremely costly changes to the way TLS stacks are developed, we might as well adopt the one that will make us safer.
I claim that writing an extensive test suite for TLS is not nearly as difficult as switching people to a new language. Rewriting in a stricter language, maybe. But Haskell has a big runtime and garbage collection and its own compiler and would be a big pain to integrate everywhere that uses C TLS libraries; smaller compile-to-C languages might be easier, but who wants to use an experimental language to develop crypto code? ...And you'd still want to test it, because although functional programming style makes many bug classes less likely, it's not a panacea. You're still basically hoping that the developer doesn't make a single thinko.
Compared to that:
- Better unit testing is much easier to integrate into existing projects. Yes, it can only prevent a bug if the developer generally thought of the class of error, but at least it sort of forces them to spend some time thinking about possible failure cases, and can detect cases where their mental model was wrong. Also, it helps detect regressions: "goto fail" wasn't a strange edge case the developer didn't think of, it was a copy paste error which good unit tests could have caught.
- Functional testing can be independent of the implementation and written by someone unrelated. They can only do so much in general, but they might have caught both of these bugs.
Yes, audits are another option, but I'd say they should complement tests, not replace them.
ed: oh, and if you want to be really intellectually rigorous, you could try to formally verify your C code; model could have bugs but could also be implementation independent. But I hear that's rather difficult...
It's interesting to think about, but I'm not as bullish.
Stipulate that we're just talking about X.509 validation. (You can still have "goto fail" with working X.509, but whatever).
Assume we can permute every field of an ASN.1 X.509 certificate. That's easy.
Assume we're looking for bugs that only happen when specific fields take specific values. That's less easy; now we're in fuzzer territory.
Now assume we're looking for bugs that only happen when specific combinations of fields take specific combinations of values. Now you're in hit-tracer fuzzer coverage testing territory, at best. The current state of the art in fault injection can trigger these types of flaws (ie, when Google builds a farm to shake out bugs in libpng or whatever).
Does standard unit testing? Not so much!
Would any level of additional testing help? Absolutely.
But when we talk about building test tooling to the standard of trace-enabled coverage fuzzers, and compare it to the cost of adapting the runtime of a more rigorous language --- sure, Haskell is hard to integrate now, but must it be? --- I'm not so sure the cost/benefit lines up for testing our way to security.
For whatever it's worth to you: I totally do not think code audits are the best way to exterminate these bugs.
I appreciate that testing a TLS stack is a major pain. But I'm a bit confused about "move to Haskell" in this context; sure, it would cut down on the buffer overflows, but TLS stacks usually fall to logic errors and timing attacks, not to buffer overflows. "goto fail" can occur in Haskell too: a chain of conditions can still incorrectly short-circuit.
Also note that Haskell doesn't exactly help in avoiding timing attacks. In a sane cryptosystem, you might be able to implement AES, ECDSA and some other primitives in a low-level language and use Haskell for the rest; but as you know, TLS involves steps like "now check the padding, in constant time" (https://www.imperialviolet.org/2013/02/04/luckythirteen.html). You could certainly implement those parts in C, too, and then carefully ensure that no input to e.g. your X509 parser can consume hundreds of MB of memory, and so forth, but you're going to lose some elegance in the process. (Those problems would admittedly be smaller in OCaml, ADA or somesuch.)
I'd be more interested in something like Colin's spiped - competently-written C implementing a much simpler cryptosystem. If only because even a perfect implementation of TLS would still have lots of vulnerabilities. ;-)
(I think the case for writing applications in not-C is considerably stronger, if only because TLS stack maintainers tend to be better at secure coding than your average application programmer. Like you, I do like writing in C, though.)
This is bullshit. You can write tests for TLS libraries just fine. And yes, I'm saying that without ever doing any programming of TLS libraries (just using them).
For certificate verification, you craft various correct and incorrect certificates to cover the various scenarios involved in certificate verification.
For communication related code, you talk to the tested code over TCP or internally as a transport layer implementation (assuming the TLS library supports user defined transport layers). Same as above, you identify the various code paths and attempt to test them all.
> for the same reason that the kernel TCP/IP stack isn't built with TDD
I believe the primary reason why testing kernels is hard is that they are intended to run on real hardware and often with strict timing expectations. This is not the case for TLS libraries, which normally run in user space.
If you knew every possible incorrect certificate format, or even every possible form a valid certificate could take, your comment would be more compelling. But of course, you don't. Nobody does.
I think this is unnecessarily pessimistic. You know what validity checks the code performs. To catch this bug it would only have been necessary to make each of them fail and make sure the failure led to rejection.
It's true, that doesn't prove the implementation is perfect, which seems to be what you're holding up as the goal. But it would have caught this bug.
If you're writing that code, you SHOULD know every possible incorrect certificate format. If you don't, you should learn from the relevant standards and other materials, or admit that you're not capable of writing that kind of code.
But in this case this is not about that. It's about testing code already written. All you need to do is go through that code and verify that every branch does what is supposed to do. Which is pretty straightforward, if time consuming.
> If you're writing that code, you SHOULD know every possible incorrect certificate format.
If we define a correct format there are likely an infinite number of incorrect formats, no? A test explicitly checking for this bug would prevent a fix from regressing, but it seems to write a test that exploits this bug before understanding the bug itself would require quite a bit of
luck.
EDIT: I'm re-reading the initial advisory and trying to decide if this applies to any cert with a ROOT CA fail or just a specifically crafted one. If it's the former my initial comment is garbage.
If you classify incorrect formats by what they get wrong, you should be able to limit the number. For example, in terms of C strings one invalid format is 'embedded null'. It doesn't matter if that null comes from jpg data or mp3 data because those distinctions don't exist at the 'C string' level.
I find this comment confusing. You claim the testing isn't done because the issues are either in state machines or in the math... yet we're looking right now at a case where neither failed and some simple unit testing would catch the issue. Isn't it a false dichotomy to say it's not worth unit testing (not TDD) unless we switch to safer languages? If the code is less secure than RoR because of lack of testing, then what's wrong with getting it to that level first? At least it won't require a rewrite from scratch.
This bug is entirely random. It causes certificate validation to break at a point where certificate validation never breaks. Obviously, once you know it's there, a unit test keeps you from ever having the bug again. It's less apparent to me that it helps you before you realize that the bug could exist.
Unit tests may be written before or after the production code in question has been written. They provide no guarantee of the correctness of the final code, but they may find small things. Corrective unit tests as you've described them will not help you find this bug.
But finding bugs like the above is TDD's bread and butter. TDD dictates specifically that the test must be written first, that it must be isolated to a particular spot as much as possible, and the dumbest piece of code possible must be written in order to allow the author to move on to writing the next test. Someone TDDing this method would stub out any required external state in order to focus only on the piece of code in front of them.
The final system may not be correct––you must still perform high-level testing as you always would, and you must understand the rules of the system you're building. But you're apt to avoid the simple stuff.
I don't think i can agree with "random". Anyone going for maximum branch coverage would find this problem. Also every point is where validation never breaks... until it does. It's like the apple openssl bug - trivial to find, provided anyone actually tests the code they release, trying to exercise all code paths.
You don't have to follow TDD at all in order to build a test suite for a project. I think all anyone's suggesting is that some of the complicated nature of getting crypto right actually can be put into a suite of tests.
I know I could come up with a very basic set of test cases, and my only experience with TLS is reading parts of some book I found in my company's library. Heck, OpenSSL has a ton of tests it runs on every build, I know this much.
And even in Haskell you'd need tests. At no time will you ever be able to write code without testing it.
I do wonder if using Haskell and property based testing in a TLS library would catch many of these bugs. Would there be no value in testing against randomly generated certificates?
It's definitely not a rocket science to have the test examples with the crafted certificates that are not supposed to pass the tests and to have such tests run at every build.
Most CPAN Perl modules for almost two decades have such tests for much less critical stuff. There's no excuse for not having such methods for security critical code.
Tptacek is muddying the waters by claiming it's hard. It's not. We have more different implementations of the same protocols so it's even easier to cross-verify.
- Here's a library that claims to validate certificates.
- Here's a TLS httpd with a forged commonName.
- for ciphersuite in $supported; do
Completing this test is an exercise for the reader, yet we're waxing philosophical about the perils of Ruby on Rails in this thread for some reason. If I'm being told by a security expert that such a framework would not be helpful, that's concerning and makes me wonder how many bugs such a framework would uncover (given that this one remained untouched for a decade).
This is very close to some of our integration test setups. It's literally a pair of folders "good inputs" and "bad inputs". To run our integration tests, we just throw each input file in those folders at the application and check if it works or if it crashes.
Some of the stuff has assertion files, too, so you can check the way it fails or check some conditions the success, but that's not common.
Overall, this took like a day or two to setup and it catches a lot of errors already, especially because people can just drop errors into the bad input folder and be done with it until someone has time to handle it.
That would be a more compelling argument if any other TLS stack had ever had this particular validation bug in it. As it stands, the test case that you suppose should have existed is specific to exactly the code in GnuTLS, and applies to no other stack.
What is GnuTLS specific in "The failure may allow attackers using a self-signed certificate to pose as the cryptographically authenticated operator of a vulnerable website and to decrypt protected communications"? [1]
Apparently just a self signed cert. It was accepted as the "CA signed." Since 2005.
The bug in this case was in fact one which could readily have been caught by unit tests, no? I don't see why demanding test coverage is intellectually incoherent. Arguably it's somewhat half-assed, but not incoherent. Why shouldn't systems developers at least pick up the low-hanging fruit of tests? The effort isn't too monumental for Rails.
Testing error handling is sadly uncommon. Lots of programmers simply run the code in the success case and call it good. Even those who write automated tests only write tests for the success case. I wouldn't be at all surprised if the author of this code either didn't test invalid certificates at all, or only tested them briefly in a way that just happened not to trigger the bug.
Yeah. While they're at it, why not change every function to return "int". Then just cast to proper type at usage. /s (And no, bool is not the proper way to go either.)
But seriously, I'm amazed that code this critical uses sloppy typing. I mean, they might as well use Python ffs. Maybe it's historical burden?
While I agree with most of these two sentences, you can't really be sure there's no such bug in OpenSSL. I mean, not this specific bug, but a similar hard-to-imagine-random-situation bug.
GnuTLS exists as a project almost exclusively due to ideology concerns -- people who prefer GPL do not like OpenSSL's license. That's not to say that technically-capable people are not behind it, but there are not as many as involved with OpenSSL. There are very few technical advantages to GnuTLS, and almost all of them have been transient (as in, certain protocols and ciphers have been supported in GnuTLS before other libraries). I do not have the time to go over why other libraries occasionally lag behind GnuTLS, but I can specify, for instance, that OpenSSL code goes through a more thorough and lengthy review process before it gets 'shipped' (inasmuch as that term applies to not-for-profit software).
Code that supports GnuTLS can easily be made to support OpenSSL (or several other such libraries). It really is a matter of which license you prefer, and licensing should never be a consideration in security.
So which projects actually use GnuTLS as opposed to OpenSSL (Apache, lighttpd, nginx) or NSS (Firefox, Chrome)? I'm aware of Debian linking some packages against GnuTLS to avoid the supposed GPL incompatibility issue with OpenSSL, but the only thing I've seen that explicitly uses GnuTLS is Emacs 24.
Back when I still used Apache it didn't have support for TLS 1.1/1.2 due to the dep on OpenSSL and thus none of the better FS ciphers. So compiling against gnuTLS allowed for those. That's now been rectified.
SSL fuzzing seems like it could be a really profitable area of research. I wonder why there aren't better conformance testing tools for SSL (other than SSL being kind of hideously baroque).
I can't speak for absolutely every package but most if not all on Arch will be dynamically linked and shouldn't need to be recompiled. (The new version was released on Arch 3/3)
Aptitude tells me there are 179 packages dependent on libgnutls26 - about 20 are installed including: chromium-browser, cups-daemon, libldap, librtmp, libvncserver0, libwireshark3, telepathy (KDE), vlc-nox.
The only thing I really know I use TLS for is email server connections via Thunderbird however.
Are you sure? At least on Debian, GnuTLS is used by CUPS, exim4, cURL/GnuTLS, GnomeVFS, Gtk, NetworkManager, libvncclient, libvncserver, VLC, and wget.
Does this make anyone else's conspiracy antenna twitch a bit? The Apple one was only a little suspicious in that way. A bug in another major crypto library that weakens it in a very similar way makes it look a little more suspicious. Since this is OSS, the investigation of the source control logs will probably be a bit more public. I wonder if they'll find anything interesting.
I am not very familiar with low level C or the coding patterns used with it, so I must ask; why do we see goto statements in both of these recent bugs? As far as I know it's a bad practice overall, is there some specific reason for using it here?
In short, it's not a bad practice in all cases and, like most issues, it depends.
The "goto considered harmful" article by Dijkstra is more nuanced (and that wasn't even its intended title) and doesn't really address the way the goto you are seeing are used.
They are a very common pattern in C, for error checking and cleanup in case of error. It's much more readable than to nest conditional statements.
if (some_parameter > max_valid_value)
goto error;
if (some_function() <= )
goto error;
...
...
if (some_alloc() == NULL)
goto error;
...
...
...
...
error:
// cleanup what needs to be freed,...
It can also be quite useful to exit nested loops for example. It's commonly used in low-level C.
However, when you start goto'ing backwards instead of always forwards, it becomes terrible for readability, so it is avoided (well, most of the time).
Since C has neither garbage collection nor the RAII pattern of C++, it is very common for C functions to have a cleanup section at the end, where allocated memory and other resources are freed again. In many C projects (including the Linux kernel) it is a common and accepted pattern to jump into the right position of the cleanup sequence using a goto.
Goto is an acceptable and idiomatic way to implement error handling. If something goes wrong in an inner loop, you can just break out of all the loops and go to the failure state.
goto statements are not necessarily a bad practice. Many people have heard the title from a famous letter by Dijkstra, 'Go-to statement considered harmful' (see https://www.cs.utexas.edu/users/EWD/ewd02xx/EWD215.PDF). In it he argues against the unrestricted use of goto statements making control flow hard to follow. The title is catchy and stuck in the collective computer science consciousness. It lead to all kinds of 'X considered harmful' articles.
The correct way to do error handling in C is using goto statements. Dijkstra did not mean local goto statements. He meant something you will not find in any C code you are likely to use on a rhel/debian machine.
It's not just that, it also doesn't have automatic destruction of objects, so any local array or structure containing a pointer needs to be laboriously freed by hand.
And in other words, nobody should be writing anything in C. Ever. The need to be compatible with programs written in C violates my rule against writing anything in C and is therefore not a valid reason to write crypto stacks in C.
Pure C does not have automatic destruction of objects, but gcc has an extension to do it. Take a look at systemd's source code to see it in action. It's very similar to C++'s RAII, and can also do things like closing file descriptors.
Would love to hear the answer to this as well. For the types of thing C is used for, I do not know of another language that can _practically_ be used for the same thing, while maintaining the same level of performance. Rust looks promising but is not mature yet.
Isn't Go designed to fill this niche? It's supposed to be a low-level systems language, but with "friendlier" features than C, and built-in concurrency support.
They kind of recanted that statement and said they meant it for "web servers" and stuff like that. However I know there has been at least a window manager made in Go.
I don't think it's the best fit, but it's a lot better than many current solutions. I personally prefer Haskell for anything others would write in Go, however many may find Ocaml or Erlang to fit them better.
However I'm biased and believe that functional programming is a better fit, more bug-free, easier, and simpler for most use cases than imperative programming.
Before there was C, there was Pascal. You didn't have malloc in Pascal. At ETH Zurich the standard OS was written in Oberon: it ran directly on the hardware with a garbage collector collecting everything.
Go also has a very highly quality crypto implementation. It has all of the good-code features that gnutls lacks: comments that make sense, unit tests, automatic memory management (of course), useful types, etc.
How are exceptions like gotos? I don't see any similarity. If by "essence" you mean they jump to a specific line of code, then lots of things can do that - switch/case , if/else, do/while, function calls/returns, etc.
Apart from the other reasons mentioned here, it probably translates to compact code which fits well within level-1 instruction cache in modern CPU when translated into assembly, without any need for additional lexical scopes and stack allocations.
goto is just a tool and not to blame here. I don't know what you could call the root case of this one. Incomplete testing maybe?
In Apple's case "goto" made it possible for the bug to occur, but there's no reason a different statement (or other typo) could cause equivalent damage.
I think a common thread with both of these bugs is testing the failure cases, but to be fair to both validating software by finding bugs and/or validating functionality is affirming the consequent and you eventually need to ship.
Demand unit tests and comments. I mean, I was incensed by the quality of the Apple code, but this code is just as bad:
* Checks if the issuer of a certificate is a
* Certificate Authority, or if the certificate is the same
* as the issuer (and therefore it doesn't need to be a CA).
*
* Returns true or false, if the issuer is a CA,
* or not.
*/
static int
check_if_ca (gnutls_x509_crt_t cert, gnutls_x509_crt_t issuer,
unsigned int flags)
{
Sure, returns "true" or "false" which are not things in C. This actually returns either 0 or 1. Many C library functions return 0 for OK and non-zero for error. This one returns 0 for error and 1 for OK.
Imagine you are reviewing the a call site of this code:
if (check_if_ca(...)) {
// CA is valid, proceed
}
vs.
if (check_if_ca(...) == 0) {
// CA is valid, proceed
}
To the reviewer who is accustomed to strcmp, either of these might appear to be fine, meaning that the reviewer (or even the author) has to carefully check the documentation of this function.
What else is wrong with this signature? For one thing it would be easy to swap the arguments around:
if (check_if_ca(cert, issuer, flags)) ...
if (check_if_ca(issuer, cert, flags)) ...
What's the difference? They both look reasonable. What are the valid/relevant values of |flags|?
Would it be easier to comprehend |flags| if it were an options struct, e.g.
Would it be easier to verify the correctness of a call site of check_if_ca if it looked like this:
if (issuer_cert.IsCA() || cert == issuer_cert) ...
Where are the unit tests. The unit tests should be right here in the lib directory as whatever_unittest.c, adjacent to the source file they are testing, so you don't have to go groping around in the tree trying to find the relevant test, if it even exists. By the way, I found no test that calls this function.
Basically this code has all the same problems as the Apple code, except for the mixed tabs/spaces stuff (as far as I can tell). It would be shameful for any professional programmer or organization of programmers to create this code.
It's a strange thing, but C99 features seem very slow to catch on in the C community, particularly features conceived specifically to alleviate bad "C-isms". The only explanations I can think of are ignorance (not really an excuse after well over a decade) or possibly a subconscious fear that they somehow make a program "less portable", even though basically every compiler supports it (and besides, I've never noticed any aversion to trivial features like //-style comments).
I'm not sure which features you're thinking of. Some are no-brainers: snprintf() is a clear win, and in my experience, it's widely used. I also really like the ability to define structured objects inline using named members, sort of like the way you do in JavaScript. That can make code a lot more readable.
A lot of the rest is just sugar (e.g., C++-style comments). Moreover, many code bases already evolved patterns for dealing with the problems that many C99 features are supposed to address. Booleans are a good example: many environments already have a fine boolean_t definition, and it's not really worth it to convert existing code or introduce a second pattern for representing boolean values.
Variable-length arrays are another good example. They're fine, but alloca() is a perfectly reasonable solution to the same problem, and it's already widely used. IMO, the safety issues usually brought up around alloca() are neither more likely nor more serious than problems like allocating a gigabyte-sized object on the stack or returning a pointer to stack memory. Since competent C programmers don't generally make those mistakes (IME, of course -- I think I've never debugged a problem that turned out to be a misuse of alloca()), I find the added complexity of a new language primitive isn't worth it. (Incidentally, that's the same reason I dislike nearly all of C++.)
> C99 is substantially completely supported as of GCC 4.5 (with -std=c99 -pedantic-errors used), modulo bugs, extended identifiers (supported except for corner cases when -fextended-identifiers is used), and floating-point issues (mainly but not entirely relating to optional C99 features from Annexes F and G).
> The support for standard C in clang is feature-complete except for the C99 floating-point pragmas.
Compiler/library support for C++ is much less complete(standards were published in 1998, 2003 and 2011, and none are completely implemented in any compiler) but that clearly didn't stop people from using C++. The reason is that the commonly supported subset was useful enough. The same is true of C99.
I think it was mostly Microsoft that was holding C99 adoption back by refusing to support it in their compiler suite (officially, they support C++ and C89/C90 only). That means you can't generally compile C99 code with the Microsoft C/C++ compiler, which meant you had to avoid either that compiler or (if you care about portability) the C99 standard. I don't think limited C99 support in GCC or Clang was a limiting factor to anyone in the past decade.
Compiler/library support for C++ is much less complete(standards were published in 1998, 2003 and 2011, and none are completely implemented in any compiler)
Impressive! I see that exported templates got scrapped in the latest standard -- they probably never supported that.
In any case, my point wasn't that Clang was bad (I think Clang is very good) but that the few limitations that might exist in Clang or GCC are probably not holding back C99 adoption.
I'll just note, that after coming back to C after a while in ruby/python/ungh, perl land everything in C99 and C11 has been a welcome change.
Seeing lack of adoption to things like just stdbool.h or even C99 initializer syntax is somewhat amusing. I understand backwards compatibility and all but there seems a general unwillingness to abandon C89 which I can only think is due to the microsoft toolchain.
As to c++ support clang has been really spearheading the implementation of new c++ standards and last I recall they even found bugs or inconsistencies. It always pays to have at least one implementation before standardizing I think.
What's necessary is to get at least two experienced cryptographers thinking about every piece of security-critical code, because that's what will prevent bugs. It's also why security audits are so expensive: they work.
The code absolutely should have had clearer comments, since if every step were commented then some of the return value confusion may have been avoided. The style of the code itself probably isn't as terrible as it seems at first glance, though.
It's also not easy to write useful unit tests for a security stack.
Pedantic point: A unit test would not have found these. Presumably in a unit test you would mock check_if_ca. Only an integration test would have found this.
Sorry for picking on your comment in particular. I could have replied to any number of comments in this thread.
If not expert cryptographers, than at least expert C programmers. I was trying to be diplomatic, but blunt is perhaps better for clarity: no expert C programmer should have any trouble reading and comprehending the syntax of the code in question, which is mainly what the parent poster was calling out. https://www.gitorious.org/gnutls/gnutls/commit/0fba2d908da6d...
It's probably tempting to think that unit testing might save us from security failures. People have written many words on both sides. But how many of us have actually tried it? No one who hasn't tried (and tried hard) to see whether unit testing works has any business arguing for (or against) unit testing. We simply don't have enough data to know whether it's valuable. And the fact that unit testing works in domains other than security isn't evidence that unit testing would produce any kind of benefit in computer security.
What's getting lost in the noise here, I think, is that it's important to zoom out and think about a broad overview of how difficult compsec is. Think about all the different situations that require security. Even something as seemingly simple and straightforward as an OS-level copy-paste mechanism requires thinking about security.
Could you write unit tests to cover all security situations? How about all possible situations? If not all possible situations, then why? What's the difference between a situation that you can unit test and one that you can't? Is it possible to write a unit test that checks whether there's a sequence of steps a user could take to exploit an OS-level copy-paste mechanism? Can you unit test the efficacy of a TCP/IP stack? Is it possible to unit test whether the Debian OpenSSL PRNG is being fed with sufficient bits of entropy? Can you unit test whether the colors of your website are pleasing?
The most interesting answer to those questions isn't "yes" or "no." The most interesting answer is "maybe." Because "maybe" means nobody knows, and that's fundamentally interesting. It's possible that it's possible to write unit tests for domains that nobody has thought of yet. (Yes, even tests that automatically infer whether your website's colors are pleasing to humans.)
Could unit testing have saved us from this one particular case? Maybe. But is it possible to write unit tests in the general case? Maybe not. But you, I, and everyone else doesn't know until we actually try. It could be impossible, or if not impossible then infeasible, or if not infeasible in a dynamic language then infeasible in C libraries. On the flipside, if you investigate, and you come up with a way of writing unit tests in the general case for security libraries, then you may have just discovered a breakthrough in how cryptographers can write security libraries. Because if it's possible to write useful unit tests in the general case for security libraries, then you will have demonstrated something that most cryptographers generally disbelieve, the same way that people disbelieve that it's possible to unit test whether colors are pleasing.
So get to work! There's nobody stopping anyone from right now going and trying to write those unit tests. Try it, write a blog post about your experience and the hardships, write about what you've learned. I guarantee it will be interesting.
All I'm saying is that we should be experienced before we advocate or dismiss an idea, and the only way to get experience is to try it out.
Cryptographic code introduces new concerns (e.g. is the code safe from timing attacks, or leaking information via cache usage,...). But it is also still code, and susceptible to common code errors. Both this bug and the Apple one are simple coding errors.
It is definitely possible to write unit tests that can help catch errors like this. I wrote some unit tests for the Apple code while refactoring it. Anyone trying to write unit tests to cover all execution paths in SSLVerifySignedServerKeyExchange would have caught that bug (since there was unreachable code). I suspect the same is the case here.
The bigger take-away for me was that the code could be improved a lot. If you have a look at the code after my refactoring, it is dramatically simpler, in my opinion.
Unit testing is not terra incognita, it's industry best practice. Personally I write at least twice as many lines of code in tests as I do in the implementation under test. That's the nature of test-driven development.
Nobody claimed that we can fix all security problems with unit testing. Unit testing should be used in addition to auditing, reviewing, and other techniques. However, unit testing can dramatically improve the quality of code, including avoiding security pitfalls and mistakes.
What we need to do is to include mandatory unit testing in all security-critical code, and ensure good coverage.
This particular bug is a bug that would have easily been caught by anyone writing rudimentary unit tests.
It's sad that there are no regression tests for these changes either, meaning we remain susceptible to such bugs in the future.
If you require comments to try to help programmers follow a confused set of return value conventions, I submit to you that comments aren't the real solution, since they're minimally helpful in addressing the real problem.
On the other hand you could say that this function returning 'false' isn't really an error, and that the idiom is similar to the isdigit(), isalpha() etc type functions in ctype.h. And fwiw, bool is a thing in C99 with stdbool.h, although this function returns an int.
So I guess the whole meme of "if the code was open to scrutiny, this bug wouldn't have happened" doesn't seem to be so valid now. There was legitimate outrage on Hacker News that Apple could allow such a bug, and that it was an indicator of the evil of closed source systems.
After Apple and now GNU, henceforth no SSL distribution should be accepted at face value without proof of continuous automated testing results. Period. This is a scandal now.
Choose OpenBSD for your Unix needs. OpenBSD -- the world's simplest and most secure Unix-like OS. Creator of the world's most used SSH implementation OpenSSH, the world's most elegant firewall PF, and the world's most elegant mail server OpenSMTPD. OpenBSD -- the cleanest kernel, the cleanest userland and the cleanest configuration syntax.
For connoisseurs of bogus conspiracy theories, it's worth noting that the coding flaw that produced the bug is much more subtle than "goto fail", but the bug is equally simplistic. Where "goto fail" required a duplicate line, GnuTLS required only a single switched token; not only that, but the bug comes from a mismatch between two common C error checking regimes (a zero-return error and a negative-return error).
I think what I like least about the bug is that it was found by the GnuTLS maintainer, long after the code shipped. That's a signal of how much attention GnuTLS gets.