This is an honest question: Having been in IT and software development for over a decade, I have never encountered a problem that required me to look at the source code. As such, I've never really understood the open source movement. I can't imagine a scenario where the software I am using, the same software used by hundreds of thousands of others, would be broken to the point where I had to go into the source and fix it. I can't even imagine going into the source to figure out how it works-- usually the documentation and online help is sufficient.
What is the typical scenario where one really benefits in open over closed source? How often does it occur? I am curious.
What you say is partially true for 'mainstream' type of things.
Open Source is particularly useful for the long tail. That particular library that is very relevant to you and maybe 20-50 other people in the world. It makes sense to join forces and make it better - but it will never be absolutely mainstream.
Secondly, I say your point 'partially true for mainstream things' because even in mainstream things, there will always be customizations/particularities that again are useful to you and 30-50 other people in the world - thus you join forces (e.g., a obscure linux module).
Agree with you that 99.9% of the people will not
It's all a matter of long tails. Without open source, all long tail efforts would be each independent person doing it for themselves (when you have 30-50 people in the world that should be sharing!)
Although the modern web is making it viable for small teams w/little capital to go after long-tail in a commercial way (many YC startups are a testament to this) --- although I don't imagine them doing this for a 30-50 people target market.
I think you either are lucky or not working on some low level problems. I had at least two cases (in last 5 years) where I had to debug code and enter Windows libraries reading assembler code using cdb/windbg. It was painful knowing that in Linux I could simply read code, understand it and potentially fix it. In some cases low level tools help avoid debugging assembler (like dependency viewer, process monitor, spyxx, wireshark and etc.).
It is really not about fixing something but understanding what behavior underlying libraries expect from software you are writing. The situations where documentation is technically correct but practically useless.
I started working with Microsoft platforms professionally at age 15 or so. I worked for Microsoft as a software developer doing integration work on Visual Studio. More than ten years after I first wrote a line of Visual Basic, I wish I could never link against a closed library ever again.
Using software is different than building software. When you're using most software for it's primary function, it's a well worn path. Others have encountered the problems and enough people have spoken up to prompt the core contributors to correct the issue. But when you're building software, you're doing something new. And there are so many ways to do it, you'll encounter unused bits, rusty corners, and unfinished experimental code paths. You'll encounter edge cases that have been known to be broken, but were worked around.
Sometimes, the documentation isn't complete. Sometimes, it's wrong. The source code never lies. For an experienced developer, reading the source can often be faster... especially if you're already familiar with the package's architecture.
I'm in a medium-sized co-working space with several startups. A lot of the other CTOs and engineers come to our team for guidance and advice on occasion. When people report a problem with their stack, the first question I ask them is: "Well, did you read the source code?"
I encourage developers to git clone anything and everything they depend on. Initially, they are all afraid. "That project is too big, I'll never find it!" or "I'm not smart enough to understand it" or "That code is so ugly! I can't stand to look at it". But you don't have to search the whole thing, you just need to follow the trail. And if you can't understand the platform below you, how can you understand your own software? And most of the time, what inexperienced developers consider beautiful is superficial, and what they consider ugly, is battle-hardened production-ready code from master hackers.
Now, a year or two later, I've had a couple of developers come up to me and thank me for forcing them to sink or swim in other people's code bases. They are better at their craft and they wonder how they ever got anything done without the source code in the past.
When you run a business, if your software has a bug, your customers don't care if it is your fault or Linus' or some random Rails developer's. They care that your software is bugged. Everyone's software becomes my software because all of their bugs are my bugs. When something goes wrong, you need to seek out what is broken, and you need to fix it. You fix it at the right spot in the stack to minimize risks, maintenance costs, and turnaround time. Sometimes, a quick workaround is best. Other times, you'll need to recompile your compiler. Often, you can ask someone else to fix it upstream, but just as often, you'll need to fix it yourself.
Closed-software shops have two choices: beg for generosity, or work around it.
Open source shops with weaker developers tend to act the same as closed-software shops.
Older shops tend to slowly build the muscles required to maintain their own forks and patches and whatnot.
True hackers have come to terms with a simple fact: If it runs on my machine, it's my software. I'm responsible for it. I must understand it. Building from source is the rule and not an exception. I must control my environment and I must control my dependencies.
This is part of the reason why I really appreciate m2clipse (Maven Eclipse IDE) ability to download the source code (when available in the repository, which, in Java land, almost always) when I tried to navigate to the Class definition, method implementation, interface or whatnot.
It's very empowering to read the source code because one can learn a lot.
A few more things to add from your points:
To work-around a software, sometime I have to understand how it works internally and given the source code, I don't have to spend a lot of time to reverse-engineer it.
Having said that, sometime the customers do care if it's my fault or IBM faults if we stated clearly in the deliverable that "hey, we're using IBM WebSphere for this...". They would give IBM some time to fix it, or ask us if we can work-around it, or they would bend certain requirements (i.e.: work-around the requirements, not the code).
This is why sometimes, probably a dying trend given the quality of open source software, an outsource project chose software based on the company behind it.
Lots of good comments above, but here's a concrete example.
It can be a lot faster and accurate searching the source code for an answer than googling it. My most recent example: the .NET WebClient throws an exception ("ProtocolError") if it receives an HTTP 304 ("Not Modified") response. This makes no sense, but is it a bug? Did I use the client correctly? Or is it by design - i.e. a vexing exception (http://blogs.msdn.com/b/ericlippert/archive/2008/09/10/vexin...).
If you google for the answer you get a bunch of other confused people and a bunch of different advice, none of which is appealing. If you look at the source code you can quickly answer the question: yes, they really did design things in this brain dead way, and now my caching layer - which is supposed to be fast - is going to have to handle an exception every single cache hit. (When you're issuing over a million requests per day and you're paying for CPU time by the hour this design decision actually has a cost since exception handling is slow.)
I am not a great open source contributor, but I always come across bugs, performance issues, feature improvements etc. which I contribute to the open source projects I am using. In closed source all I can do is raise a bug request / feature request and wait till the team decides to fix it or never. All the features / scenarios / corner cases may not be encountered / thought of by the main development team and it is great that a real user can contribute to the project to make it better. With Github, feature improvements / bug fixes are just a fork away.
Granted, this is not going to be applicable in the same way to something like the ASP.NET etc. but open source works, for me.
I've had about four situations in my current project (Java/GWT) where going through the source has helped me understand what the developers expect, or have TODO comments for unimplemented bits.
In a previous project (jQuery/Python) I had an issue with a Python library that I needed to patch. It worked fine on my collaborator's Mac but not on my machine.
Even .net - I created a downloadable product that happened to require a fair amount of custom UI work and being able to look under the hood (with Reflector) helped a lot. There are still a couple issues that I could never get resolved that are buried in the stack.
Happened to me today. I'm working on an older version of Ruby on Rails (3.1.3) and hit a bug that the documentation said should have worked. Turns out that a bug had been introduced into the codebase and wasn't caught by any tests. It was fixed in a later version of Rails (3.2) but I was only able to ascertain the above because Rails is open source and I could work my way through the source code to trace the bug.
The scenario you describe likely happens more often than you think. Sometimes (often) documentation gets out of sync with code.
My experience reflects yours. However of late I have had to crack open the source code for ASP.NET MVC & the Azure SDK. Not so much to correct a bug but to ensure functions that I am overriding follow a similar design pattern to the original.
I've never had the need to get into ASP.NET or the .NET framework itself but being able to get into components built on top of these lower level technologies has been very helpful.
I've found things like low-level Bluetooth stuff to be appallingly poorly documented, and I would love to be able to explore some Windows source code to see how they're calling their own, convoluted APIs. Pretty much any time I've been calling out into other people's libraries, there have been times that I wished I could step through their source and/or fix their bugs.
After working in a Linux/Open-source environment for several years and then switching to Microsoft a few years ago, I noticed that things seemed to be getting progressively more open and developer-friendly. I've wondered if it was just me being biased toward my current environment, but there has definitely been progress.
I think that Microsoft has finally realized that the best way to build and maintain a community of software professionals is to treat them like esteemed associates. It's too bad Haack left, but with Scott Hanselman evangelizing (he even came to my mid-sized midwest city a few weeks ago) and ScottGu running the show, things are looking bright for MS devs.
I started my career in MS ASP.NET WebForms/MVC, and then recently switched to RoR, and it's obvious that they're mimicking the success of RoR development environment and, more importantly, the culture. Their Nuget packaging system is pretty much RubyGems + Bundler. Last year was the last time I touched MS stuff, and it was pretty raw and not ready for the masses yet. With people like Hanselman and ScottGu pushing the MS culture towards more open, I'm hopeful for the direction of MS's Web stack.
To give some context, my first job out of college was in a small WebForms/MVC shop in LA. It was very progressive in that my managers always pushed me to question the conventional "MS" way of programming/architecting. However, what frustrated the hell out of me was all the other MS companies around us: they were so damn dependent on what MS gave them.
I've heard countless MS devs in local meetups screaming bloody murder when MVC first came out, "Microsoft, you HAVE to train us on MVC! It's a black box, we're effed!" This shocked me, because these people wanted to literally be force-fed with MVC. Microsoft (intentionally or not) encouraged this kind of behavior with their close-mindedness, and this dependency/babying was crippling to the MS dev community.
Accepting pull requests signifies a tremendous step on MS's side, because they're finally admitting that there are actually smart developers OUTSIDE OF MS BUILDING who can help out with the framework. Their closed-software drove me away from them to RoR. Now, I have some peace of mind knowing that the bleeding of good devs away from MS stack may finally slow down. I can't wait to see the next generations of Scott Hanselman and ScottGu pushing the envelope for MS.
Well, that's a first! I'm really happy to see this change. MS has some excellent dev tools out there, and really the only thing holding back .NET development is the strong "if it ain't MS, it ain't good" culture among many of its users. Moves like this might help break that strong singular dependency.
Wow. I thought I saw a pig just fly past my office! I never thought this would happen - but then neither did Mr Haack.. Kudos to Microsoft for coming to join the party.
Hopefully with a bit of luck I'll never be using or coding on MS stack ever again though!
What is the typical scenario where one really benefits in open over closed source? How often does it occur? I am curious.