I think the biggest takeaway for me from this is how he had the power to, and actually said, "no" to feature requests. This has definitely been a problem for several projects that I've worked on.
I was thrust into a similar role to this guy when I was all of 24 or 25... It took awhile to learn to say 'NO', not just because it's hard but because it often led to really strong disagreements. The guys on the sales side of the product just couldn't understand when we couldn't do something. There was always an undertone of 'are you guys just not good enough?'
I think that's the hardest part. Generally you say know even when it's technically possible. Hell you probably know exactly how to do it, just not in the timeframe you have. Learning to keep your ego (I can do this) from getting in the way of what's best for the product and the business is the hardest lesson a good developer has to learn IMHO. It's the programmers version of crossing the chasm.
Had a great read. Growing up watching gates, wozniac, jobs and other success stories in documentaries, I never wanted a 9to5 at a big establishment. You touch upon homebrew club which brings me back & close to what I still long for - the working in the garage/backyard moments. Perhaps its time. Thx!
I love the story and the writing style. He shows his passion in an honest tone; is proud of his successes and is honest about his (and the company's) mistakes. Very smart coder/hacker but also with a talent for writing (which people always think are mutually exclusive!). Props!
bill: i recall viewing a startup school 07 attendee wiki and being impressed with your self description. i think you went by "bill bland" back then. good to see quality people doing great work.
I absolutely admire the work by your team. Startup has those energy which I am hungry for , but I am stuck in enterprise world with lots ...., just watching in sheer amazement.
Great story, also looks like the guy really knows what hes doing.
That said i read a post by him a while back where he talked about how useless unit tests are and how they dont help finding bugs: http://abstractstuff.livejournal.com/60388.html
I found that quite irritating from somebody who seems so experienced.
Are there any "hacker gods" that are really big on unit testing or use TDD? Most of the recognizable names that I see talk about it are agile book writers. Not actual devs. It was Paul Bucheit who says (from that link), "I'll make a somewhat heretical claim even: Unit test are 20% useful engineering, and 80% fad. They are great for tricky code with well defined behavior (like a parser), but wasteful for most code."
Does Sweeney, Carmack, Cutler, Joy, Thompson, Knuth, Page, Steele, Zuckerburg, Anders, etc... use unit tests in a big way?
I'm actually a fan of unit tests, although I generally dislike TDD.
Nobody in Peter Seibel's "Coders At Work" seemed to have very much time for unit testing as far as I remember, and they pretty much all qualify for that title in my opinion.
I'd stand by pretty much everything I wrote in my original post, for what it's worth, but I guess I don't really understand why it seemed to ruffle so many feathers to be honest.
Unit testing doesn't help me enough for the work I have to put in (compared to things like good logging and monitoring, for example, unless I'm writing something like a parser, with well-specified behavior). It doesn't seem to have helped justin.tv too much either, when we've tried it. But if it helps someone else, then good for them!
I agree with you: if it works for someone else, good for them. Definitely.
I think the main reason why people want to burn your article to ground was because there are way more companies out there that would benefit from their people writing unit-tests.
Now when some lazy developers who think that they're smart starting to quote "the VP of Software Development from JTV sez unit-test is useless", then we have a problem there. These passionate software developers don't want to go back to the time of Wild Wild West where coders slinging code with no reasons and start throwing incomplete shitty crap to QA and call it "DONE", get the gold medals from their team lead for being able to solve things very fast.
You know that movie... Spiderman? "With Power comes great responsibility"... since you're advertising how cool your company is and you're their "VP", then there's that responsibility attached with it.
The thing is, it requires a smart person with high level of discipline, strong principal, and the ability to say "No" to crazy requirements in order to have the luxury of not writing unit-tests. Especially when there's no safety nets (I mean QA) around that person.
I think the main reason why people want to burn your article to ground was because there are way more companies out there that would benefit from their people writing unit-tests.
But he's not the only bright dev who tried it an ran into issues or lack of benefit in doing it. The thing that I seem to read from the more zealous of the TDD crowd is "you don't know what you're doing -- you don't understand code/software/testing/development if TDD doesn't obviously work for you".
My concern then is if a large percentage of strong devs don't seem to "get it", does it make sense to push into an organization w/o strong devs?
And the reason I asked the other question about the "hacker gods" is that outside of Josh Bloch (now), I don't really know of outstanding devs who do TDD in a big way. And this scares, because I remember when I used to have certain devs tell me that UML was the way to do everything, and if I didn't use UML throughout the product lifecycle I was no real software engineer -- and UML was the only way that in 20 years people would still be able to understand the design and code. I pushed back on UML and don't regret it for a second. I don't feel like I'm going to regret pushing back on TDD/BDD so far.
Maybe the confusion comes from conflating unit testing with TDD. Different but related concepts. If you write a unit test after the code, that is no longer TDD.
TDD is a major change in coding style and doesn't seem to work for a lot of people, including myself. Unit testing, on the other hand, is absolutely essential and fundamental to the task of building large, complex systems.
I haven't read Coders at Work, but if the guys in there are doing their thing without some kind of automated testing, I would be very surprised, and I'd wonder why they're making it so hard on themselves.
But on the other hand unit testing isn't the same as testing. Integration tests, regression tests, stress tests, "miscellaneous automated tests" are all valid sets of tests.
Have you used a language with strong static checking? It might change your mind about the central role of testing. I'm not saying tests are never necessary in such languages, but you can expect to spend orders of magnitude less time thinking about them.
Random guesses are a poor substitute for logical certainty.
You're confusing TDD and unit-test... and unit-test is not the only safety net around.
Take a look at Dr. Hipps projects. I believe almost all of them have a suite of test automation.
Keep in mind that there was no unit-test library for C/C++ back then. Even now, the unit-test libraries for C/C++ aren't as solid/easy-to-use as those in Java, C#, Python, Ruby.
The software back in the age of Ken Thompson, Dennis Ritchie are different than what we have these days as well.
I agree with nostrademons, some of these so-called "great developers" that you follow, they don't do maintenance programming. They seed a project and let someone else maintain it. Similar to the much-debated "software architect" role; when you don't feel the pain of maintenance programmer, you don't know what's wrong with the process.
Update: UML still exist today. For certain projects, there are values for using UML especially for communicating between people across different functions. Just like anything else: when you abused it, it starts to crumble down.
Brad Fitzpatrick talks about unit testing as a method of easing maintainability. I haven't read all of the book but of the profiles I did read, he seemed most similar to the kind of developer I aspire to be as a startup worker.
Zuckerberg and Carmarck are typical geeks that just love coding and don't care about the rest (like documentation, etc). Besides, it's their own company, they can do whatever they want and don't have to deal with the consequences; they have hordes of developers to clean up.
Knuth writes something for himself. He's probably the only coders there.
Anders has thousands of qualified MSc, PhD, and a bunch of other smart people that would write automated test for him (they call them SDET at Microsoft). (Besides, he seems to be an "architecture" kind of guy, does he still write 1k LoC everyday?)
Half of the people you mentioned came from the time before unit-tests were relatively unknown and they believe that to ship means to spend countless night and sleep below their desks. Coding, Coding, Coding is definitely the mantra. They don't know the other type of software development.
If it breaks, just fix it. No punishment.
I don't mean to paint it in such a bad way. If you look at Facebook during the earlier years, if it breaks, who cares, you're not going to lose million of dollars. The situation, culture, environment, products, and market segment are totally different.
Not surprisingly, I find myself like Knuth. I like unit tests where I feel very uncomfortable. And I usually known when designing.
At the TDD level it does seem like religion. There doesn't seem to be strong evidence that it hurts nor strong evidence that it helps. A lot of people are successful w/o it and some are successful with it.
There's a bit of an implied selection bias in your question: people get to be hacker gods usually by starting something that later becomes big, and it's in the "just starting out phase" where unit tests are least useful. You never hear about the maintenance programmer that joins five years down the line and finds unit tests indispensable, because there are lots of maintenance programmers and hence little reason to think that any one of them is a hacker god.
This should be written in almost all arguments/discussions about unit-test, especially when people start referring to famous developers who don't write unit-tests.
Speaking from personal experience, there are a lot of fads in software development. From the most recent language to the latest and greatest library. From design patterns through to work practices like Agile and TDD.
You must be critical in your thinking for each one of these. Just because it works for someone else doesn't mean it will work for you.
Personally, I think that most design patterns are used as an excuse to make a simple system complex. I think that unit tests are more painful to maintain than they are worth (in many cases). I think that new technology should be approached carefully and only proven tech with minimum lockin should be used.
Don't go round saying that "experienced developers always do X" because a real experienced developer knows that X is great for some things and a huge waste of time for others.
The dumb part about this debate is that it's being framed as all-or-nothing. The important question isn't "Are unit tests useful?" (the answer to that is "It depends..."), it's "Under which circumstances are unit tests useful?"
If a particular unit test costs you more to write than it saves you in bugs, don't write it. And if it starts costing more in maintenance than it saves in confidence, get rid of it. There's a fair bit of cargo culting going on about how much to test - in my experience, it's not worth testing getters/setters, it's not worth testing simple "if this then set these variables" logic, and it's not worth unit testing database or external calls (you should have some integration tests for them though).
It is very much worth testing tricky cases like a parser, or a bit-level decoder, or complicated algorithms. I just recently got up to 100% code coverage on a UTF-8 decoder I'd written. I found about a dozen bugs in the process. They were silly things, like leaving a digit off a hex constant, but they rippled throughout the decoder so that the results were completely wrong. If I'd just included that in another library without testing, it'd make the dependent library subtly wrong too, and I'd never hear about it until users started complaining, and they'd waste hours trying to find bugs in their code that are really bugs in mine.
I'll also mention one other thing: the point of unit testing isn't to find bugs, it's to give you enough confidence in your code that you can freely change it. The real cost of bugs isn't that one erroneous event that customers will probably never notice, it's that the fear of that one unpredictable event makes management (which is sometimes you, if it's your startup) say "It works. Don't touch it, because you'll break it" and never improve the software. That's how you get shitty software that you can't fix.
I think it's fair to say that integration tests (make sure that the functionality works) are useful; period. Not knowing whether your app is going to work when you push it is not fun.
From a maintenance perspective and assuming that a problem will show up in integration testing or via a bug report, further testing is only really useful to the extent that it lets you debug a failure faster. However that acceleration is a tradeoff against the time taken to write the tests in the first place - ergo: is it really worth it?
I personally don't think the real win from tests is in saving debugging time. I have found it's in making it quicker to write code. I know that sounds crazy as most people view tests as overhead but I view them as development tools.
The key to them being useful though is writing them before you write the code. If you write them afterwards you will have already verified the code in an ad-hoc fashion and the tests are pure overhead. If you write them before they do the verification for you.
I'm coding in Rails and I've found the outside-in approach to development: write the six line Cucumber integration test and then write the RSpec expectations for the object; act as a verified to-do list and make my development far more efficient and robust.
I'm sure that other developers with better focus than I might not get the same benefit but I find it tremendously useful to immediately list down all of the edge cases and general behaviour that a piece of code should exhibit and then keep coding till growl glows green. I trade the time that I'd otherwise spend faffing around in a browser or a debugger for writing tests instead.
It's a bonus that all of those tests then ensure ongoing integrity, the real win is that tests stop me from wandering around the trail and keep me knocking things off in the order that they need to be done. [Caveat: it has taken me months and months to come up to speed on how to write tests well] (great article btw Bill)
FWIW: With Socialcam, Eric implemented unit tests for the API (which powers the client apps), and initially for the website but later we decided against them for the site. Having them on the API definitely has been worth the initial cost as the backend has needed to be very stable for client development, and they are easily testable.
Tim, perhaps you should have made your point more vigorously when you were here :). That said, I think that many / most of our projects fall into the bus count 1, cowboy coder hacking (at least, initially), so we've been able to move fast by not having them.
He really does not come off well in those comments. Repeatedly saying I am successful, therefore what I am doing is right.
In his defense, it sounds like he has mostly worked by himself. Some practices don't have a very good ROI when you are just hacking away by yourself. I would argue that unit testing is one of them, if you are not very experienced with it anyways.
It sounds to me like he got beaten up in his own comments section. The people commenting don't seem to realize that he is talking about a different type of programming than the commenters are doing.
Great read, it's nice to see non-founders that are so enthralled with their jobs. Off-topic, though:
All of our early press had evaporated, it turned out nobody actually wanted to broadcast their lives, and we really had no idea what to do.
How did you solve this? I look into Justin.tv every few months and there's nothing compelling there for me, but it must be aimed at a different demographic.
From the guy's page, it looks like he's a super hacker - learning to program at nine, embedding a Scheme interpreter in the Linux kernel. Between this and jacquesm's recent "it's easy" post, I wonder, do you have to be this kind of crazy passionate coder to be able to accomplish this kind of thing? I feel like I could be passionate about coding certain things, but nothing I'm doing now comes close.
Don't worry about people being better at hacking than you though. There's always someone better, and in the end the only thing that really matters is your ability to get stuff done.
If you can get things done, then this counts for a lot. I know people who are geniuses and polymaths -- the kind of guys who do the calculus to optimize patterns for making their own juggling balls as a leisure activity. That doesn't mean that they are the best at getting things done though.
Be able to get things done. Be good at taking into account what you don't know. Know how to cooperate and delegate. This is the stuff that's most useful.
Another important thing isn't the complexity of the project, but the impact it has. Sometimes people tend to build on the big side when the best thing they can do is the simplest, just look at Facebook or Google.
