Hacker News new | past | comments | ask | show | jobs | submit login
How GitHub Works: Be Asynchronous (zachholman.com)
299 points by jroes on Aug 17, 2011 | hide | past | favorite | 60 comments



...meetings pull you from doing actual work in order to talk about doing work. It’s easier to push a branch up, check out the diff, and then iterate on that diff rather than assuming you’re going to perfectly whiteboard system design ahead of time.

This seems over-broad to me, and seems to violate the accepted wisdom "Good programmers spend 10% of their time coding, and 90% of their time thinking." A good system/design meeting can go a long way to producing better code.


The problem with the whiteboard design is that you have to defer, "oh, let's see if that works" indefinitely. While you spend 90% of your time thinking and 10% typing, you don't spend 9 hours thinking and then 1 hour typing. You spend 5 seconds typing, then 30 seconds thinking, then a minute or so typing, and so on. Getting feedback from the computer is important for anything that's complicated enough to require a design. Seeing what the code looks like that your design enables is as important as a conceptually-sound design.

The main problem is that 99% of programmers don't know how to prototype. They assume that whatever they commit is what goes into production. No. Try an idea quickly and throw it away if it's bad. That's why we have languages like Perl, Python, and Ruby. While it's 50/50 on using them for production, they are absolutely the right tool for testing your ideas quickly.

And if you can test ideas quickly, you don't need meetings. Spend an hour coding what you would have talked about in the meeting. Share with coworkers. Get feedback. Tweak the prototype. Then write the "ready for production" version.

I've been to a lot of meetings but I've never seen any piece of code look anything like what was described in the meetings. As soon as you hit that one point where the computer's view of things and your meeting's view of things diverges, everything else you discussed in the meeting is invalidated. The computer is always right.

(Actually, one of the answers I got to an interview question at Google was so cool that I had to code it. And the code looked exactly like the design I sketched on the whiteboard. And it ran really fast on a large dataset. So maybe meetings are useful; but only for algorithm design, not for "real-world" stuff like "how should we refactor this thing".)


The main problem is that 99% of programmers don't know how to prototype. They assume that whatever they commit is what goes into production. No. Try an idea quickly and throw it away if it's bad. That's why we have languages like Perl, Python, and Ruby. While it's 50/50 on using them for production, they are absolutely the right tool for testing your ideas quickly.

That's one possibility. However, I can't help but find the idea that you have some kind of arcane knowledge that 99% of all programmers don't have is a bit... well, egotistical. Prototyping certainly has its place, but its purpose is completely orthogonal to that of a meeting. Meetings serve to plan. They get everyone on the same page. Prototyping serves to test ideas out.

The problem is that meetings are almost always ruined by a small minority of people who insist on dragging them out. However, a good meeting is very helpful.


My experience is that 99% of the time in meeting is spent on some issue that the computer could tell you very quickly. "This is O(n^2), but it's faster than the O(n log n) algorithm on modern hardware." Rather than debate, benchmark and explain the results to the group. Speculating gets us nowhere.


> The main problem is that 99% of programmers don't know how to prototype. They assume that whatever they commit is what goes into production.

My main problem is other people having no concept of there being any difference between a proof of concept (or prototype) and production code...


That's why we have languages like Perl, Python, and Ruby.

More relevantly, it's why branching is such a lightweight operation in Git.


> Share with coworkers

wouldn't this part be easier to do in a meeting than in a huge email thread?


The article talks about using Campfire (http://campfirenow.com/), which is a lot nicer than a huge email thread. From the campfire page:

Team collaboration with real time chat.

Campfire is like instant messaging, but designed exclusively for groups.

Share text, files, and code in real time.

I don't think anyone is advocating huge email threads here!


This is a point I've been making for a long time:

"What's a long email? 1000 words? 5000 words? 10,000? Let's call it 10,000, just to be over the top. I can scan read at 800 wpm, and finely read at 250 - 500. So absolute worst case scenario I'll have to spend 40 mins reading an extraordinarily long email. That is still better than an hour long meeting where I'll listen at 40 to 80 wpm and have completely no recollection of it two weeks from then. So why the fuck do we consider ten thousand word emails completely insane but we even entertain the idea of an hour long meeting?"

Obviously something like campfire or flow or whatever is better than emails, but I'll take long emails over meetings any day of the week.


You have a point. But you are assuming that meetings are about exchanging textual information. Usually there's more than that.

It's often important to get an idea about how stakeholders feel about different ideas. You can see facial expressions of everyone in a room and hear the way people speak at the same time as you're processing what's being said. All this information will not be in your chat/email thread at all. Also, not everyone is a great writer, many people will communicate much less details in writing than in person. Part of the reason why management likes meetings more than developers is because non-verbal data tends to be more important for their jobs.

Plus, what about visuals? Drawing on a whiteboard is easier than drawing in any software package I know of (even if you've got stuff like Wacom tablets). It's also easier and much faster to point to something than to explain it verbally.

Overall, I think the benefits of asynchronous workflow far outweigh the drawbacks (btw, anyone got a list of companies that use it? :), but it's important to understand the drawbacks as well.


You definitely have a point with the visuals.

As for how people feel about certain things, I've always preferred informal lunches over meetings.


Ideally, the hour-long meeting is a collection of many small communications from several different people. If meetings consist of getting talked at by one person for an hour, I think most would agree that something is wrong.


A good system/design meeting can go a long way to producing better code.

The problem is that these are so rare. The vast majority of meetings in every company I've worked at are of the "invite everyone so it turns into a contest to see who can make their point the loudest" variety, and almost always end up with a few people who view it as their chance to be the "star" to a captive audience and thus draw out the meeting as long as they possibly can.

Plus, if your programmers are any good, they're probably holding these meetings already without management having to formally call a meeting. I think that's the real problem: "formal" meetings almost always suck. Ad hoc meetings held by people who need communication to finish their job usually make up for themselves in increased productivity.


Agreed. I should nave been more specific with what I meant by a "good" meeting.


Weeks of coding can save you hours of planning!


Sure, if you don't know how to prototype.

Think about it. Did Linus Torvalds sit down at a big meeting and design Linux on a whiteboard? Did the Mozilla team design Firefox at their daily stand-up meeting? Did Emacs evolve from detailed discussions and specs, or did someone write some macros for their favorite text editor and it evolved over time?

Great software isn't planned. Great software happens. Bitching about minor details in multi-person meetings is just a waste of time.

If you want to waste time, get a beer and read HN for a while.


Gall's Law:

"A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system."

http://en.wikipedia.org/wiki/Gall%27s_law


Thinking != Meeting..


When your project is innovative and solves a problem in a difficult domain that you, hacker, are not familiar with (say, healthcare), meeting with domain experts is invaluable because it enables a bidirectional exchange of ideas.

Without these meetings, you find yourself so lost you can't even start prototyping. Good luck with that.

This kind of project would be classified as Inventions as opposed to Implementations in Zed Shaw's C2I2 Hypothesis, which says that you need Collaborators for Inventions, and Clients for Implementations, see here : http://zedshaw.com/essays/c2i2_hypothesis.html).

I think the only reason GitHub does not need such meetings is that they are developers building solutions for developers. So think again before you start "despising" meetings.


I really wish the author didn't use such an inciteful title for his section on meetings. The title offers another catch phrase to avoid meetings. I'd much rather have seen Zach focus more on his actual point: holding short, focused, and tiny meetings. I fear that his title, and other catch phrases like it, encourage siloing. Siloing is far more "fucking toxic".


Where did you pick up the fact that his actual point was to hold tiny focused meetings? Working asynchronously is pretty anti-meeting, and frankly it's totally independent of a culture that supports silos. I work in a very asynchronous culture right now. Very few meetings.. but that's not to say we don't communicate. We communicate very thoroughly.

However short and focused they are, my definition of a meeting is a schedule block of (any size of) time where a number of people are expected to party on something and decide on some action.

To extend Zach's metaphor, meetings are a communication mutex. They force a topic to be wrestled by a group of people at the lowest-common-denominator of pace. I think they're rarely the best or most efficient way to collaborate.


The trouble with working asynchronously is that you're relying on everyone to contribute their knowledge diligently. Github has a great culture and this works well for them. In places where it doesn't, brief standup-like meetings help get everyone on the same page at least once a day - a chance to say 'sup.

The goal isn't to decide on anything, just to let everyone know where you're at.


I'm reading a bit into his words, yes. Here's the part of interest:

"""You don’t have this problem with chat transcripts. Forcing people to reduce their otherwise rambling thoughts into concrete sentences helps focus discussion, too.

We’ll have meetings at GitHub, but I can count the number of full “meetings” we’ve had in the last year and a half on one hand."""

The former bit implying that the focus of sentences helps, the latter bit implying to keep the membership of the meeting to only those required.

Again: siloing is far more toxic than meetings in my experience. It's far too easy for a dev to wander down a path that has no value when operating in a silo. Agile tells us that we need to exist in a state of constant communication and feedback. This necessitates meeting with people in some fashion.


Apologies, but my brain turns off when the word 'Agile' is used in a sentence in such a way that it could be comfortably replaced by the name of a deity.

Anyways. Maybe semantics.. My points were just that meetings are basically awful, and that there's a distinction between communication and meetings... Meetings aren't the only form of communication.


They haven't really eliminated meetings, they have changed the way meetings are conducted. Thanks to the technology, you can step out of a meeting at any time to have lunch or, you know, work without missing out on anything. As a result, meetings never need to end.

I have worked under a similar model for almost a decade now. It comes with so many benefits over a traditional meeting that I am uncertain why any software development company would do it any other way.


they are on campfire, they are constantly in communication

I echoed his sentiments in a post I wrote about working remotely - http://arandomurl.com/2011/09/03/working-remotely.html


> I'd much rather have seen Zach focus more on his actual point: holding short, focused, and tiny meetings.

That is nowhere in sight in the post. In fact, the content of the section is even more damning than the title:

> I tend to loathe meetings even more than 37signals. I despise them.

> meetings pull you from doing actual work in order to talk about doing work.

> meetings are utterly forgettable. Even if you take meeting notes, you can’t capture them all.

Zach does not say he likes "short, focused and tiny meetings", Zach says he hates meetings as much as a human being can hate a concept.


Well, then you object to more than just the section head, right? Because he didn't suggest anywhere that he approved of short, focused, tiny meetings. As far as I can tell, he meant precisely what he said in the title.


Most engineers focus in meetings about "what I got". The reality is meetings should be about "what the organization gets". Sometimes an engineer may come away with nothing but the organization profited significantly.

That said, too many meetings are intensely toxic for any company, and nothing is worse than working at a company that thinks meetings are "getting things done".


This seems a little far from the truth. What if someone needs to fix an urgent live issue? You can't always work async. Sometimes things need to get done, now. E.g. issues on a production system that are breaking the site for everyone.

Sadly, people often think that everything needs to be done now and I think forcing less meetings on people helps that happen.


We'll still work in Campfire though. We have an infrastructure room where we get all of our notices and systems-level logging piped to, and all of our on-call sysops can get together and deal with stuff immediately when it's necessary.

But it's still async... from the perspective of others. I'm not typically deep into the systems-level maintainability on GitHub, but I can check the transcript during or after the downtime happened and can figure out what happened, how the discussion around it went, and how it was dealt with.


At my job, chat (via IM client) is urgent and email is (a little) less urgent. This makes a lot of sense to me. If I want to not bother someone, I'll send them an email. If I want to bother someone, I'll IM them.

Is there something special about Campfire that makes it non-urgent?


What I like about Campfire is looking back through the transcript is easy - even for the times when you have left the room. Plus it has the ability to star parts of the conversation so you can see the important points as you scroll through.

So you can use it as a synchronous "have a chat now" tool - or as an asynchronous "leave a message and I'll deal with it when I return" tool at the same time.


Both IM and email is private. Chatrooms are public.


Do all 35 people share one campfire room?


We have a bunch of topic- or project-specific rooms, one room for animated gifs and pictures of cats, and one room for serious business stuff.

Pretty much everybody hangs out in the cat picture room and serious room. The rest are just people interested in that topic.


Totally agree that something have to be sync. However, from my experience Campfire is the single best place to "meet" and address an urgent issue.

The ability to have two or three conversations going on simultaneously, share links and log messages, and have a transcript/timeline to use to guide a postmortem is fantastic.


"Meetings are toxic."

One of the strangest things about working at Apple is they have meeting ALL THE FUCKING TIME. You spend more time in meetings than doing anything else.

And now they're the highest valued company in the world, and 37signals is ahem not.


I'm tired of the "meetings are toxic" claim. If you're going to use such bombastic and broad language you need to back it up with a broad set of examples. Sure, business culture has gone overboard with the meetings, but let's not throw the baby out with the bathwater.

Meetings I love:

- an expert pulls me and some others into a room to update us on some secrets where we can ask questions

- lunch

- a very busy production lead needs to "use" (not waste) everyone's time so that she can efficiently figure out where a team is at

Yes, all of this can be abused, but lots of well run meetings leave me with a sense of purpose and camaraderie. Rah rah.


- an expert pulls me and some others into a room to update us on some secrets where we can ask questions

Sounds like gossip.

- lunch

Nice. Nothing like food time to discuss business.

- a very busy production lead needs to "use" (not waste) everyone's time so that she can efficiently figure out where a team is at

Email or a status board are much better for this.


Useful terminology (working asynchronously). There was a discussion recently about working remotely and I struggled to convey the difference between a remote-employee and a remote-team (http://news.ycombinator.com/item?id=2847012)... This is really the core of it. Teams that operate like this have a much better ability to adopt remote workers, for one thing.. and I suspect a much higher index of developer happiness, for another.


> I tend to loathe meetings even more than 37signals.

Zach, can you go into more detail? From the few things I’ve seen, you seem to admire them more than despise them. Is there a particular reason for your displeasure?


It should read as

> I tend to loathe meetings even more than 37signals loathes meetings.

(assuming you are not being facetious)


Doh. I honestly misread that line. Thank you.


Just a heads up we do async chat all the time here at my office, but we use Skype instead of Campfire. I don't want anyone to feel like they need to pay a monthly fee to get that service. Skype does persistent chat, you can add people to rooms and they remain persistent as well. You can 'favorite' rooms to keep them sort of pinned in one spot in your list of active convos. It's handy.

http://cl.ly/2d3d1M072M1H1h3P1A0i


We use Skype too.. and it's fine. By fine I mean, it sucks.. but it's a chatroom so we have very few needs.

Wanted to correct something. Skype's chat is NOT persistent (if I'm wrong, lemme know.. I'd love to find out there's a feature I'm missing). That is to say, the chat persists.. but only on the distributed hosts that participate in it. So if I sign off at 10pm, and wake up before any of my team members are signed on at 7am.. I won't see anything that happened in between.. until one of them signs on.

This feature alone is probably worth the price of admission to Campfire, IMHO...


Yeah that's seems correct. You won't receive a message until you're both online. We use it here at work for interstate chat and it works well as we have our work machines on 24/7. So logging in from a laptop off site will sync the messages. Note that the read status doesn't get synced.


Skype is really terrible imho. If you're looking to move away from skype because it sucks, you can try an XMPP server + Candy, or an IRC server plus CGI:IRC or qwebirc.

Both solutions take about 15-20 minutes to set up and cost nothing, provided you have a VPS to host the server on.


I'm tempted to move to IRC...does it persist messages? I assume so, based on the discussion above. I'm quite sure that when I've used IRC in the past I haven't been able to see what has occured after I've left a channel but I imagine if I run my own server there is a simple solution to this problem. Anyone care to share (though I'm Googling as we speak)?


It does not! But the usual way people get around this is connecting to IRC using an IRC bouncer or irssi on a server somewhere and then connecting to that. This way you are "online" the whole time. But it's a thing everyone has to set up individually.

The other way it's handled is by having a bot sitting in channels (doing things like pushing warnings and error notices from your server, too) that records logs. Then you can visit a webpage that the bot hosts with the transcripts on. I'm sure there are packages that perform the logging directly on the server and host them, but it's useful to have some unlogged rooms, too.


I agree that most things called meetings today are toxic and unproductive, but gatherings in the work place that occur organically and ones that resemble group work sessions are healthy. For anyone who wants more detail on what Zach mentions regarding "the zone", breaking flow and job autonomy lookup the book Drive by Dan H. Pink. Great Post thanks!


What do you think about some authors (i.e. The Clean Coder) that claims that you should try to avoid entering "The Zone"?


For a number of years I had a team in five offices in four timezones, so asynchronous working became the norm. We used Skype however. The only "rule" that needed to be enforced was "Away means Away, Available means Available"

A lot of realtime problems got stymied by people thinking others were ignoring them.


The "away" rule seems counter productive. The value of async is that I can ask you a question when convenient for me, and you can answer when convenient for you. If I have to wait for your status to become available, I'm wasting my attention.


I wonder if by 'available' he means 'will answer immediately'. My group uses Skype like campfire also.. and Away means 'I won't see this until I get back'.. Available means "I'll be notified immediately, and am likely to respond relatively soon". I still leave messages for people who are away all the time.. I just know that it's unlikely we'll end up in a real-time conversation at the moment, and as such.. I can just fire-and-forget until they return and reply


Basically it means we should trust the "away" message, but still can drop a line if you want. Away could also mean "mentally busy" or "on the phone"

If I could trust the away messages, as a team lead I always had a status board as to where my team was (since we were in five offices)


This seemed implicit in the article to me, but the way they use chat seems to be the way IRC chat is used for devs (though I haven't used IRC that much).

If my understanding's accurate, then I'd agree with you: ignore someone's status, ask a question, don't expect an immediate answer.


This appeals to me immensely because I absorb information way faster by reading than when someone talks at me. I wonder what percentage of the population is like me as opposed to those who absorb better by listening?


"I tend to loathe meetings even more than 37signals."

Not trying to be the grammar police, but after reading this sentence, I chuckled because I thought the author was expressing his hatred for 37signals :)


With a private repo in GitHub, after adding a second user, can that second user just start using "Pull Requests", even if they just cloned the repo initially at the command line?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: