> you must explain to your stakeholders beforehand that these estimates are only for course correction during the development process, and they’re separate from estimates you will give of when features will be done overall.
...
> if you do put in the time to do estimates, you will spend less time coding - but because your team will be able to respond to things immediately, you will still reach your objective more quickly.
This sums up the best argument I've read about the advantages of best practices agile software development.
It's a shame some companies cargo cult agile as a way to bully engineers into being "more productive" in a brute force manner. That is, unrealistically cramming tickets into a sprint to gamify higher "velocity." Really agile is just about a more efficient development process. Agile provides a framework for discovery and course correction via planning/imagination/whiteboarding, and that is significantly faster than discovery via writing code.
Why stakeholders decide about anything related to your development practices? Is it because...
A) your hiring people like well known keywords
B) someone is beholden to a bureaucratic process
C) someone lied and broke trust?
If you cannot deliver, just say so. If there are insurmountable barriers, as well.
Why provide a bad guess to which you will be held, and this cause people to make wrong decisions based on the guess?
Estimates are only good when you have strong basis for them. Software developer cannot in general have such a basis unless they're copying or adapting in trivial ways already made software.
A civil engineer, on the other hand, can use a reference group strategy easily to compute cost and materials required, but not necessarily timescale. They can provide at best a reasonable, very wide range.
Stakeholders will use your work product as part of some larger process. They need estimates so they can plan. They can't just turn on a dime when you say your finished.
If you break someone's plans it is often worse than not having planned at all. Unless the plan is extremely flexible... Agile. Then the plan barely matters so why make one at all? Have a detailed unordered playbook instead and potentially a guiding strategy. (sometimes a playbook is timed, that is less flexible than untimed conditional plays and generally shows that you lack information)
In case the words are confusing, plan is on tactical level and related to day to day operations. A playbook is a set of small roughly sketched plays to be used when conditions are met. A play differs from a plan by level of detail and size.
"Release by the end of June" is rarely a good plan, "release before competitor adds a similar feature" might be.
The date is reifying unknown to look known but does not make it so, and if leaked helps your competition plan.
Deployment can be done whenever and unlike development is often much more predictable.
Likewise marketing.
Usually the story is that the stakeholder is targeting some annual demo event and that gives a strict timeline. Any of your estimates is worthless and useless then, only tracking and feature set near the time of such event so that marketing can decide what to show and whether to show anything at all.
And the marketing takes some time too, so the stakeholder probably wants to know how ready something is before launching that machine, but not too much before or they might overhype and burn.
Trying to do it backwards, when the features are time boxed target generally goes poorly as the features get implemented in a checklist but not useful, usable or extensible matter. (Scrum results in this too.)
Second type of "stakeholder" is a time minded investor. Again that is a commitment not an estimate, since it you do not provide a good enough product you will not get paid and probably fold.
Then there's a combination of both when trying to outdo established competition with known upgrade patterns.
Yes, exactly! It's such a shame that people think "more efficient process" means "we'll ship more lines of code" when really it is all about getting to the _correct solution_ more quickly.
Presuming you know what the correct solution is. This is the key point of agile processes, that you get to quickly discover what is required, optional and completely unnecessary. Generally by having a real customer (not a silly proxy) in the room or by sending out probe releases often and collecting feedback.
> If you never use the estimates to adjust what you are working on, then putting in the time to do estimates is a pure waste
yes! using plans to create context for the larger project is the critical missing piece.
Business stakeholders just want delivery dates but the team doing the work benefits when the estimates fit into an organic, flexible master plan that is changed when things are late or hard.
The book The Influential Mind does a chapter on (essentially) tips & tricks for estimates (of all kinds). Long to short, be careful of group think. It's best to collect the estimates independently, then aggregate. Else the group members influence each other and the advantage of "crowd sourcing" goes out the window.
The book is worth a go. It reads quick. A bit light- weight but still useful.
The biggest value of bad estimates imo are that they teach you to get better at estimating. You should record your estimates and the actual time it took. That way the next time you need to make an estimate you have some more data to plug into your formula for making a guess: the previous slippage. So you improve over time and your post-mortem of why something slipped will help you to get your initial guesses closer to where they should have been in the first place.
Between those two anybody can learn how to estimate with some reasonable degree of fidelity.
Just a few days ago, one of the restaurants I own was getting a new air conditioner installed as a condition of lease renewal. Of course, the landlord did not tell me the AC people were coming to do the work and just showed up at 7AM with the expected high temperature to be in the mid-90sF. It was already 78F when they started.
I asked my manager to ask the AC guy how long it would take. "He doesn't know", she said. Such an answer drives me insane. You mean he has no clue? Will it take three months?! 30 minutes?! How can we plan to work around no AC in a restaurant when the installers have no idea how long it will take?
But of course they had a rough estimate. Why didn't they give me one? It would relieve a lot of anxiety if nothing else even if it was wrong.
At the three hour mark, I was told it would be about an hour. After 5 1/2 hours, I was told half an hour. It took 6 1/2 hours to complete.
Here are the issues with all that:
1) If I had an estimate, I wouldn't have needed to drive to the restaurant and talk to the installers myself.
2) If I didn't get an estimate, I would have had to raise hell with the landlord who I have a good relationship with.
3) Once I had an estimate, I didn't feel the anxiety or need to constantly check on the progress which would have irritated the installers. And believe me, I would have irritated them!
Unfortunately, a lot of people treat "estimate" as "a binding commitment written in blood" - so I sometimes understand when people are reluctant to give such a thing.
So much this. Whenever I'm trying to pull an estimate out of someone that is reluctant I always try to reassure them that I know its an estimate, I know it might be wrong, I know conditions can change but its still helpful to have a guess.
There are just too many unknowns to do estimates well in software. And getting things wrong in estimates isn't a matter of being off by a couple percentage points but being off in by orders of magnitude.
It isn't unusual for what appears to be a simple unit of work that seems like it should take a few hours to explode into days or weeks of work. And it does so in a way that is completely opaque to stakeholders.
Partly this is because of how we build software as layers of abstraction on top of other layers of abstraction. When you are at the top layer things are relatively quick and easy to estimate, but once you have to work at a lower level than normal all bets are off. And it is really hard to know beforehand if you will need to dig into the lower levels.
For example at my job I am updating an app that queries an api to filter its requests by some parameter. When the api supports the desired filter that is great. When the api doesn't, I have to dig into unfamiliar code for the api to support it. But turns out the api is just forwarding its request to another api, so I have to dig into yet another api to build the filter. And so on. The rabbit hole can get deep. And of course there isn't any documentation to help me know about this beforehand.
The article is right that the key is frequent updates and communication as soon as "blockers" like this are discovered. But it can be very difficult to communicate this to stakeholders who are uninterested in the technical details and are inflexible in scope. Trust is key, so that they believe you when you say that this task which normally takes an hour or so will now take several days.
A team cannot reduce scope without involving the customer. And if you have the misfortune that you're dealing with someone who thinks difficulty is negotiable, you're in a real pickle.
You're missing the critical point - measure the estimation error and you'll see how hopeless almost all estimates are. In practice, they do not improve.
Akin's 10th law of spacecraft design: "When in doubt, estimate. In an emergency, guess. But be sure to go back and clean up the mess when the real numbers come along."
One caveat: the relative level of definition of work has a huge effect on the work. For example, well trodden engineering implementations can have rational top and bottom end estimates. A UX design or Eng implementation of a "never been done before" or strategically-shifting problem is far far harder to estimate.
This was one of the very few valuable things I learned working towards a bachelors degree.
One of my professors popped a quiz on the class with the question “how many piano tuners live in the state of New York”. After collecting the answers, he spent the rest of the session lecturing (especially those like me who left it blank) on how to make estimates without having any firsthand knowledge in the problem space, and how valuable that skill could be.
...
> if you do put in the time to do estimates, you will spend less time coding - but because your team will be able to respond to things immediately, you will still reach your objective more quickly.
This sums up the best argument I've read about the advantages of best practices agile software development.
It's a shame some companies cargo cult agile as a way to bully engineers into being "more productive" in a brute force manner. That is, unrealistically cramming tickets into a sprint to gamify higher "velocity." Really agile is just about a more efficient development process. Agile provides a framework for discovery and course correction via planning/imagination/whiteboarding, and that is significantly faster than discovery via writing code.