> During planning, everyone understood that the estimates were _estimates_. Tasks could take longer or shorter, that was OK and even expected. The goal was to improve velocity, not to accomplish every story, every time.

> Engineers felt comfortable taking stories in areas they knew nothing about. A stated goal was that every engineer understand every piece of the project.

> Standups were team only. Engineers felt comfortable saying they got stuck or needed help or got lost in a rathole and didn't make the progress they hoped.

> Demos were important so that the team and the clients could understand what had changed and what new features were available.

But it quickly got lost - managers and PMs and Directors got involved and Jira boards became published for all to see. (In the beginning it was whiteboards and sticky note). Standups and Demos were all about self-promotion. No one ever wanted to take on a task they weren't 100% certain they could do. If a task was 3 point it needed to be done in 3 days or there would be questions.

When I left the last company it had gotten outrageous. Every task should be completed in three days and in production in five days - and if not that team was Doing Something Wrong.

( We were told that this was standard FAANG practice, I have no idea how true that was )

What happened was what you'd expect - shortcuts, tech debt, unit tests cynically design to pass and meet code coverage expectations instead of actually usefully testing.

The reality is that any process is going to fail one senior leadership decides it's a way to evaluate engineers, not create a good product.

I've come to believe that that main driver for Agile adoption has come to be something similar. Making visible to outsiders what software development teams are doing, and making progress (or it's lack) visible to management. I thinks that's a completely reasonable expectation. Businesses are paying exorbitant salaries and providing ping pong tables, why shouldn't they have visibility into what's being done?

Where it becomes toxic is in areas this posts parent indicates; posturing, one upsmanship, pressure to perform. Effective teams need "safe spaces" to learn, discover, try and fail. Agile isn't that anymore. Hasn't been for a long time.

Businesses are paying market rate salaries to employees who generate them revenue. If anything engineers should be the ones questioning the exorbitant compensation and perks at the top of the org structure.

> why shouldn't they have visibility into what's being done?

Because they shouldn't be micromanaging. They should be setting high level goals and expectations and giving teams autonomy and trust. Intrusive surveillance and low level metrics create perverse incentives that detract from those high level goals.

You use this argument to defend engineers but then question c—level executive salaries. The exact same principles apply but at a multiplicative level.

That’s simply not true, workers and managers are qualitatively different. There are no shortage of examples of managers of companies losing money and marketshare who nevertheless are paid bonuses every year then leave with a golden parachute.

Why? That multiplier is often what's at issue, and why "the exact same principles" don't apply.

There's an earlier version attributed to Mohammed - "Trust in God. But tie your camel first." - so the sentiment has been around for a while.

It feels like it's just a way of reducing cognitive dissonance, which is useful I suppose, but I wish people wouldn't use it because it allows a feeling of resolution without a real resolution of the tension between trusting and verifying.

https://en.m.wikipedia.org/wiki/God_helps_those_who_help_the...

I'm not saying agile is the right solution, (it probably isn't), but expecting higher-ups to fund a black-box team is kind of naive.

"This just became a JOB..." -Gilfoyle

[1]https://www.youtube.com/watch?v=oyVksFviJVE S01E05 Scrum scene

I do like Agile, it's the typical problem of managers misusing it to micromanage, engage in useless metrics etc.

Reporting on random internal stuff instead of the actual problem at hand is the #1 problem I see with corporate reporting, everywhere. I see various combinations of people wasting time measuring:

1) The thing that is easy to measure, typically money or time.

2) The things they "understand", typically people for HR, compliance for legal, money for finance, etc...

3) The things their manager wants to know, no matter how irrelevant that is to executing their own job well.

Meanwhile what they should be measuring is the qualities of the end-product or the overall external customer end result.

It doesn't matter one iota if Bob the Developer Guy missed an internal 3-day deadline that John the Manager made up on the spot if the end product is a winner in the market and makes the users ecstatically happy to part with their money.

This happens everywhere, with everybody. For an IT-centric example, the common one I see is:

Helpdesk: "The users are complaining that the app is slow"

Admins: "The load is only 10%, but fine, we'll add more capacity!"

Helpdesk: "The app is still slow!"

Admins: "The load is only 5%! They should have no reason to complain!"

Do you see the issue? No, seriously: do you? Because practically nobody does, in my experience. Take a minute.

What happened here is that the admins measured the thing that is easy for them to measure: the load. There's a cute little bar graph in VMware, or a chart in their network appliance, or whatever. What they should have been measuring is latency from the end-user perspective, but that's hard to measure and practically no product tells you this number out of the box. So their entire process, their reporting, their troubleshooting, their forms, requests, everything becomes focused on the thing that they can see and control. Even if it's pointless, ineffective, and basically a waste of everyone's time and effort.

This happens with developers in exactly the same manner. Software quality is stupid hard to measure. Long term supportability is borderline impossible to measure without a time machine. Technical debt is hard to even explain to a manager, let alone keep tabs on in terms of numbers. So what's easy to measure? Time! Deadlines, sprints, release dates, etc... That's super easy.

That's why inevitably the unimportant internal time metrics become critical to everybody, but the actually important metrics aren't even measured and become invisible to management until it's far too late.

The core problem with Agile (most forms of software management) is that it massively overweights "first mover" advantage. I keep hearing, as a justification of agile, that software needs to be delivered quickly so that the company can go to market first, and gain marketshare while its competitors are still floundering. But, in practice, that's hardly ever true. I can't name a single product that succeeded solely because it was on the market first. I can name many products that were first to market and failed because they were clunky and difficult to use.

Heck, Apple's entire business model consists of being second to market with a product that is more polished and easier to use than its competition.

Yes, if a developer or team is well and truly stuck (as in spinning their wheels on the same problem, week after week), that's a problem. But you don't need Agile to tell you that. A simple weekly status meeting with incremental demos is sufficient. The only thing Agile does is create a bunch of graphs that allow management to comfort themselves with "story points" and "velocity" so that they don't have to confront the hard reality that they have no idea what it is they want to build.

And for the pm to feel safe enough to communicate to you something is wrong.

This feels like a Taylorism. Knowledge work isn’t factory work.

I totally understand keeping tabs on delivery speed to enable the team to benchmark themselves but the act of identifying a problem from that (if there is one) should be the teams responsibility imo.

As a manager, my job is to enable other people to do their job the best they can.

Which stops misunderstandings like throwing the code to “Ops” and expecting the VMware admin to understand how an app behaves.

User response time is a fairly simple metric to measure if you’re a developer. They should expose that, and other metrics, that the customer values.

This is one of the reason I’ve enjoyed some “true” agile teams with a strong product owner: they encourage an end-to-end focus on outcomes.

vs.

"Not everything that can be counted counts, and not everything that counts can be counted"

It is recognizable to many people, some of whom use it for their benefit, which can be very effective. When I learned that last fact a lot of things made more sense.

But time and money are the things by which companies live and die. Not keeping tabs on them is suicidal.

OTOH measuring time and money consumption of some technical internal steps is just uninformative.

Nah, you have it completely backwards. If the users said “this specific job took 5 minutes today but was only 1 minute yesterday”, that’s actionable, you can e.g look at what changes were deployed overnight.

But users always say “the system is slow”, even if they have only the vaguest idea of what “the system” is, and even if it’s actually faster than yesterday. It’s not really clear what any sysadmin can do other than spending hours every day painfully extracting the details from the user only to find nothing is wrong. Every day, forever.

That's not true. It's just that most sysadmins don't bother to upskill to find out what they can and should be doing.

> painfully extracting the details from the user

Asking users for any information is a recipe for disaster. Much like witnesses to a murder that can't agree on the most basic details, users inevitably conflate totally unrelated things. E.g.:

"Citrix is slow?"

"Okay, how so... are button presses slow to respond to a click?"

"I couldn't log on. Something to do with my password. It's slow."

"ಠ_ಠ"

So don't ask. Don't rely on your users at all. Build synthetic transaction tests that act like users. Measure end-to-end latency. Sit down with them and watch them work. Don't rely on their verbal feedback, use your own eyes. Use your tools. Measure. Then measure some more.

Conversely, capacity metrics are largely irrelevant in the era of 10 Gbps networks and 64-core server CPUs. Focus on latency. Look for delays. Timeouts. Deadlocks. Firewall packet drops. That kind of thing.

> only to find nothing is wrong. Every day, forever.

Of course something is wrong! Something is practically always wrong, that's why the users are complaining!

Here's a fun rule of thumb for you: For every 1 user that complained, there are between 100 and 1,000 that had the same issue but shrugged it off and didn't call support.

I got that from a scientific paper. I couldn't believe it, so I measured it in a large 10K user system. The error-to-call ratio was about 500-800 in ours. It blew my mind, and it blew the minds of a lot of people in IT management.

We started gathering every error, tracking every possible latency measurement we could, and it was a horror show. 30K app crashes per day. I shit you not. That's about 3 per user per day! Data loss. Hangs. Login failure rate of nearly 50%.

It tooks months to triage the issues, push patches, and apply workarounds. We had to rewrite several components. We eventually got the errors down to less than a hundred per day. Believe me, that was a real achievement.

Users were so happy they were begging to be migrated to the new system instead of pushing back and refusing to upgrade.

If the users are complaining, something is probably very wrong and you just don't know it. Go look.

I wish I had your user community. Here in Wales the ratio is reversed, I guarantee it.

Nowhere is this more apparent than the so-called Hackathon, where you do days of overtime in exchange for a few slices of pizza and worse, are expected to be grateful to management for providing the opportunity to do so! Nothing infantilises the profession more.

Granted, one can filter out 99% of product ideas on a whiteboard but the remaining 1% is still enormously huge. And that 1% is actually the stuff that nowadays gets started.

What about the value of those who cooked your lunch? Without them, software engineers would starve and die, so that makes them have far more value, no? Do you think that food service workers are also underpaid?

Now contrast a Facebook programmer, who in one hour might be able to fix a bug that has been annoying 0.1% of Facebook's 2.5 billion users, causing them to be frustrated rather than delighted for, say, two minutes a day, for the next three years before the feature gets rewritten. Maybe that sounds trivial, but if so, shut up and multiply: 2.5 million hours of delight per month for 36 months gives you 90 million hours of delight, for the same hour of work.

So, at a rough estimate, then, the gourmet hacker is 1.5 million times as productive as the gourmet chef. Maybe if I've been overoptimistic it's only a factor of 100,000 or 10,000, but it's huge. And that's how Facebook can be profitable at all despite all the shitty and stupid things they do: capturing even a tiny fraction of the value they produce makes them wildly profitable, as long as they can successfully externalize the harm they do. Some software companies don't even need to externalize their damages.

And that's why software is eating the world.

That isn't the only reason hackers get paid more than foodservice workers. No business is going to pay more for its inputs than it is forced to; that reduces its profits. Hackers also enjoy a dramatically better bargaining position than most foodservice workers, because a hacker with US$3000 has a better BATNA than a cook with $3000: the cook is going to be trying to sell $3 burritos out of an Igloo ice chest outside of concerts while the hacker can buy a laptop, bring up a couple of VPSes on AWS, and spend a few weeks putting together a useful web service, maybe get a few dozen to a few thousand users, but at any rate can easily scale to hundreds of thousands of users. The cook is dependent on someone investing a few hundred thousand dollars (in the US) to have top-quality tools and a good location. Difficult to do yourself unless your family is rich or you graduated from the Cordon Bleu.

This is purely factual reasoning, so it cannot answer normative questions like whether it would be ethically better to pay hackers more or less. It only purports to explain the chains of cause and effect in the world that give rise to that situation, and illuminate the other possibilities inherent in the current state of affairs, and how they might change.

(Also I don't know where you're from that burritos might be sold out of an igloo chest, that just sounds gross)

That's what FAANG are competing against when they hire hackers. And that's still true even though most hackers didn't apply to YC last fall, because they can sign on as employee #2 or employee #20 with someone who did. Because the other factors of production are not scarce enough to matter, jobs for hackers are abundant in a way that jobs for chefs are not. Patents, noncompetes, H1Bs, the anti-poaching conspiracy, and API keys are efforts to change that, but mostly they haven't been very effective.

(What makes you think the cook is a "he"? Both of the people I was basing that sketch on were women.)

° By "talent" I mean "strenuous and persistent effort by highly skilled people", not some kind of inborn genius. You can't start a business by sitting around thinking deep thoughts; you have to work hard. But for a restaurant, working hard isn't nearly enough, and that puts foodservice talent at more of a negotiating disadvantage with respect to investors.

It's a fallacy to view anyone's work as less valuable when contributing to the whole -- everyone's work is essential in the ecosystem even if all they're doing is unclogging the toilets of all these knowledge workers toxic turds

Software engineers are perfectly well capable of making a sandwich themselves, bringing a packed lunch, or surviving on an empty stomach for a few hours until they get home. Not to mention buying lunch from somewhere else or ordering delivery food if one cafe is gone.

Unless you're positing a world where all food production, farming, fishing, etc. has vanished, then software engineers are in no worse a position than chefs, cooks, baristas, grocery store employees, or anyone else.

I have an idea! Let's call it DevFood, and make it a requirement for everyone.

I mean, we already expect the software engineers to analyze the requirements, so we don't have to hire analysts; we expect them to test their products, so we don't have to hire testers; and we expect them to do the operations, so we do not have to hire an administrator. So tell me, why do we have to waste money on an extra guy who makes the lunch? DevFood is the future of software development!

Their productivity would drop a bit, though. The idea is that people making food enable developers to deliver their best work.

Yes.

>Without them, software engineers would starve and die

Have you ever cooked your lunch?

Right, CEO salaries skyrocket, not necessarily because of the value they create, but because they can get the board and stockholders to back them more than the people actually creating value in a company.

They can do this in the form of stock buybacks, dividends, etc.

The standard software engineer does not have the power to "bribe" the stockholders in that way.

I realized that some level of not getting 100% of the value i generate is worth it to have someone else deal with all that stuff.

I'm not saying whether or not engineers are underpaid for the value they create but it's worth something > 0

I've been thinking of going the same route, and I'm confident I can handle the paperwork and deliver, but it's the finding the clients part that worries me the most. Not sure if the problems I see are widespread enough to warrant founding a business and going through all that...

If there were only a handful of qualified engineers available on the market, then they'd get paid CEO-like salaries due to the value they provide and being in high demand. But because there is a whole market of qualified engineers they get paid far less.

It's pretty basic economics and doesn't have much to do with stock buybacks/dividends/etc... Regardless of the value of an individual position, wages will be lower or higher depending on the supply of an occupation. In the big picture engineers are very replaceable at market price compared to executives so their wages are reflective of that.

This would be counter-intuitive, because low demand results in high salary, regardless of supply.

https://i.redd.it/lnkq6l9zc0v41.jpg

Engineering is one of the only professions that is paid commensurate to the value they provide

Calling our salaries 'exorbitant' leaves you without superlatives left to describe executives that get paid thousands of times our wage, and at least they do work compared to the owners of capital who are 'paid' incredible money without doing any work at all.

To call an engineer salary 'exorbitant' you must be comparing it to the salary of people who function as modern serfs, and are not paid commensurate with the value they provide, or even enough to live a safe and satisfied life. Being paid enough for modest freedom and security is not 'exorbitant' though- no matter what you're comparing it to.

I can only say that IME a team is going to be a lot more productive if they are able to own their development process. Good infrastructure, knowledgeable engineers, healthy relationships instead of competitive ones - that's what creates a highly productive team and highly productive engineers.

The number one main headline takeway axiom of Agile is that this question is completely banned.

Agile is about delivering working software continuously, delivering whatever increment you get working at each timepoint, adapting to observed reality as it comes, and not having deadlines for specific features that might turn out to be impossible or harder than estimated.

If you babble around saying plans and deadlines are for suckers and that things will happen when they will happen, I am not sure they will keep paying you and your team's salaries for long. This new contrarian cargo cult of waving away any kind of medium-term planning or estimation is hurting businesses as much, if not more, as when all projects were waterfalled and timeboxed to death.

Yes, estimating is hard to get right. No, it does not mean we should simply get rid of it. Because engineers do not live in a little bubble of code: there is usually a whole company around them who need insight into what is getting built and when they can expect it. And anybody who believes that asking this question is too much or intrusive has never worked in a non-engineering position, or think of themself too much.

This process never stops, until you have some toothless PM slave driving devs because they're 8 levels removed from the strategic planning and they've pitched a 40-point/week development rate to their boss because they don't have enough information to do anything else. Anybody that tries to break the pattern is going to find themselves on the receiving end of an unfavourable performance review in any big business I've ever worked for.

There's a happy middle ground somewhere, where devs are given plenty of space to work but their output is still tied to the business's long term objectives. How you reliably reach that middle ground is probably a 8/9/10 figure question depending on the business.

The challenge is that management is only looking through a small window and everything from promotions to raises to favours are dished out based on that window.

Anything not in view of that window immediately obviously becomes useless to one's advancement in a company, so the window better cover everything which matters and it usually doesn't.

The question of whom things are visible to is an important one. Your comment says "management", and yeah that's one common way of thinking about it. But managers are just individuals with their own motivations. It is really "the business" that wants visibility in order to best accomplish its goals. There is an assumption that "management" is better aligned with "the business" than the employees they are paying "exorbitant" salaries to. But that may or may not be true, managers are also just individuals with their own motivations. So to the extent that it is true that managers are better aligned with the business, why is that the case? I see a few things that trend in that direction: compensation that is more tied to the performance of the business (through bonuses or what have you), more visibility into the goals and reasoning for decisions being made by leadership, and a better grounding in the dynamics of the business (what is the strategy, how does its market work, how does it make money). The theory is that because of those things (and probably others I'm not thinking of), the managers are better suited to be representing the interests of the business, whereas the engineers are just tools those managers can use to further those interests.

That is one way of doing it, probably the most common way, and it definitely seems like management visibility into work is extremely important in such a setup. But another way might be to align the engineers with the business in the same way managers are: pay bonuses based on the performance of the business, give them visibility into goals and decisions, teach them about the strategy of the business and the dynamics of how it makes money. That is, give the engineers ownership in the same way management has ownership, and trust them to make decisions that are in the interest of the business.

To put this another way: if you trust managers to make decisions about trade-offs that are in the best interest of the business, why can't you trust engineers as well? Are the engineers dumber or shiftier than the managers such that they can't be trusted with this?

Assuming that neither side is doing significant amounts of extracurricular work to bridge the gap, it therefore follows that management understands the business better, because that is explicitly the purpose of their job.

In a best case scenario, it makes sense to provide engineers etc. with the context to understand the decisions. On the flip side, business decisions are often made upon a huge heap of context that is generally invisible to the engineer (unless they spend an equal amount of time in building up their business knowledge, spending time in meetings, etc. - but at that point, they're basically a manager?)

It's not to say that engineers can't develop the background, etc. to make the decisions, but that it's not exactly part of the expected job functions, and not something that's explicitly looked for as part of the recruitment process. In a sense, this is a bit of a tautological issue.

What I'm saying is: what if we rethink specialization on this front? What if we expect it to be part of everyone's job to have deep understanding of and context on the business? Recognizing that there are significant efficiencies in specialization that we lose with this approach, maybe it still comes out ahead. Maybe the improvement in decision making outweighs the inefficiencies introduced by requiring everyone to be a businessperson.

Personally, I think this is a really great way to run a business, if you can get broad buy in for it. The best places I've worked have been those that most closely approached this structure.

But I will admit that this is biased by having a personality where I neither want to be "just" an engineer, nor give up engineering entirely in order to be a businessperson. I think lots of engineers don't want the business stuff to be part of their job, and I guess I can understand that, even if I lament it.

One interesting question is: if everyone is acting with the business ownership mindset more commonly expected of managers, what are managers supposed to be doing? My answer is that front-line supervisor managers are there to support their reports, help them get and understand the information needed to make the right decisions. At the higher level, managers are there to make decisions on tough cross-team and cross-organizational trade-offs. At the very top level, they are there to set and communicate the overarching strategy of the business, which everyone else should be using to inform their own day to day decisions. I think there is plenty of that kind of work to be done, without expecting managers to be making the day to day decisions.

that's crazy talk ;-)

As far as business deserves to know what they are paying for, that is absolutely correct as well.

The problem is that business is not paying for story points. They are paying for actual product and delivery. So when they measure output by story points, you end up with a situation where teams are trying to maximize story points delivery, etc. as opposed to actual product delivery.

but that's the whole problem, if you constrain time, cost and result there's no any space left for agility.

the point of agile is to maximize the value of what can be produced with a given team within a given project, you picked time and cost and you apply agile to emerge stories that are actually important to you and push back on the gold plating.

if you have all three fixed, waterfall works just fine, actually even better.

Having working software is line two of the Manifesto. They can run it each week and see whats new.

Even better, they can provide feedback on the working software and the team can then quickly respond to that feedback. In a week or two, when the outsider next looks at the working software they will see the software improved.

The devil is in the details and in this case literally because the granularity of progress is too fine-grained to be exposed away from its context. The project manager or whatever you call the nearest person with power over your time needs some leeway to organize, ask for some extra, compensate, forgive, and reward you, without higher powers micromanaging. Freedom is the premise of responsability.

Absolute evil happens when customer demands access to hourly activity log for each developer in the (not on premises) project.

Is it? Maybe it's the opposite? Citation needed.

Most often it is not. The problem is that management does not understand shit about software. And they usually don't care to learn about it. Hence this is about as effective as me asking for transparency regarding a COVID-19 vaccine. Yes, the practitioners could tell me they are in a phase-II trial. But to process that information fully and put it into context, I'd effectively need to study medicine.

And as you say, managerial involvement is the problem. Managers and execs mostly get paid to appear to be in control, so having the appearance of control is vital to them, even if that makes the results wildly worse.

A better saying would be, you should only use your target as a measure.

If your target is working software, measure how well the software works.

In your imaginary case, what would your specific numerical measure be?

I've legit sat through 1.5 hour long calls with 6 other devs so we could watch our manager type notes into a presentation that he'll give to a collection of other managers who will only be half-listening until it's their turn to be visible, and all because some other, more powerful manager, decided this was important for the org to do.

It was layers of bureaucracy so deep I wasn't sure where I was anymore.

The primary point of Agile is to empower developers. If management is determined to micromanage them anyway, then Agile is pointless. If the term Agile is to have any business value at all, then let it be the stick with which developers beat some sense into stupid managers. Points are indeed rough estimates and can easily be off by a large factor, and they're definitely not meant to measure performance. Standups are about the team and about helping each other move forward. Demos are about getting feedback from stakeholders. It's fine if others see the Jira board, but it's not theirs. It's the developers'. And if a task isn't done, and it isn't done.

I fight for this if necessary, but fortunately I'm currently working in a team of very vocal and opinionated developers who take charge of their project and push back against managers if necessary. We do Scrum, but we're not dogmatic about it. We change the sprint scope if the situation calls for it. And we rebel when it's management that's calling for it.

It's certainly reasonable for those various stakeholders to ask for visibility. Demos are great for that, and a lead engineer or front-line engineering manager can work with stakeholders to establish some metrics to report on, but those are epiphenomenal to the process chosen.

I'm not saying it's a guarantee, just that nothing else really works well.

and it was understood and expected that this meant the estimate would go up

Today we have 'bidding', which if that sounds like something a three year old does to their parents, you'd be right, and it's the same thing. You just ask someone else until you get the answer you want to hear (instead of the one you need to hear).

It's not the same thing. It's not "bidding", even if it's called that because the lowest or highest or middling bid doesn't necessarily get the work. I know, I know, it sounds like another "you're doing Agile wrong", but Agile already fucked it up by saying "bidding", so we get these wild stories of how companies have their laziest incompetents in charge of trying to make improvements to process.

> You just ask someone else until you get the answer you want to hear (instead of the one you need to hear).

Otherwise you will only have the most jr or senior people working on everything. The system of bidding just doesn't work, which is why it's not bidding in a practical sense.

You ask why people put down their estimates by describing what they know about the work. This is an opportunity to transfer some cursory knowledge and reinforce pain points. At a very progressive "Agile" company (we would invite speakers and A/B tested various practices across teams), I notoriously inflated/deflated all estimates by bringing up edge cases and/or constraining objectives. Story grooming aside, there are always people trying to over-engineer and estimation is another gate to prevent that.

Agile was a good try to create software process, but it's incomplete at best. The idea that it would evolve is laughably naiive (https://app.spectator.co.uk/2020/04/22/the-illusion-of-certa...). Companies want S&Ps, not sauce that they have to mix themselves.

I'm talking about the dickering about whether something is N points or N-2 points. Or whether the points even mean the same between two teams. Or whether it's appropriate to assign it to another team or individual who says a smaller number because they have a different definition of Done or are more or less invested in the long-term health of the project.

I'm going to ask this person or the question in this way because I'll get the answer I like, even if that answer is unhealthy or even pure fiction.

We should make "estimate" a banned word and replace it with a word like "guess".

Too often what starts out with an off-the-cuff chat about implementing some feature while people are getting coffee with a statement like "Oh I don't think it will be too hard to do this - will take a week tops <filddles with coffee machine />" etc ends up growing its own legs and becomming a gold-plated and irreversible commitment made to a director/VP/CxO somehow. What was a casual guess made without all the information required is now a rod for.our own back. Again.

Perhaps as engineers we should give ourselves a sort of mental style-guide to never ever say the words hours/days/weeks/months/quarters/etc without alarm bells going off in our head and requiring extensive peer review in the same way we do when we're writing "dangerous" code (e.g. user-provided values going in to SQL queries etc) Only half joking really :)

[1] https://www.scrum.org/resources/blog/commitment-versus-forec...

I'm getting really tired of companies trotting out the "because [Facebook|Apple|Google] does it" excuse. Those are massive companies. Are we so delusional to believe that what works for a $70B/year company will automatically work for our $20MM/year company?

It was a good way to get your head bitten off.

The idea that FAANG are faster then your startup is absurd on its face. The revenue and legal risk is absolutely huge.

One difference between those highly paid software companies and others, is the expectation of performance. Netflix famously considers their team a sports team rather than a typical corporation. It's totally common for a great engineer to bomb out of those companies, just like how great athletes get kicked off top tier sports teams when they failed to perform due to reasons out of their control.

While most other companies, lacking the ability or willingness to pay top dollars, are probably better off creating a safe environment for engineers who can be on-par or even better than FAANG engineers, but lack the political maturity to survive those high pressure environment.

They saw a process that did an end run around them in favour of more self-management, and it worked well and got a lot of hype, that's terrible for their careers obviously.

So they fixed it and kept the same name for marketing.

Agile is about culture and the culture of the pioneers and early adopters was different.

I‘m seriously thinking I should try to go more the Kanban direction these days to aboid the story point hitting / fitting in ever shrinking sprint trend.

The reporting is a whole other issue. I don't use this, but my PM does because upper management keeps changing their reports and my PM has to keep linking and organizing. But as a developer, doing spring planning and burndown charts work quite nicely.

Gonna play devils advocate here...would those two be mutually exclusive?

I feel there's a fundamental flaw in the garden of agile eden you initially described and that's trust...if you don't have it or lose it then gl trying to convince someone to essentially write a blank cheque for something they don't understand

If trust can erode a whole methodology then it's something to consider imo

As a customer, you get to decide what you want to buy, and you get to be happy or unhappy with the delivered product.

But you don't get to control or even monitor the work in the factory/kitchen/etc.

That's instead the job of a boss! Of course, an agile team also has a boss. But it's not the customer.

But usually some proxy for the customer in your organization has to take on that role.

There are no "rules" in agile that says who can do what, just teams of people negotiating what works best for them. "Individuals and interactions" over process.

Not claiming it's inscribed on some sacred tablet.

A great talk that I make everyone watch is John Cleese on Creativity In Management, it is a must watch for value extractors dealing with value creation that ultimately is creative work [1][2].

The open mode is allowing space, time and confidence. Closed mode is finishing/ship it mode.

"CLOSED" MODE

> By the "closed mode" I mean the mode that we are in most of the time when {we are} at work.

> We have inside us a feeling that there's lots to be done and we have to get on with it if we're going to get through it all.

> It's an active (probably slightly anxious) mode, although the anxiety can be exciting and pleasurable.

> It's a mode which we're probably a little impatient, if only with ourselves.

> It has a little tension in it, not much humor.

> It's a mode in which we're very purposeful, and it's a mode in which we can get very stressed and even a bit manic, but not creative.

"OPEN" MODE

> The open mode, is relaxed… expansive… less purposeful mode… in which we're probably more contemplative, more inclined to humor (which always accompanies a wider perspective) and, consequently, more playful.

> It's a mood in which curiosity for its own sake can operate because we're not under pressure to get a specific thing done quickly. We can play, and that is what allows our natural creativity to surface.

...

> Humor is a natural concomitant in the open mode, but it's a luxury in the closed {mode}.

COMBINING OPEN and CLOSED MODE

> Once we've taken a decision we should narrow our focus while we're implementing it, and then after it's been carried out we should once again switch back to the open mode to review the feedback rising from our action, in order to decide whether the course that we have taken is successful, or whether we should continue with the next stage of our plan. Whether we should create an alternative plan to correct any error we perceive.

> And then back into the closed mode to implement that next stage, and so on.

EXAMPLE

> ...one of Alfred Hitchcock's regular co-writers has described working with him on screenplays.

> He says, "When we came up against a block and our discussions became very heated and intense, Hitchcock would suddenly stop and tell a story that had nothing to do with the work at hand. At first, I was almost outraged, and then I discovered that he did this intentionally. He mistrusted working under pressure. He would say "We're pressing, we're pressing, we're working too hard. Relax, it will come." And, says the writer, of course it finally always did. We need both modes.

The world is value creators (product/engineer/creative/imagination) and value extractors (business/management/finance). Once the latter group get control, they end up squashing the "open" mode that is so key to making good products. The value extractors increase pressure but sometimes lowering pressure will let the product iterations flow to a better product end result.

Value must be created first before value can be extracted, but you can't force value creation with only the "closed" mode.

Value creation processes that make sure there is an "open" mode along with the "closed" mode are always more successful, and sometimes the "open" mode doesn't look like work so the value extractors cut it first unknowingly killing the product slowly.

The wrong type of Agile removes the "open" mode and cuts prototyping, iterations and refinement of creative value creation. A more open type of iterative development is always better.

[1] https://www.youtube.com/watch?v=Pb5oIIPO62g

[2] https://genius.com/John-cleese-lecture-on-creativity-annotat...

It's well known that the systems an organization produces are bound to reflect the communications structures within that organization this is known as Conway's principle

In order to maintain flexibility in the design it's necessary to maintain flexibility in communication but engineers have no incentive to push for free communication and stick together to see the job done right, each of them is attached to a management minder who reports to a rigid top-down governance structure with very different ideas on how software should be done, effectively stifling any potential dissent at the source and acting like an authoritarian regime with the desire to divorce the workers from ownership of their work and turn them into compliant and productive cogs in the machine

The reason for this is not because corporacy sucks but because humans suck at corporacy and when placed in group environments of over a handful of people we tend to resort to mob rule and groupthink at the expense of culture and individual rights

It's rather complex but incentives are the clearest indicator of outcomes people simply have no incentives to help each other but every incentive to seek permission and recognition from management

In such an environment it's been postulated that four types of personalities emerge as people attempt to exert control and these are known as Control Dramas:

- The Intimidator

- The Interrogator

- The Aloof and

- The Poor Me

You can look these up there are several good articles online about identifying and mitigating these but it's not always just one but a combination that people exhibit

Framed in this context you can see corporations as implemented naively operate on only four cylinders even if it's only figuratively or intuitively true it's far from the humane ideal of what agile intended and this also has to do with metrics becoming targets and incentives trumping ethics

In the end we trumpet DRY and try to 'focus on our work' while we repeat the same inane insane dramas day after day expecting a different outcome that's the worst form of repeating yourself but we collectively walk into it in every case

I'm personally carrying a lot of resentment at the engineer class because we've been trained to know better but we simply can't get over ourselves and wake up to the fact that we're not just part of the problem, we are the problem but as such we also possess the solution if we really sit down and think about it

The solution lies in the following concepts

A) it's possible to have alignment and autonomy it's not necessary to choose between chaos and blind compliance

B) It's possible to have investment and productivity but it's necessary to trust people and that ultimately is where we fail

It can still be evaluated on the merits but IMO this greatly pollutes the speed at which software devs as a broadly conceived community can come to consensus understanding of this.

Also I think the comparison to lean manufacturing has always been very shallow. I get the metaphor, I just don't think that human resources in engineering can be optimized like manufacturing processes. This quote is the best part of the article:

> "You’d never hear anyone say, 'We help mechanical engineers be agile. That would be silly. And I mean that in the worst possible sense of the word".

As for the rest of it, I'm not dying to hear what the person who invented Agile thinks we should do next lol.

Absolutely. And, having done some agile consulting long ago, I'd say it's more pernicious than that.

The people who truly want to make deep change in pursuit of deep improvement are a small segment of the market. Worse, they don't need a lot of help. In the aughts, I had a few clients who really got it after 3 months of focused work, and then they were off and running.

But a large company that only wants to talk about change and maybe make some 5% improvements if they aren't too hard? That can be milked forever. Well, I can't, because I care about results. But consultants who either don't care or don't notice? They're golden.

And I think this failure of the Agile movement has been obvious for a decade. I gave up on Agile conferences circa 2009, and wrote a long piece about this in 2011: http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how...

I've seen several director types get promotions promising to get faster and more reliable work out of existing engineers by implementing this new religion... Agile.

The results were always predictable. Layering more meetings and stress on people who already have too much work to do, doesn't help things. But, you're still vice president.

Agile has plenty of good ideas.

The problem is the almost religious following and, as you mention, the whole industry that has sprung up around it.

Even the initiators said that it was supposed to be a rough framework, to be adapted to your individual circumstances and teams. It was also a (much needed) counterpoint to the then prevalent waterfall model.

Now we have consultants, people strictly following something they read in a book or learned in a course, adhering to strict structure of meetings/processes, and even a big association with a single software product, Jira. ("You are doing it wrong!")

When you step back, a lot of the ideas make sense, and many teams will even implement similar workflows without having ever heard about "Agile".

Common sense has to prevail though.

> Now we have consultants, people strictly following something they read in a book or learned in a course, adhering to strict structure of meetings/processes, and even a big association with a single software product, Jira. ("You are doing it wrong!")

Yeah. Part of agile is that your process is an adjustible parameter. If you're doing it with a rigid process - any rigid process - then you are not actually doing agile.

The team/organization has to become a learning organization. That means a number of things, but the critical one here is learning from past failures and successes and incorporating that feedback into their model (ideally continuously, but in Scrums it'd be the end-of-sprint retrospective). You start down a path, and you find it's difficult. You don't press on just because it's the one you selected, that's the way of idiots. You examine the hardships you're facing, and you address them.

It still is and it's barely in the fight still in some corners of the public sector.

If Agile dies (and SAFe might succeed there) then there would be nothing left except Waterfall and people others would call cowboys.

Saddle my horse.

Whether it was right, I'm not sure, but people have been happy with my work, so that allowed me a bit of leeway.

Yes. I have found doing agile "properly" to be too rigid. What we need is something more flexible than agile.

I have literally heard this pitch.

More generally, it's correlated to it bringing some real or perceived advantage to the person doing the talk.

I mean, have you been watching the news lately?

The closest it gets to that is where it say:

We follow these principles:

<snip>

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

<snip>

It basically states a set of attributes your process should exhibit.

Agile as I have experienced it in practice almost never displays those attributes.

The whole thing makes very little sense. I mean read it.

>"We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more."

How the hell do you get from that to where we are today?

What it ought to say is:

"We value short feedback loops that minimize risk and maximize learning.

We value flexibility.

We welcome change.

We don't know what we are doing, only what we intend to do. The outcome is a guess. Our guesses could always be better. We strive to make them so."

https://www.bloomberg.com/news/features/2019-05-09/former-bo...

First words of an agile consultant.

First words of the next agile consultant.

First words of an agile zealot responding to any complaints or negative comments about agile.

https://i.imgur.com/ecsh9yp.png

> > "You’d never hear anyone say, 'We help mechanical engineers be agile. That would be silly. And I mean that in the worst possible sense of the word".

This quote is silly.

To me, agile is just good engineering practice, applied to software. Of course mechanical engineers apply its principles, and have for decades before the term Agile was coined.

And as such, this practice is far older than software.

The Apollo space programme is my favourite example: the ultimate goal remained fixed (man/moon/before end of decade), but all steps of the way were discovered and redefined over the programme's course.

Mission objectives were changed depending on what was learned, often even in flight.

This was a very nice and agile (and sensible) approach, regardless of what it was called.

Of course, even more ironically, it doesn't fit the original agile manifesto:

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

https://agilemanifesto.org/principles.html

Which bothers me much, much less than selling things like CMMI/PSP certifications or EUP/RUP which are done purely for paper pushing and selling the paper value.

Agile is an improvement on waterfall and you don't need to be certified to do it.

Agile and Lean are empirical process controls, they are based on the same concepts. Ken Schwaber explains all this on the first chapter of his book "Agile Software Development with SCRUM":

Defined process control: same inputs always result in same output (manufacturing widgets on a production line).

Empirical process control: same inputs not always result in same output.

Schwaber conceived SCRUM (and was among the founders of Agile) after realizing software development required an empirical process control: give 2 dev teams same specs, 2 different apps will come out (they might do the same thing, but in different ways)

I understand why it would be super cool if software worked like that, but I have to iterate on features holistically and sometimes speculatively.

At least for me, software development has never been able to be so cleanly broken down into "first, implement the button" type tasks.

But maybe I'm just dumb.

What I do with my guys is I make them decompose the feature, but with the understanding that they'll probably be jumping from one subtask to the other, back and forth, as they understand what it is they'll be building. AND they'll probably come across new subtasks as they work through the feature.

It makes Jira look crappy, but it Reflects Reality (tm) which to me is far more important than keeping Jira clean.

Nobody can; that's the primary failure mode of waterfall. Anybody who is telling you to decompose a feature into that many subtasks might claim to be doing agile but they really are trying to do waterfall.

That's the primary failure mode of a straw man software development methodology called waterfall that Scrum practitioners like to use as a bogeyman to appeal to for rhetorical purposes during the inevitable arguments about niggly little details of how to implement Scrum during sprint retrospectives.

I've never actually done real waterfall, but I did study it way back when I was in college, and I have worked in government contracts, which tend to handle the work in a way that is similar to the waterfall I read about in school. One thing I distinctly remember is that it was designed to be a flexible and iterative methodology that would have been difficult to cram into a ticketing system in the first place, let alone cram into Jira while simultaneously carving the work into tiny pieces, up front, all at once.

FWIW, the only place I've ever been asked to do something like that is in ostensible Scrum shops. In fact, the only time I've ever seen anything that looks like the Waterfall of scrum lore is in shops that are trying to do Scrum. I am beginning to suspect that the Waterfall of Scrum lore isn't actually a thing that happens outside of Scrum at all. It's actually what naturally tends to emerge when you try to apply Scrum methodology in a situation where something along the lines of the real, textbook waterfall model would have been more appropriate.

As a concrete example, I currently work at a team that ostensibly uses Scrum, but where QA is a separate department with its own practices that the dev team does not control. They do still want some ability to anticipate work that's coming their way, though, so they're monitoring the scrum board for that purpose. This is the moment where it gets messy, because we're then asked to document things ahead of time, and it's a minor crisis if we keep it flexible during the sprint because any changes to the set of tickets becomes an inevitable hassle as they complain that we've screwed up work product that they generate based on those tickets. It's absolutely something straight out of waterfall horror stories from Scrum lore. But it's only happening because we're doggedly insisting on something that's at least cosmetically similar to scrum. If we weren't doing that, we'd be freer to choose modes of interaction and business artifacts that are better suited to the reality we occupy.

Out of this chaos, eventually came order. It turns out what mattered most in practice, just like before, was pleasing managers and executives. Effectiveness was generally secondary. So what we ended up dominating is Scrum, the most waterfall of Agile processes. Most shops "doing Agile" these days are still following a top-down, control-oriented, ineffective and unrealistic process, just like before. But now they use Agile labels for their mostly-unchanged process, albeit with a faster release cadence.

* A quarterly release cycle is indeed fast. * 12-18 months between releases (ignoring perhaps bugfix/point releases) is quite common.

You write that "everybody knew that couldn't work" - for some projects.

Definitely agree with your second paragraph - except that Agile labels can be used even without a faster release cadence :-)

The biggest difference for Waterfall-in-Scrum versus Waterfall is that they've sufficiently (hopefully) decomposed tasks to have short target dates for delivering something to a test team or facility. They don't use it to get feedback from customers (the actual users, not the ones writing the checks). They don't use it to replan when things go wrong. They just use the whip and OT and get back on track.

But as an example of early Agile intent, here is how we worked in 2004: http://williampietri.com/writing/2004/teamroom/

You'll note that we never did detailed planning more than a week in advance. We could have, but it would have been a waste, because it would have been speculation on speculation. Instead we'd agree on something small to build, get it working, and then see what we thought.

Some things need to be designed, and don't work terribly well if they're evolved incrementally from user-visible features. Incremental additions can be, but there's also a risk of gradually degrading the architecture of a system through risk-minimized local changes by interchangable resources, I mean developers - which is what I generally see occur under Scrum.

No, nothing is wrong with you. But let me note that the Agile Manifesto doesn't say anything about Jira, or anything prescriptive about how small to make your stories.

If your methodology asks you to do that, I'd argue that your methodology is broken. But it's broken because it's shit, not because it's "agile."

Yeah, it's hilarious how people are being told to do something in the name of "Agile" that is almost completely opposite of the spirit of the Agile Manifesto. sigh

Parkinson's Law doesn't get enough of the blame for bad dev environments. It's so easy to feel like you're doing 40 hours of difficult, important work a week, while actually accomplishing fuck all at best (or generating more work overall at worst).

I bet a big chunk of the problems raised in this thread don't exist if everyone experiencing them cuts their team size in half.

No, not at all.

Decomposing a new feature into digestible pieces is hard work and a skill that is learned over time. It's also a skill to know when something is small-enough, or has enough unknowns that further decomposition is wasteful.

Good product owners and managers know this and are good at it themselves. And the decomposition is typically at a story level - "can we take story X, break it into 2 smaller stories, and still deliver something of value?" The tasks are usually pretty self-evident once you break down the stories a few times.

For a lot of my work, if the story is "As a user, I need to do X" the tasks are simply broken into distinctly testable pieces.* Is there a database change? That's a task. Is there an API change? That's another task. Front-end change? Third task. And if you get half-way through and need to start over, that's life and it happens reasonably often. Yeah, there might be some re-testing. Or, a new story for next sprint.

It's up to me, as the manager, to make sure the stakeholders are informed and on-board. At the end of the day, as long as the team is making progress, I'm happy. And if the team isn't making progress, it's probably my fault (and almost never the fault of an individual contributor).

* In my world, tasks don't exist in isolation. They're all subordinate to a user story. Every task (piece of work) we do is done in the furtherance of a well-defined need of the user. And developers are never defining tasks on their own - it's done as a team.

It wouldn't be a complete decomposition, of course, a person watching would see your thinking unfold. I think speculative decomposition would be very interesting, especially in retrospective if we could see the rabbit holes we went down.

Decomposing is easy since we naturally have to do that as we work. Heck, those 500 tasks are probably in your shell, browser and commit histories, mixed in with a bunch of other junk.

The problem is tools like Jira make this heavy-weight, doing it up front makes it nigh impossible, and having others review all this stuff and it becoming promises blows up the LOE significantly. And, I don't think I'd want to be under that much of a microscope as I work.

But if it were very lightweight, where I'm just posting my thoughts and can see them as a quick dependency diagram, and maybe attach notes, commits and URLs to them, and other people can see them, that'd be pretty helpful.

That's precisely the objective of "user stories" and other things. You write a high level version, put it in a backlog with priorities. When you get to it, you realize it's bigger than anticipated ("Oh, I can't just do X, I have to do A, B, and C."). So you turn X into three things, one of which you work on now, and the others in the backlog. Repeat until done.

I don't think Jira is necessarily too heavy-weight, it's the way everyone seems to use it. They want to assign the tasks now, not treat them as backlog items or things that can be modified in the future. Which pushes them back towards big-design-up-front and entirely defeats the objective, you're back to low throughput, high latency development.

Forget the developer for a moment and take the user perspective. What's the first thing the user wants to do? Log in? Ok, that's a story. What's the next thing the user wants to do? See a list of widget prices? Ok, that's a story. What's the next thing the user wants to do? Change a price? Story.

If you can't sit down and think of a dozen narratives like this, then the fundamental problem that you don't really understand how people are supposed to use your software or what your software is supposed to do.

(Also a user doesn't actually want to log in, they want to do something and you are making them log in)

The task of login is more about security, access control, authorization and tracking, not just giving the user access to a web landing page.

That level of complexity makes it a project in itself.

But sure, you can have follow-on stories like "user is denied access protected resource" or "admin sees resources accessed by user". Enough individually-useless stories and you end up with a useful app. That's kind of the point of agile.

YES. THANK YOU! I've been trying to tell people this for ages. Maybe other people are somehow capable of perfectly predicting every conceivable edge-case and consequence without even starting to work on something, but I can't. I have to start building something in order to see it clearly. And this ENRAGES certain kinds of managers, I have found.

Here's how you can solve your problem: Make as many tickets as you can think of, it doesn't really matter if they contain the essence of the problem or not, it just matters that there are many tickets associated with the task. Once you have your tickets, estimate them with as much padding as possible. If the manager complains that you're padding a ticket too much, make two tickets and split the estimate.

Once you have a big pile of tickets and estimates, you've bought yourself enough time to do the work. Go do the work, then mark all the tickets as complete. You'll figure out the edge cases as you go, as long as you've made enough tickets and time to do the job at a reasonable pace.

If you have extra time left over, get started on the next thing, improve your tooling, or try to help someone else with their tasks. If you don't have extra time left over, make more tickets next time.

My boss's goal is to try and get everyone to work every waking hour so we can deliver faster. My job is primarily to keep the peace between the executives and the workers. When I fail the workers, it looks like unpaid overtime. When I fail the executives, it looks like undelivered features.

I think the actual purpose of agile, from a developer's perspective, is to create more time to deliver quality engineering work product. From an executive's perspective, agile exists to squeeze developers into completing tickets faster. As a manager, it's my job to keep both parties as satisfied as possible. If engineers have enough time to build and executives get enough deliverables, then the company succeeds.

What's your strategy?

As I noted in a parallel comment, on my team, developers are never creating tasks on their own or in a vacuum. It's done as a team effort during planning meetings.

Perhaps I should add that my boss (senior director) has never asked me about my board, velocity, or anything like that. Discussion is higher level - "is project X on schedule?" or "are you going to miss anything on your roadmap?" not "why is task X or story Y not done yet?" If your director is asking that, he has too much time on his hands.

Asking specifics from higher ups is just smart.

I've reported to two different directors since moving into management. Both were very clear that they'd rather hear about problems early. And both have been nothing but reasonable when I've reported problems. Of course, the entire team needs to be reasonable for this to work, from the product managers to project managers to individual contributors.

And as a manager, nothing annoys me more than an employee who tells me everything is fine then misses a deliverable. Tell me you're having problems so we can find a solution. Please. That's why I'm paid what I'm paid.

https://blog.codinghorror.com/tending-your-software-garden/

The reality is that your ability to break things down depends on experience. I’ve built so many landing pages, for example, that I can tell you an exact breakdown of tasks. Been a part of so many SaaSes, I can tell you exactly what non-core features you’ll need, when, and how much work they are. I can also predict where you’ll hiccup.

But ask me to build something new that’s never been done before (at least by me) and the most I can do is shrug and guess. Maybe give you a rough sketch of an outline of subtasks.

Here is why it is important:

a) Risk management - if you break a feature down into sub-tasks at any granularity, you are creating an agreement (or at least a conversation) among your peers that this is how something will be implemented, and digging in beforehand to uncover areas which might impede shipping the feature sooner.

b) You're going to have to break down the feature at some point. Being able to think through this ahead of time can be challenging, but often times is the meat of the work you do. You have to get into the hang of it -- think top-down or bottom-up ways of approaching it.

But what if you don't have enough information or understanding yet to do that? Agile is not great for these tasks where you need to take some time learning or experimenting.

I would offer you a couple extra tools here:

- A "spike" ticket -- Agile is all about deliverables and commitments, but sometimes that doesn't work. So create a spike ticket, and define what it is you want to investigate. If you deliver something, great! If not, no worries. The important part is that in the future you've done work that enabled you to learn how to estimate or break down that task in the future.

- A time-boxed ticket -- similar to the spike, but you just make sure you don't spend more than an allotted time on a task.

Basically, you blaze a trail to the most disgusting, kluge-y, and otherwise slapdash implementation that validates your assumptions and satisfies your constraints while making note of everything that needs further work. That lets you bail early if some unknown issue will block success given the current criteria and constraints, and sets you up with the a list of the 80/20 work required to deliver the completed project.

Company and developers are busy and successful, so they hire more, which continues until they are no longer successful.

They remain busy, however, progress on the product is roughly constant. All the additional work capacity is spent on meetings and otherwise organizing the increased worker count.

I believe the above describes every situation in which I have been asked to break down Jira tasks into unnaturally small tasks.

To be able to decompose a feature, you need to learn how to build the software in your head before you go to paper. Consider a large billing page, for example.

Step 1 is to look at the front end. What components do I need to build? (React might use components, or a set of partials and templates in rails, for example.) So, then, I know I need X subtasks, where I will build the pieces I need to expose that behavior to the client to start.

I then look down into the backend. That might be a simple task on the backend, or given 20 minutes of looking, I might see problems that I'll need to tackle. The problems are their own subtasks. I may be able to group some of the front end to a single back end ticket, or I might need a separate subtask for each.

Now, I'm likely to miss something. However, that's why this is an estimate rather than a concrete task list. It's to get started so that we know, at minimum, what it's going to take.

It's not necessarily simple, but it is a skill that can be developed.

Even easier than building the software in your head is actually building the software. I often don't know what the shape of the problem is until I've built something. The users often don't know what they want until they've seen something.

The failure of Agile as implemented is that it's still all about extensive meetings and planning. Software is not like building a bridge where you write out the blueprints and the expensive part is then building the bridge. Software is the blueprint.

To be truly agile is to be able to get software features and changes running and out to users as quickly as possible. You can never get all the information -- if users could provide the details absolutely necessary to build the software on day one then they'd be the programmers. Being experienced helps you naturally scope problems but you'll still be wrong from time time. Being able to iterate on users feedback and dealing with your own failed assumptions is the key.

First off, it means your code is brittle. Second is that you can't trust tests that must change every time your code changes. Might as well not even have them.

Unit tests are supposed to tell you if your refactoring is still working. For the most part, you should only refactor implementation details, not the "API".

There is also a design to uncertainty trade off. The less you know about something, the less time you should spend designing because you can't design something you don't understand. But the more your understand something, the more time you should probably spend designing.

Knowing how much design is too much is subjective and personal. I forget where I read it, XP(Extreme Programming) or Code Complete, but someone worth listening said, to paraphrase, "I've never regretted spending more time on design before I started coding". And I'm pretty sure it was part of XP that said the main purpose of XP(Agile) is to deal with the unknown. The more you know, the less Agile and the more waterfall you should be, saying that for some projects, a 6 month sprint is perfectly valid.

I am in an interesting situation where I am both the programmer and the domain expert. Generally customers come to our team describing a problem that other companies can't figure out. We're pretty much allowed to design everything how we feel like because we're the best at it. We can do virtually any project in a single go of it without feedback from the customer, but we still do agile because it helps with scheduling, planning, prioritization, etc.

We're a "value add" team, which makes things really interesting. None of our work is sold, but the work is important to land large contracts for the companies core services.

This is the majority of estimation failures as far as I've seen. You can only "estimate" so far and then leave significant padding for all the other minutia that will likely creep in.

You should still have more granular planning, but it should serve the purpose of aiding the dev in completing the feature. You decide to do more granular planning when, as a dev, you look at a feature and go "The implementation is still ambiguous or I still have questions about how it relates to business goals".

Basically, invert the communication channels and shift responsibility to the devs. They have some loose business goals to hit, and the other staff provide support when they need it.

This solves so many problems I run into in enterprise dev. It kills Parkinson's Law (since the team members that are in the bottleneck are now responsible for the bulk of work generation within the team), it also gets rid of a whole class of poor decisions that are made without the proper context (since now you have a closer relationship between business goals and development).

Maybe it was just my particular job, but so much of my work was figuring out how to do xyz, so it was hard to give estimates for something whilst I was still figuring out the scope and complexity of along with how it even works and I was rarely, if ever doing the same/similar thing multiple times. Whenever project managers did push for me to break things down further they’d then immediately complain about too much technical detail.

It has always been at systems thinking, design and architecture, in that order. Doing it organically is a magic feat rarely done well.

Which is to say: To all the people jeering for Agile's demise, please provide a superior alternative. I came from a world dominated by Waterfall, and I never want to go back to that. A lot of companies get Agile wrong, and it isn't a panacea, but it is much closer to what developers are naturally inclined towards (e.g. rapid incremental improvements) than Waterfall ever was.

So I challenge anyone who wants to replace Agile, please lean into how developers work rather than trying to mold their work onto your rigid front loaded methodology. Trying to bring in ideas of other industries, like engineering or architecture, that build physical goods and only have one shot at is a folly.

One of the reasons Agile (with a big A) is so successful is that its peddlers have convinced everyone that there are only two options: Agile and waterfall. When you're up against a strawman, it's easy to win. But it's absolutely a false dichotomy.

There's a quote in the article that I like:

> Find me an actual tech company that talks much about Agile, and I will be astounded.

In my experience, people at tech companies (at least the FAANGs) rarely talk about Agile, although they do talk about things like continuous integration/testing/deployment. They do not obsess over methodologies for how to move post-it notes around a whiteboard (Scrum vs Kanban) or agonize over how to word a user story narrative, or other parts of Agile Theatre.

People at some non-tech companies, especially those supposedly going through a "digital transformation", seem to have fully bought in to the crap that Certified Agile consultants are selling though.

So a superior alternative to both Agile and waterfall is what's in use at a lot of the big tech companies. For example, the engineering culture at Google, which relies on design docs and a very good set of developer tools and infrastructure. It's not perfect, but it's far, far superior to Agile.

And before anyone makes the argument that those non-tech companies are not doing "real Agile", but Google is, let's be linguistic descriptivists and accept that Googlers don't call their methodology "Agile", whereas the Scrum consultants at big corporates do.

That said , plenty of tech companies have inconsistent practices that don’t necessarily lead to great success. To say Google’s methods are “far far superior” to Agile assumes they’re uniform, portable or relevant externally, or that they lead to above average success. That’s debatable given Google’s reputation for abandoned half-built products and an almost comical lack of customer focus.

It was utterly frustrating to be powerless to improve specification quality and find out you built exactly what was asked but you were asked for the wrong thing.

The silos were horrible. Devs were often as bad as BAs, dev’s just were just crapping on the testers instead. Spare a thought for the production ops person at the very end of the chain. Here comes 3 months of developer work in one weekend and no it doesnt work but we’re still going live because the entire tech org is invested in this release. Now you have smaller squads, you can postpone a release without it looking bad on the top person and thus affecting your career prospects.

In agile models you have the power to fix crap processes that don’t work. You can call out any BS on the retrospective and make it super visible when things are being swept under the rug.

I worked on the customer delivery side of a $750M software project that was a steaming pile of shit, with critical defects that were known for a year with zero effort to address. The integrators were paid to deliver a spec, and it took about a year to get change orders created to diverge from the spec.

The old school waterfall outsourcing models are truly awful, but you don’t really understand how bad unless you’ve lived it. Stupid agile religious practices are dumb for any startup type org, but are probably better than the alternatives in many big enterprise scenarios, where the goal is chunk up the work so marginally qualified people can do something.

The goal of Agile is to let smart people work incrementally towards a somewhat nebulous goal. The ceremony that's arisen around that tends to be put in place by people who feel the need to manage, but don't know how to help their developers achieve that goal.

The alternative, as far as I've seen, is to hire smart, curious people, let them work closely with the end user, and pay them a lot of money. In this situation, the engineers will typically self-organize effectively.