As a developer in my 40's, I see things like this and worry that the software industry will come to rely only on the young because they don't ... I don't want to say care ... but put as much emphasis maybe? ... on work-life balance.
Developement-wise, I can hang with the 20 something crowd, no problems. I just can't compete with the single/no kids thing.
I'm working on a contract tangentially related to this work and it's really not about young people working long hours. There are plenty of older people emailing me at 10 or 11 pm after the kids are in bed that don't know what's going on. This is solely about the quality of the talent.
I encounter people and teams that are just unwilling to adopt contemporary development and deployment practices. The article notes that Hipchat was a struggle to get approval for, yet I often run into people at CMS that never log in to it and prefer tons of emails. Deployments aren't automated and still happen for some teams on calls during maintenance periods once a week. I've had people in technical capacities ask what GitHub is.
The problem is institutional, it really has nothing to do with young people working 10 hour days. I care about my work-life balance -- I'm too old to crash on the couch at my startup's office like I did at past jobs in my 20s -- but I also keep up with contemporary development practices, make a point to study a new language every year, read academic papers, and care about my work. I don't think I can say the same about most devs I've encountered in government. The consulting firms are incentivized to build walls and protect the way they do business in order to keep getting that contract money.
If you're an older dev that is intellectually curious in the practice and art of software development I think you'll be fine.
You might be fine，but I'd be ready to make a lot less money as we move into a more freelance oriented market
the problem is that labor doesn't scale very well. 2 talented young programmers working all-out 12 hours a day on one project will generally be more productive than 3 talented folks with family working all-out 8 hours a day. (Agile, git, etc. all try to fix this to a degree... ) The current economic situation is that startups have very very deep pockets and will pay an insane premium to have stuff ASAP - which will mean very small, very overworked, very talented "rock stars".
If you want to fix healthcare.gov and the president's reputation is on the line, you'll get the rockstarest rockstars you can find and slave drive them till it's done
If you want a good work life balance I recommend the defense sector - but again - expect a pay cut. (like $120K end of career in SoCal vs. $250K in SF)
> The current economic situation is that startups have very very deep pockets and will pay an insane premium to have stuff ASAP - which will mean very small, very overworked, very talented "rock stars".
Another, more descriptive way to formulate this is: some startups are really fighting against time, because they are led by financials who only watch short-scale (10y) ROI, and those will pay a premium to have stuff ASAP, whatever that entails.
Of course, startups (or other orgs) that are not only driven by financials have to manage time and money as well. But they have other plans.
Look what/who you're working for. A sprint, a marathon and a trek are different journeys.
> 2 talented young programmers working all-out 12 hours a day on one project will generally be more productive than 3 talented folks with family working all-out 8 hours a day
There's actually a lot of research that says it's the opposite.[1] The big difference, I think, is that young developers, like the previous poster mentioned, are up-to-date on the latest technology and are willing to change things to work faster.
I've been dealing with 'experienced' Ops trying to maintain control of everything, while slowing down development. The number of hours spent coding doesn't change much if the way development is done is incredibly slow.
I'll give you that not everyone is productive at 12 hours - but I'm not talking about everyone or even averages. The point is there are plenty of counterexamples of people that can work 12 hours days for months and are way more productive than anyone working 8 (hence "rock stars"). Some people really really love programming and don't have a life outside of work. A simple example is the overzealous founder/cofounder programmer that works 12 hours a day. I've seen people do that and they are very very productive b/c their heart is in it. I imagine some people with lots of stock options and/or coolaid-drinking also are very productive. Some people I've met that work at SpaceX sound incredibly productive. They probably eventually burn out, but with turn over being so high in our industry as it is, your employer doesn't really care. You'll leave after a year or two anyways.
.. The link you put up is full of conjectures, but I didn't actually see any numbers... (it's a link to class? .. I don't even understand the website)
From my experience this isn't true. I make a lot more money now freelancing than I did working much longer hours at startups. Sure, there are promises of my options being worth millions, but the reality is that they're sitting in my desk gathering dust. I'd actually take a pay cut to go back into a startup setting once this contract is over, mainly because I often miss the level of intensity, creativity, and the feeling of ownership that comes with a startup. I prefer being the dumbest person in the room.
Driving young programmers through 12 hour days is just a recipe for burn out and technical debt that you'll have to pay back later. I've definitely been there.
I guess it all comes down to what you consider a good work-life balance. For me being able to work from wherever an Internet connection exists and the ability to make my own hours is it, and yeah sometimes that'll mean a few weeks of 10-12 hour days here and there. For some it means a strict 9-5 with an hour lunch at an office park off a highway, which I don't judge, but for me sounds like a nightmare.
Again, as long as you're keeping up with contemporary development practices and are pushing your knowledge I think you'll be able to choose whatever definition of work-life balance is for you and be fine.
I see this more as "the people who just show up" vs "the people who actually work when they're at work".
Dividing them up like this the typical "just shows up" person is purely stimulus driven. Nothing happens if you don't tell them to do exactly something. They won't tell you they're blocked on Bob sending a purchase order for a month. They won't tell you because you didn't ask. You didn't ask so they just... they just do nothing.
Or you have contractors who really don't care[1]. They bill by the hour, and if you don't give them tasks then they can take a second contract and bill more.
Most companies I've seen are full of stimulus-response people.
You don't need to work more than 40 hour weeks to be productive. What you need is managers and coworkers who understand what your job kinda entails. People who when you say during the two week planning meeting that you'll fix a typo and that's it, will call you on it saying "that won't take two weeks, what else?".
I've seen competent people (with children) fall into a comfort zone where they can keep up and "perform well" working just an hour or two a week, because nobody else gets any work done so it's not like they're blocking people. These same people actually perform in better companies.
[1] I'm not saying no contractors care. But they have extra reason not to care if they happened to be paid and aren't "stimulated" (in the simulus-response sense).
> Dividing them up like this the typical "just shows up" person is purely stimulus driven. Nothing happens if you don't tell them to do exactly something. They won't tell you they're blocked on Bob sending a purchase order for a month. They won't tell you because you didn't ask. You didn't ask so they just... they just do nothing.
I'm sorry but I'd argue that I cared more when I was younger (mostly because I was stupid). I've spent entire days in someone else's cubicle (not so much pair programming as me standing guard against distractions such as business analysts and quality assurance who apparently kept popping in) making sure they finish their part of work.
Actual code writing for me was fairly easy once the roadblocks got out of the way.
I've told people to shut up in meetings when they start going off in tangents. Clearly, it was not good for my career.
> I've told people to shut up in meetings when they start going off in tangents. Clearly, it was not good for my career.
It's an awkward situation, I usually just ask "is this relevant to everyone in this room?"
Sometimes a manager says "yes" (it's not) and I just sit there and die quietly inside, otherwise they just say "yeah let's take this offline". Whatever that means. As long as it's not fucking with the meeting ¯\_(ツ)_/¯
> I'm sorry but I'd argue that I cared more when I was younger (mostly because I was stupid). I've spent entire days in someone else's cubicle (not so much pair programming as me standing guard against distractions such as business analysts and quality assurance who apparently kept popping in) making sure they finish their part of work.
I think this is one of the main points. Some managers literally abuse the commitment a good worker has toward the company - for any possible reason: first job, belief in career advancement (within the same company), pure love for work (workaholic), etc. However, once you change a couple of companies, and once you see what managers really value, you start to wonder: what am I doing here? who am I kidding here? Your priorities start to shift toward something that becomes more important for you (family, kids, your own person, friends, and so on). My 2 cents. I might be completely wrong, so ignore me :)
In general, I feel that the "proactive" or "reactive" mindset is not always the main reason of such issues occurring at work. Sometimes the environment is so literally fucked up, that you can't do anything, even if you are super proactive. Really, nothing, even when you try to find yourself a task, something to do; and this will kill you inside, until you change job. I believe that good managers make you productive, even when you "literally" work 50% of the time. After all, that's what they are also paid for, isn't it?
I also want to believe that a good manager doesn't let that happen (that you work like hell), for the simple reason he is aware of the consequences.
However, I find it's hard to generalize, it always depends on the case/company/team/person.
Over a quarter of the top CEO's in the world have an engineering background and the median age is 59.
The world is becoming more and more flat with globalization. If you are limiting yourself to pushing code and not learning business skills you are probably replaceable.
I think decently smart people are all capable of learning business skills. The problem fundamentally later on is that you need skills and a network that is aligned with your goals for your post-individual contributor career.
I'm not entirely sure how much that is true... I walked out of a Director position in well under a year simply because it wasn't work that I was the best at... and by that, I mean that I would never be able to handle 2-3x the load of someone else in that position. It was rather constrained.
I'm far better off in a senior/lead/architect developer role over being in any form of management. I provide better value in that space. What's disappointing to me is that you pretty much cap out and aren't likely to make more anywhere near in proportion to the value you provide.
A huge problem that many smart people face (myself included) is dealing with "average" employees. The sort of person who sees software engineering as "just another job" similar to folding T-Shirts at the Gap or flipping burgers at McDonalds. Very, very few people actually care about the influence their work has on customers, investors, vendors, and other employees... so it's a struggle to mentally understand how self-absorbed wage-earners actually think. To the point of feeling like throwing temper tantrums in response to "normal" behavior.
I don't understand why you would care about the level of dedication of your co-workers to the job? I know for a fact that many programmers do treat their job as "just another job" and are not as consumed by technology as someone who is more passionate about it.
From personal experience, it becomes very clear rather quickly who the top team contributor really is. But having those kinds of expectations from other people is rather toxic. You would either have to change your attitude or your job.
Finding a job where you're a good fit isn't easy either. I'm somewhat temperamental; in a job where I'm uninterested, it's easy to be the worst performer. In an interesting job (for me, these are often what others consider difficult), it's easy to perform several times as well as most others and be the top performer by a wide margin. Admittedly, my ADHD may give me a higher spread than most people, but I imagine it affects everyone to some degree.
I'm a decade behind you but having my daughter was a huge wake up call. I was leading a machine learning side project with plenty of of 20-somethings. They were very eager and involved, until my daughter was born. Suddenly half the team quit. One guy admitted he just assumed the project was over as soon as I had a kid, like I was dead or something. I ended up coding a majority of the project by myself.
As a married person with no kids I feel the same way. I'm not 40, or even 30 yet, but I look at my peers and wonder if they'll grow to regret putting in 60 hour work weeks just to make someone else rich. I'm guilty of doing it too, for sure, but it wears you down quick.
Being in my 40s and having been there (in my 20s) I can tell you they will grow to regret it.
That said, its up to people like me running teams and businesses to set the tone - encourage balance while not squandering inspired work.
Re inspired work: I once made the mistake of telling one of my employees "slow down a little, i don't want you to burn out and hate working here".
He had an astute response: "Andy, I'm inspired I have to keep working on this until its done!"
He worked weird hours - night and day for 2 or 3 days ..and when it was done I remember him needing some well deserved R&R.
The message from there on out to my teams is - if you're inspired, work. When you're done, rest. I'm not watching when you roll in in the morning, I'm watching the team's accomplishment and morale.
I've been lucky enough to have managers with a similar approach most of the time. I cannot overestimate the positive impact that has had on my performance and professional growth.
You're the rockstar!
FWIW, it can be a bit more complicated than that. I spent my 20s working 60 hour weeks to (mostly) make someone else rich, but doing that helped to get me to a better place by my 30s. Having graduated from school immediately after the dot compost, I'm not sure I would have been able to start out in a good tech job from the get-go.
What worries me more is when I see colleagues who have children and are _still_ doing the chronic overtime thing. Missing out on getting drunk in the evenings just to make someone else rich is lame. Missing out on the early years of your kid's life just to make someone else rich is a tragedy.
It depends on what you define as "making someone else rich".
I think it is a good trade off to have someone else shoulder all the business risk while I cap out at ~$330k at a reasonable senior level(Staff SE). IMO, $330k is "rich". Most people could retire from 5-10 years of working after reaching the senior level. Once someone reaches the senior level, I assume that the knowledge gained while hustling to achieve the senior level would let him/her dial down to the standard 40 hrs.
The only way anyone is making that kind of money as an engineer is if they're working at a handful of large tech cos in the bay. $330k is far from realistic for 99% of tech employees.
I was watching a documentary on prodigy musicians one time. The person presenting the show was a prodigy himself and he asked some interesting questions. One was, "How much did you practice when you were young?" and the answer would invariably be 12-13 hours a day. He then asked, "Do you regret not spending that time doing other things?" The answer was also invariably "No".
The prodigies in the documentary explained that the time when they were practicing all day was exciting and fun. They looked back on that time with nostalgia because after they got older they neither had the time nor the stamina to practice like that again.
"Nobody ever died wishing that they had spent more time in the office" is a popular quote, but I think it is false. I personally know of several people who after they were diagnosed with terminal diseases chose to spend their remaining time working as much as possible.
The key to what you are saying is the "just to make someone else rich". I think if you are looking at it as some kind of equation where "I have to do X in order to get Y", often you lose out because it turns out that Y was not worth it after all (especially if you are trading your life for money). However, if you are looking at it as "I can't believe I get to do this all day long!", then I don't think you will regret it.
This is not to say that those who prefer a "work-life balance" existence over "work is inseparable from life" existence are much different. It's just that you need to spend your time and energy doing what you love -- getting that balance right. Both groups suffer if they divert their attention away from what is important to them.
For me, things changed ~18 months ago when I had my son and suddenly, I was less willing to travel, more willing to take breaks when he's awake, and I take ridiculously early morning flights so I'm there to tuck him in the night before. Those opportunities are fleeting.
I'm working on a project now that's all single 20-somethings. On the one hand, they are very smart, very aggressive and put a lot of time in on weekends.
On the other hand, I'm crafty, and experienced. Also, I think kids that age do respect the idea of a family life -- I just had a chat this morning and said "have to go, family event" -- it's okay.
While some 20 somethings are workaholics, a lot of them are not, they get their work life balance in at other times in other ways, more periodically. I don't think there needs to be a worry about stigma. Different strokes, and good communication are the most important.
My experience is similar; it's sad, really, because though I enjoy what I do and am often among the most productive, my unwillingness to forgo time with my family in order to work unreasonable hours, often a result of poor leadership decisions, has become somewhat of a liability. After a career of small companies and the occasional startup, I am eyeing the cubicle-and-collars world of B2B software development with more interest.
Well, I work way more than I should, because I have no other occupation outside of work. That's all. I'm not seeing myself competing with more experienced people for one thing or another. I'll take care of my work-life balance when I'll have a life, but in the meantime I'm happy to have something to occupy my brain and people to talk to 8h per day, otherwise I would definitively go crazy.
I find this quite sad. There are a LOT of worthwhile organizations and purposes that need your time - badly - and would add needed balance to your life.
I volunteer with the American Red Cross which needs volunteer personnel for all kinds of purposes ranging from two week deployments to disaster areas to occasional help at blood drives. You'd develop all kinds of new skills and meet a lot of great people, all while having a very tangible impact on the world. And that's just one random example from what I personally spend my time doing.
Look around your area, community, and industry, and find more things to do. If nothing else you'll get a lot of good stories to tell. You might just find your purpose on life.
I'm a twenty-something and I see this everyday in my peer group. Most twenty-something's have way too much of their identity and self-esteem tied to their career. I would encourage other people my age to go in at 9, leave 5, work at a comfortable pace and take an hour for lunch. If you're good at your job and your employer is well run, that shouldn't be a problem.
The youth helped us mostly because we were able to hop up and live in that house for months in order to be close to CMS - know our customers! But even for us those days are over - we have offices in SF and DC and have multiple more established folks on our team. Just like major tech companies, we want to provide a comfortable environment for skilled engineers of any age and family situation.
As a fellow developer in my 40s, I enjoy getting as much work done in two hours as more junior people get done all week. Of course this isn't a function of age but two decades in the business doesn't hurt.
Then you are proving my point (other comment), and calling the grandparent comment either completely incompetent or a lazy bum who's upset that he can't keep up using 2h weeks when compete with 20-somethings who actually work 40h weeks.
He "worries about the software industry", but my point is that he is worried for the wrong reason.
The problem isn't that you can't do the job with a work life balance. It's that it's so full of incompetent people that someone competent can keep up with no effort at all. And they mask the incompetence. And eventually if you do that for 20 years, you too risk becoming incompetent. Or at least only have a tiny fraction of your potention competence.
I think it takes practice and discipline to really enjoy your life _outside_ of work. I take my kids to school, eat breakfast with my wife, handle my personal business in the morning. Then I go to work, WORK WHEN I AM AT WORK, and go pick up my kids from school and go home.
Working when I am at work being the key point. If I walk around the floor at my office half the people will be reading Facebook or whatever, which I completely understand and I myself do read Facebook from time to time but I don't understand why you'd want to do that while sitting at your desk at work.
Someone once made this remark to me when I was in an undergrad architecture program (not the CS kind). He said "the biggest difference you'll see between grad students and undergrads is that undergrads will live in studio but waste most of their time; graduate students come in, work, get their work done, and leave."
As a developer in my thirties, I see this more as flexibility and exposure to adopt new technology, instead of sticking with old tech stacks.
It's clear that the earlier contractors didn't know enough about user-centered design, responsive sites, mobile design, online login systems, modern web scalability requirements, benefits of cloud hosting and much more; which is where the younger developers now start.
Also a minute to thank Stackoverflow, for the amazing work it has done in uplifting software development work.
I think that's the case right now with many jobs in mgmt consulting / investment banking. You can end up with a negative situation where people devote every waking hour then feel all subsequent success is owed to them - not great.
> they don't ... I don't want to say care ... but put as much emphasis maybe? ... on work-life balance.
or they just have a different view of it.
You have no need to be home at any particular hour, and your social life is more evening/night based then why not work 10 hours a day. It's dumb, since you risk burning out, but otherwise I'm not really missing out on anything.
The entire culture has to change. Had a female programmer who works in Washington D.C. tell me a few years back the unwritten rule is that unless you're at one of the top rungs of the GS federal pay scale you aren't supposed to talk at all.
She had a boss who met with developers and told them nobody cares about your opinions. You see million dollar mistakes being made and you either accept it or leave government which is what she did.
And this is miniscule. The wars in Iraq and Afghanistan cost hundreds of billions more than they needed to (not talking about whether they should have been started, just the effects achieved for the cost). I flew F/A-18s off the carrier for 6 hour missions that could have been handled by a small propeller plane in country. These missions required a carrier (millions to operate), extra more-costly flight hours ($10k/hour for F/A-18s), and 3 air-to-air refuels (cost of tanker flight time and gas). Not to mention the deterioration of incredibly expensive assets that need to be replaced. I also saw artillery and tanks in country that had no use. Could go on about the waste.
The sad truth is government bureaucracies spend all their budgets so they can ask for more. Limiting cost is the least of their concerns. I applaud 18F and the Digital Service for their efforts to advance and streamline IT and development costs. It’s an uphill battle. There is no reality check in the public sector as there is in the private sector, where inefficient behemoths eventually go bankrupt to more efficient companies.
Outsourcing is supposed to prevent this, but the big contractors just milk it for all it’s worth (Oracle failing to build Oregon’s healthcare website for $250 million is an example, HP and NMCI is another). I don’t believe it’s just the byzantine contract bidding process but the fact it is easier for leaders to say they chose a big “reputable” company so it’s not their fault it failed, whereas if they had chosen a small no-name company and it failed it would be their ass. So real competition and efficiency end up being non-existent in government, unless the president’s legacy initiative is in jeopardy and he throws his weight behind the fix. I don’t know of any way to fix this behavior except to limit the size of government and make it more local so accountability is easier to maintain by the electorate. I wish more good people in government could change things, but there are already a lot of good people trying their best in a top-down system where CYA and optics are more important than efficiency and results.
Government and non-profits alike: people shoot for mediocrity and avoid taking risks. This is how you avoid getting fired. This is what happens when incentives & accountability are misaligned with your organization's supposed goals.
There's also a culture of not recognizing or admitting failure; they're spun into success stories. This means people don't learn from mistakes and keep repeating them.
Smaller teams are weeded out very early. The barriers to entry for working with government and non-profits are high, demanding all kind of certifications and processes to be implemented.
If you don't have teams to build excruciatingly detailed (and inaccurate) proposals (complete with 100 slides presentations), jump through the hoops of procurement and fill up the deluge of paper associated with tracking every single hour working on the contract, you won't land the big contracts.
Personally, I threw the towel in. Enterprise pays more, is more loyal and while procurement can still be a PITA, the overall organization is clearly aligned behind growth and revenue. If you help them do that, you can circumvent or negotiate things.
I too experienced several million dollar efforts that went nowhere. Governments in general are not geared to think about returns on investment, but our current one is quite terrible.
And I'll say, outside of the military, the public sector does not attract competence. There are exceptions, certainly. But the pay and the level of bullshit tends to attract those who either want to play politics, or who want a secure paycheck and benefits with little accountability, or just can't really perform at the level most private sector work demands.
I think government, in general, is built to be slow and deliberate, but as the pace of tech change has accelerated, the responsiveness of government hasn't kept up. Git was invested in 2005, for example.
Yeah, I have to think that the only reason this story exists is because there was a very public humiliation for the Administration, and that the original project is business as usual 99% of the time. I highly doubt billions of gov't contracts are now being rerouted to "young people," and in fact (on a parallel track), the last 18F headline I saw was that they came out with a to-do list.
Changing the government contracting process is going to be the slowest part of this -- yep nothing drastic is different now that a few years ago. But the winds of change are in the air. The fine folks at USDS are working hard to fix this from the root. There was a recent California child welfare software project that was broken up into smaller pieces and served as a prototype of a contract that was made to appeal to modern firms, and reward sound development practices. Expect more of this to shift the playing field and lay the groundwork for competition to do the rest.
Feature creep, "technical managers" who promised close-to-impossible functionality in totally impossible time frames. So many teams developing in isolation on loosely defined specs so nothing worked as it was assumed then when it was "fixed" it broke it for another team, etc. Higher ups making decisions on what stack to use based on who bought them nicest lunch or some other "bribe" so nothing worked together without a shit load of extra work, then obviously the price went up when they needed to purchase more for phase++ as none of the higher ups had any kind of negotiating or technical architecture experience.
Essentially it came down to too many people in too many isolated teams who were led by people that would struggle to do IT for a small Windows-only business.
At the time I used to say that the people running the project would get nine women pregnant in the hope of having a baby in a month. I would hope the problem with that is obvious but you would be shocked at how many people thought it was actually a good idea!
I dealt with this in the military, and it left me with a completely fatalistic look on work in general.
Welp, you're signing my paychecks. I will do whatever you want me to do for 40 hours per week. If that's productive work that makes a difference, awesome. If that involves me wasting a gazillion dollars on something stupid, I'm not paid to think, man. I just work here. If you want me to think, then pay me to think.
On the bright side, I have absolutely zero stress when it comes to workplace drama because I couldn't care less to begin with.
Nope, because I have passion for things outside of work. Hiking, lifting, spending time with my girlfriend, programming, playing Kerbal Space Program, etc.
For me, work is something that I neither dread nor enjoy. It simply is.
Since I find non-descriptive headlines like this annoying (edit: looks like it's been changed on HN to be much better), here is a quote from the article that surprisingly provides its own TL;DR:
>Here is the tl;dr version of their story: Marketplace Lite, or “MPL” as they came to be known, devoted months to rewriting Healthcare.gov functions in full, working as a startup within the government and replacing contractor-made apps with ones costing one-fiftieth of the price. And when, nearly a year after the initial launch of Healthcare.gov, the website’s second open-enrollment proved much healthier than its first, it was the MPL team who celebrated.
And perhaps the biggest takeaway of the whole article, buried in the middle of it:
> Instead, if the successes of the MPL team confirm any guiding principle for the future, it is this: Technical workers—not only engineers but designers—have to be involved with a process from the beginning. They will know that features must be described separately from needs, and that, when building software, smaller teams often perform better than larger teams.
Speaking as someone who worked in the past for a government contractor (DoD, so these issues may not be entirely the same with the feds, but I bet they're very similar), the issue isn't solely that bureaucrats don't get tech, it's also that the bureaucratic red tape that evolved to try and keep costs down has instead led to them rising, -especially- with software.
From what I saw, government contracting requires all requirements up front. It then floats a bid, and the cheapest bidder wins. Whether or not the contract has been met is determined by that list of requirements; even if you fail to deliver on a non-functional requirement that is obvious ("the system should be able to scale out and handle X simultaneous users"), you aren't penalized if it wasn't one of the requirements. In fact, delivering on obvious things that aren't part of the contract -hurts the contractor-; it requires development time and effort (and thus eats into profits), and the government is legally prevented from taking such things into account on future contracts (that is, they can't say "Well, company A is the cheapest, but the last contract they took was a broken piece of shit; company B is a little pricier, but they delivered quality, we'll go with company B". No, you go with the cheapest, period, because if they fail to deliver what you specify you can sue, and if they fail to deliver what you -wanted-, but failed to specify, too bad). It's possible to renegotiate the contract mid-stream, but it's expensive and time consuming.
So the contractor has every incentive to make sure that there is no time spent on quality, security, stability, scalability, etc, unless that's -explicitly- what was asked for. This article points out that having developers involved from the get go on the Healthare.gov rework led to it being successful, and that's very much true; you need technical people who care about more than just making bank to point out non-functional requirements that are missing. And, in general, trying to list all requirements up front is a fool's game, but such is the government's bidding process.
I was a little too quick on the upvote for your comment. Your first paragraph is spot-on, but your second is not.
> the government is legally prevented from taking such things into account on future contracts (that is, they can't say "Well, company A is the cheapest, but the last contract they took was a broken piece of shit; company B is a little pricier, but they delivered quality, we'll go with company B". No, you go with the cheapest, period
This is absolutely wrong. There is no requirement that the federal government must go with the cheapest bidder or that they cannot take performance into account. The government can and does regularly reject bids because they are unrealistically low; in fact, I was on one DOD program where both bidders were told to go home and come back with something realistic. Past performance definitely factors in to what the government considers reasonable. It's one of the reasons the government requires labor tracking even in situations where the labor expended doesn't affect the final cost, such as fixed-price contracts and uncompensated overtime.
That's some comfort then. I never saw any indication of it; across multiple projects I never saw any drive to refactor, or any of the signs I associate with developers taking pride in one's work (documentation above what is asked for, root cause analysis even when a surface issue has been taken care of, or really, any addressing of non-functional requirements that were obvious, but which had not been demanded in the contract). Perhaps because they were mostly long running projects that had already been bidden on, and just having the domain knowledge meant that they'd continue to have the advantage on any future bids to extend it.
Also, while I've seen bid wins/losses talked about from the perspective of the quality of the solution, I've not seen them talked about having been won/loss due to past performance. I hope you're right that that's a consideration, but the lack of controls around quality, and the obvious apathy I saw, lends me to think that if that's a consideration, it's not one the government knows how to actually judge.
>From what I saw, government contracting requires all requirements up front.
Strictly speaking the government has loads of flexibility at defining exactly what form contracts should take. It's just that in reality, most people who decide on the form of government contracts rarely have the skills required to imagine doing things better. Furthermore they have very little incentive to do so. So the least risky and the simplest option is to "do what didn't work last time". After all, everyone involved (on the government side) will continue receiving paychecks no matter what, so why bother?
Also, if you think about it - if you are a median government worker who lacks the skills required to actually perform the task for which you are writing/monitoring the contract - what is the probability that you will do a good job getting results on the taxpayer money you are spending?
In the big picture, the taxpayers must employ the most skilled and talented people available to imagine/design/oversee technical solutions acquired by the government if they want any to see value delivered for outgoing tax dollars. This constraint is completely incompatible with the existing civil service personnel policies.
To add one more thing, the red tape itself adds non-negligible costs. I once worked for a sovereign native nation, and the community manager was explaining that the amount of money that he got from the Feds for paving a road (IIRC maxed at $50K) was mostly eaten up in labor expended to cover their reporting requirements.
Hey all, i'm one of the team members of Nava, the public benefit corporation that emerged from the MPL team to continue to improve how the government serves its people...happy to answer any questions!
A lot of the most effective pushback we end up never seeing at all -- it happens in decisions and conversations we aren't party to. We do think in the long run delivering a radically better product will win out... But the presence of pushback does really demonstrate the need for tech-savvy partners inside the government, like those we have been lucky to work with at the Center for Medicare and Medicaid Services, and a reformed contracting process. The only way there will be better technology in government long term is if the people signing the checks are able to discern between the modern way and the old way, and insist on a better way of doing things. (Apologies for brief responses - on mobile)
The core of the issue was that EIDM, the old system, was an assembly of products designed for corporate intranets on the orders of 1000s of users -- it was never meant to be used for a consumer system. So all this super complicated permissions logic was bogging everything down.
They mitigated the effects for a while by running it on bigger computers - when we came in it was running on Oracle's BIGGEST computer, the Exadata. $6MM per environment. Terabytes of ram. But still just completely bogged down by a set of software not designed for its use case.
Thanks for this detail; it's vivid and upsetting. One assumes that VAR pricing was at play here as well: not only did the government get sold a pointless super-computer, but the contractor made points on that sale.
Exactly! The root of all this (I think) is the contracting structure for building jet fighters -- cost plus percentage -- was applied to software, which has such a different cost structure. With that structure there is no motivation to ship working software, or work efficiently at all. So people who value that won't work for any company that behaves like that, hence the shift of talent elsewhere.
God. I thought cost plus contracts were only used in exigent circumstances such as war, but apparently they are getting used more now, such as for the F-35.[1] Are software contracts really cost plus? These contracts shouldn’t be used for fighter jets or anything else. They should be banned except in critical situations and even then used sparingly. I remember stories about the boondoggle in Iraq, $20 coke cans and such.[2] Really sad if these are being used more now. Indicative of a lazy, corrupt government contracting process.
Na, this happens everywhere with software, startups too. The dev just wants to throw more resources at a problem than fix the complexity problems that make it inefficient. Perhaps it's a sign the dev is just out of their depth.
We basically rebuilt things from scratch on a node/MySQL/nginx stack while providing the same API as the old system, and were able to swap it out. We have become masters of the strangler pattern of software design...
We had the dubious fortune of coming in in a crisis situation, which let us insist on our way of doing things. (Bringing own computers, AWS, modern stack, quick hiring, etc)
Even then, though, launching our first product (App 2.0) to production was really uphill as we fought to earn the trust of CMS and the other contractors, who didn't really believe we could deliver.
But after we shipped, and people saw that we could deliver working software, things got way easier. Now we are consulted on major software architecture decisions and are a key part of the design process.
It goes the other way round though as well -- we have learned you have to respect your partners and champions on the inside, and conform a bit as well. When you are dealing with really sensitive personal info you can't play as fast and loose as you could in a brand new startup.
we are at a point now where we can often look at the underlying need and suggest alternate ways of approach. For instance after a big breach a few months ago security wanted us to use some VPN for all our prod systems- we managed to compromise on amazons MFA which is quite solid but gets us the enhanced security they were going for
Long term we want to help reform contracting so that requirements are not framed in this lots-of-boxes-to-check-none-having-to-do-with-serving-users mindset
We love 18f!!! We work a little more closely with USDS as our relationship with them is more complementary - they find opportunities in government and will often consult with us and other modern firms on how to approach big problems like identity management, identity verification, etc. we don't work directly with 18F but feel very much like we are part of the same movement towards reforming the ecosystem
The sad thing about this is that said young people didn't get paid anywhere near the $250-500 million paid to the contractors and cronies[1] who built the broken healthcare.gov in the first place.
This is a hard problem to solve perfectly. I'm still stuck at the ID verification step, after sending in my documents. Never heard back. I'm the perfect corner case though, having gotten a green card recently, left a job, moved to another state, so on so forth...
Same, I was never able to get past this point, apparently because I don't exist in Experian's system due to having no credit history. Attempts to proceed beyond have never been successful the few times I've tried since, so I've given up.
With words that have meanings? To the layperson, "data center" and "supercomputers" translates to "tech stuff".
"AWS is a service by Amazon that runs software for you in computers connected to the internet" is still inaccurate, but has a meaning for the layperson other than "tech buzzwords with superlatives".
It's reasonable to expect the typical Atlantic reader to know what a supercomputer is and to ascertain what a data center is in this context. Painting with a broad brush that's what AWS is.
AWS is a computing service built on "datacenters" located all around the world, which are large warehouses packed tightly with racks of computers that are networked together and efficiently powered and cooled.
I'm missing a description of virtual machines / containers but that might be going into too much detail.
> A supercomputer is a computer with great speed and memory.
AWS's machines have more memory and CPUs that you would find in personal computers, with gigabit networking.
Granted, AWS does a lot of other things: split these machines into smaller VMs, allow users to bid on prices, offer higher-level services like queues and distributed storage.
If this is the greatest technical inaccuracy, they're doing pretty good.
It's a disgrace that no one is held responsible for healthcare.gov. We get article after article about "Silicon Valley" and their noble role in this shit show, yet not a single article about the criminally idiotic and corrupt politicians and their contractors.
"The MPL team was not the ideal workforce: They were (and remain) contractors to contractors. They were not protected by a union, nor did they enjoy the many benefits of working as public employees. They are coders-for-hire who could relocate across the country quickly, and they reflect the larger industry they work in: Mostly young, mostly male, and highly educated. It does not have to be this way. This kind of precarious employment—lucrative, quasi-nomadic—is as much the result of poor planning as a natural consequence of writing code for a living."
Why are any of those things not ideal? What's wrong with being contractors to contractors,non-unionized, young, male or educated? Or am I misunderstanding the author's point?
I think the focus was on the temporary employment part. The government does not need fly-by-night developers who come in when things are completely broken. they need a strong technical core that builds things well and with low costs the first time.
How much would they have made if they were unionized workers who got OT for the countless hours? Would they even have been asked to? Could they have gotten a proper office if they weren't non-union contractor to contractor workers?
What sort of benefits do they get? You're working yourself at an insane pace, do you get health benefits and some guarantee of time off at some point?
> young, male or educated
At the exclusion of older, female, and minority workers.
Todd Park, the ex-CTO of the United States, started to process of pulling in Silicon Valley talent. We were called "Todd's kids" for the longest time :)
Todd reached out to the CEO of Civis Analytis (Dan Wagner I believe?), who had pitched in with the Obama campaign. Through him Todd reached Mikey Dickerson, who was at Google at the time (Mikey is now the head of USDS). Mikey looped in a few other googlers...and so it spread.
I found the MPL team through friends in the startup world, many of whom were ex YC founders
It's pretty wild to think they had to go to the opposite coast to source a node/mysql/nginx solution. I know people who were doing such things much closer to DC. Granted, we are lower profile and maybe less pre-certified as "sure things". I wonder if borrowing Googlers is the government tech fire fighting equivalent of "nobody ever got fired for choosing IBM"?
The site can only be considered a success compared to what it replaced. It is still slow, the workflow is confusing and this past enrollment period actually being able to login was still an issue.
the original healthcare.gov was made using SCRUM. SCRUM fails by design on projects involving more than one team with dependencies between them, and the result in such cases is usually complete devastation like the original healthcare.gov. There is no surprise that the [small] "team of young people" were able to incrementally build a [simple] webapp and an app using SCRUM - that is the sweetspot of SCRUM, pretty much the only situation where it doesn't screws things.
Interesting, does anybody know how or if this relates to 18F? Is there a correlation? I ask because there is currently another post on HN today about 18F.
node/MySQL/nginx for both app2 and SLS. both are on AWS -- the first federal govt systems to use them. Even with the urgency, needed to wait almost a year for them to be approved, mostly around security concerns.
Given that AWS is HIPAA compliant and you can use most database products for PHI (https://aws.amazon.com/compliance/hipaa-compliance/), the likelihood is that you're going to see both public and private healthcare information moving into AWS (or Azure or GCE), if its not already there.
But what about Amazon makes you not want healthcare organizations to use it?
If I ran servers which processed temporarily-plaintext health care information, and I couldn't snoop on it even with effort, I'd be ashamed of my poor skill at villainy. Encryption is not all-powerful.
I saw Mikey Dickerson talk about his experience in the initial recovery phase of healthcare.gov at Velocity in 2014. I think it shows just how insane the situation was.
101 comments into discussion and no one mentioned CGI? the actual morons that fucked up the project and wasted 100's of millions of taxpayers' money? how many times should they piss in our face till we actually start noticing things?
I think a key factor for our success was in being a small team, using a modern technology stack, and building in an agile way. I think the article overemphasizes the youth of the team. While many of us were quite young, our team included individuals in their 40s and 50s. I think the skew towards youth is more a byproduct of the risk profile of the project, and the need for us to spend significant time in Baltimore/DC.
Developement-wise, I can hang with the 20 something crowd, no problems. I just can't compete with the single/no kids thing.