Hey all, i'm one of the team members of Nava, the public benefit corporation that emerged from the MPL team to continue to improve how the government serves its people...happy to answer any questions!
A lot of the most effective pushback we end up never seeing at all -- it happens in decisions and conversations we aren't party to. We do think in the long run delivering a radically better product will win out... But the presence of pushback does really demonstrate the need for tech-savvy partners inside the government, like those we have been lucky to work with at the Center for Medicare and Medicaid Services, and a reformed contracting process. The only way there will be better technology in government long term is if the people signing the checks are able to discern between the modern way and the old way, and insist on a better way of doing things. (Apologies for brief responses - on mobile)
The core of the issue was that EIDM, the old system, was an assembly of products designed for corporate intranets on the orders of 1000s of users -- it was never meant to be used for a consumer system. So all this super complicated permissions logic was bogging everything down.
They mitigated the effects for a while by running it on bigger computers - when we came in it was running on Oracle's BIGGEST computer, the Exadata. $6MM per environment. Terabytes of ram. But still just completely bogged down by a set of software not designed for its use case.
Thanks for this detail; it's vivid and upsetting. One assumes that VAR pricing was at play here as well: not only did the government get sold a pointless super-computer, but the contractor made points on that sale.
Exactly! The root of all this (I think) is the contracting structure for building jet fighters -- cost plus percentage -- was applied to software, which has such a different cost structure. With that structure there is no motivation to ship working software, or work efficiently at all. So people who value that won't work for any company that behaves like that, hence the shift of talent elsewhere.
God. I thought cost plus contracts were only used in exigent circumstances such as war, but apparently they are getting used more now, such as for the F-35.[1] Are software contracts really cost plus? These contracts shouldn’t be used for fighter jets or anything else. They should be banned except in critical situations and even then used sparingly. I remember stories about the boondoggle in Iraq, $20 coke cans and such.[2] Really sad if these are being used more now. Indicative of a lazy, corrupt government contracting process.
Na, this happens everywhere with software, startups too. The dev just wants to throw more resources at a problem than fix the complexity problems that make it inefficient. Perhaps it's a sign the dev is just out of their depth.
We basically rebuilt things from scratch on a node/MySQL/nginx stack while providing the same API as the old system, and were able to swap it out. We have become masters of the strangler pattern of software design...
We had the dubious fortune of coming in in a crisis situation, which let us insist on our way of doing things. (Bringing own computers, AWS, modern stack, quick hiring, etc)
Even then, though, launching our first product (App 2.0) to production was really uphill as we fought to earn the trust of CMS and the other contractors, who didn't really believe we could deliver.
But after we shipped, and people saw that we could deliver working software, things got way easier. Now we are consulted on major software architecture decisions and are a key part of the design process.
It goes the other way round though as well -- we have learned you have to respect your partners and champions on the inside, and conform a bit as well. When you are dealing with really sensitive personal info you can't play as fast and loose as you could in a brand new startup.
we are at a point now where we can often look at the underlying need and suggest alternate ways of approach. For instance after a big breach a few months ago security wanted us to use some VPN for all our prod systems- we managed to compromise on amazons MFA which is quite solid but gets us the enhanced security they were going for
Long term we want to help reform contracting so that requirements are not framed in this lots-of-boxes-to-check-none-having-to-do-with-serving-users mindset
We love 18f!!! We work a little more closely with USDS as our relationship with them is more complementary - they find opportunities in government and will often consult with us and other modern firms on how to approach big problems like identity management, identity verification, etc. we don't work directly with 18F but feel very much like we are part of the same movement towards reforming the ecosystem
