This article is wildly off base, particularly when you look at the examples the article cites.
WhatsApp was able to scale to so many users while keeping the engineering team small precisely because they hired great engineers who wrote an extremely efficient system in Erlang. (Does he think commodity coders know Erlang? Really!?) They heavily tuned their architecture and built efficient paradigms which kept developer productivity & performance from degrading as their user base grew. [1] Commodity engineers don't write that sort of code.
Yes, we increasingly have tools which allow mediocre coders to get stuff done. (For example, Ruby on Rails.) But the inevitable byproduct of letting mediocre coders loose on a stack they only partially understand is a mess of spaghetti code which takes even more developers to maintain. This is already what you see with non-tech corporations who employ dozens/hundreds of developers to maintain Drupal/RoR/Java apps which are significantly simpler in scale & complexity than WhatsApp or Instagram. If you want to use tools well, you need to hire people who understand the tools.
Yes, the tooling has improved dramatically. A 10x engineer of yesteryear might actually be a 100x engineer today. But if you give a bad engineer modern tools, it just allows them to fail faster.
I might be able to build a house with legos, but I expect to be spending a lot more time repairing it than if I'd just hired a carpenter in the first place.
>WhatsApp was able to scale to so many users while keeping the engineering team small precisely because they hired great engineers who wrote an extremely efficient system in Erlang. They heavily tuned their architecture and build efficient paradigms which kept developer productivity & performance from degrading as their user base grew. [1] Commodity engineers don't write that sort of code.
The problem is they did it with only a few (16?) "great engineers". That's the premise of the article -- our world today might need 100,000 great engineers to solve great problems. Tomorrow 1000 might be enough.
As for the "near great" they would too be less needed, and commodity engineers would be the equivalent of burger flipers.
>Yes, the tooling has improved dramatically. A 10x engineer of yesteryear might actually be a 100x engineer today. But if you give a bad engineer modern tools, it just allow.
The first yes, the second not really. A bad engineer can get enormously more done with modern tools than he could 10 or 30 years ago. Essentially a bad engineer today can build stuff that took high level senior engineers to build -- and do it in less time too. It might be a little crude, but it will work.
E.g a bad engineer can automate a workflow that would have taken expert engineers, custom software and several man-years to do 20 years ago, and can do it in a day.
And because it's so abstracted, building something like that needs little influence from his skills, so his "badness" doesn't show up much.
> The problem is they did it with only a few (16?) "great engineers". That's the premise of the article -- our world today might need 100,000 great engineers to solve great problems. Tomorrow 1000 might be enough.
The premise of the article seems to be that we no longer need great engineers at all. Or that they only need to work at infrastructure companies (not application companies like WhatsApp). The very example he chose shows that's empirically untrue.
The fact that a small team of amazing engineers can accomplish huge things (given some ops support) is not new. The original Macintosh team was also tiny.
There's no evidence that we will ever stop needing great engineers, particularly as more and more applications require huge scale while maintaining agility.
>The premise of the article seems to be that we no longer need great engineers at all.
Well, not really, because somebody will have to build all those elaborate abstraction layers people will use.
The core idea is that we'll need much fewer great engineers -- which is another way of saying we won't need "great engineers" at all. Read "not need them at all" liberally (very few), not literally (nobody).
After all if job demand goes from 1,000,000 people to 1000 people (arbitrary numbers), it's like it's the end of that job, even if technically people are still employed (there are after all horse cabbies and film lab technicians and letterpress printers employed still today).
This is absurd. How many people where creating their own languages 20 years ago? Everyone and their grand ma is building new languages on top of javascript. The expertise level of software engineers is going up! 20 years ago, how many people could build a VM from the ground up? Have you visited github lately? Sure, an average engineer can do much more today, and the reason is because of these great engineers, and they are not fewer by any means.
>How many people where creating their own languages 20 years ago?
20 years ago? Far less. That means writing a language is getting more commoditized -- and we work in a higher, easier, level of abstraction. That's what TFA says about other spheres of development and deployment too.
>Sure, an average engineer can do much more today, and the reason is because of these great engineers, and they are not fewer by any means.
Neither I, nor the article, said there are fewer great engineers. Only that fewer great engineers are (and fewer still will be) needed. Both as a percentage of all great engineers, and in absolute numbers.
I can't substantiate it, but there were in fact a lot of languages being created through the 60's to 90's. One could argue the language implementors of old were quiet daring, push boundaries much more than today (see APL and ICON as examples). sed and awk arose in an era of language explosion, make, sql, the list goes on.
> Well, not really, because somebody will have to build all those elaborate abstraction layers people will use.
It's almost as though you completely ignored my next sentence, which clarified that great engineers continue to be valued outside of abstraction/infrastructure companies.
Well, I don't find it valid to be frank. They might be valued, but they are valued less and less, and will be valued even less in the future.
An enterprise that used to need 100 engineers, with the cloud, SaaS, PaaS etc, can even today make do with 1/5 of that. Is there any trend showing companies needing MORE software engineers?
I would say yes. More and more stuff is getting done by computer. The majority of IT staff are still building / maintaining in house applications as far as I am aware, not building teh next Facebook / Whatsapp / etc.
The problem with the article's conclusion (and title) is that they don't stop with 100,000 -> 1,000 engineers.
"Tomorrow, that billion-dollar startup acquisition might not need an engineer at all."
The article's claim is not that startups are going to keep having smaller and smaller teams of higher-value engineers, they're saying that they're going to have zero engineers. Which I don't see at all from the evidence provided.
Well, they might technically have an engineer, but it could be some average joe, using a "21st century" equivalent of Hypercard -- building a whole startup with some composable cloud components and some logic.
After all, the basic skills for a successful startup, even today, are not hardcore engineering skills, but idea, business, finances, etc.
I mean, most startups bought for multi billion dollars work on BS trivial engineering problems (mostly: make a social web service scalable), compared to what talented engineers do elsewhere (in research, in space exploration, in robotics, in DSP, etc).
Heck, wasn't Facebook started with crappy PHP code a 15 year old could write (and a 20 year old did wrote)? Or, for a smaller scale example, Derek Shivers (which I admire) sold his CDBaby startup for a nice sum, and that was also started and based on amateurish code at best that he wrote himself.
It's a problem if you are an engineer. It's the kind of "going homeless" fuckery.
It's also a problem, longer term, if you are an entrepreneur. Initially it starts as "lower costs", but with an impoverished middle class, you soon find there are not that many people to buy your fancy products and services.
Automation is only good is it's combined with a scheme for the surplus of labor -- e.g less hours work week, creation of new kind of jobs, etc. Without that is a huge creator of poverty and inequality, and eventually the collapse of a market economy.
If I remember correctly from a high scalability post about WhatsApp it was also on bare metal. So more complexity to manage then a simple AWS setup where you can tear up/down in seconds.
Whatsapp also makes design decisions that I would call 'engineer-easy'. Their iOS app is pretty much all default system widgets and behavior for example, which is significantly less code to maintain than something that was customized heavily.
On the server side they don't retain conversations, have multi-session usage (many clients, one user), etc which simplifies what is required.
I haven't used whatsapp in a while so things might of changed, but before they were acquired by Facebook, that was definitely their direction.
> As software becomes a high-impact, low-skill trade, we decouple the technical ability and experience needed to write tricky software from the ability to solve problems for people.
It's one thing to know that capital secretly delights over the commoditization of labor. It's quite another to watch a16z gush so openly about a future where we can be tossed aside.
It seems this claim has been made forever. COBOL was supposed to make programmers unnecessary. So were rules engines. And code generators and an endless list of other technologies.
I agree that Gerstenzang is premature. But it'll happen eventually. If it doesn't happen by making programming easy it will happen by flooding the industry with enough competent programmers to have the same effect (depressed wages, high unemployment, etc). See: non-software engineering occupations (excepting doctors, who have managed to turn the risks inherent in their profession into a legally mandated union).
Demand for engineers is increasing much more rapidly than non-software engineering occupations. Soon every company will need an software engineer of some sort.
Of course there's an underlying BS premise that "in that wonderful future humanity would be freed from the burden of non-essential programming work, and would move on to higher level problem solving" -- but in real life they could not care less if humanity actually moves forward or just rich companies have to pay less while millions suffer and fall from middle class.
I cannot believe that this post by Sam Gerstanzang, a partner, speaks for a16z. I certainly hope it doesn't get much traction internally and there have been a few internal notes explaining to him the number of ways he is wrong.
What I don't understand is how someone who is so obviously wrong about the software craft and isn't even coherent within a short blog post can be a partner at one of the best VC firms. I sincerely hope he's there because of nepotism.
Great programmers move up the chain as well. If your skills are not being valued by your company as it is setup in a way that it needs only average engineers, you can move on and create your own company now.
If you are a great engineer, then you are a walking capital by yourself.
If somebody with an idea, and no skills, hires you, then it probably needs at least 300-500k (even for a smallish prototype) to make it happen. If you are a full stack engineer, and pair up with a buddy, you can make it happen, with a lot less. If you are a great engineer, you are the capital!
Programming talent and skills do not translated into business talent and skills. And vice versa. It does not even translate into an idea doable by just two guys that can sell well and is not easily doable by other two guys. How many such products does exist?
> If you are a great engineer, you are the capital!
It only seems that way because engineering is currently the most capital-heavy part of bringing a softwarebusiness to market. Once that ceases to be the case (which may take a while, but it's the future SG was gushing about) then as long as there is a single scarce or mutually-exclusive commodity necessary for success its price will ratchet upwards until it is out of reach for most engineers (after risk-adjustment).
Pertinent example: the implosion of the indie app dev market that's been all over HN this past week.
But once the Engineering component has been hammered down as much as server costs were in the previous generation ('Sun pizza boxes with Oracle' turned into AWS on OSS), what's the next largest expense line?
If the next most expensive thing is the marketing spend, so that the company gets the required exposure in overcrowded app stores (for example), then the VCs are painting themselves into a Hollywood hit role - at best.
Troubleshooting and problem-solving are specific skills
that can be completely unrelated to software, even tricky
software.
It's great when both problem-solving and outstanding engineering exist within the same individual, in some positions even essential.
Realistically, problem-solving is the more uncommon natural ability of the two. If it were not, there would be a lot less problems with software by now.
It seems to me that with crowd-funding and big data, the people who have jobs picking winning companies out from a lineup will be obsolete before the engineers building the crowd-funding and big data platforms will be.
> the people who have jobs picking winning companies
Remember, capital has TWO strategies for earning money:
1. Investing in new ideas, hoping they yield a positive ROI
2. Collecting dividends on ideas that have already panned out
#1 might conceivably be replaced by algorithms, but I doubt it (has HFT eaten Wall St?). #2 is inherent in our economic system and will never become obsolete.
What's wrong with a future where some software devs can be tossed aside? We have been excited to disrupt all other areas of labor. Disruption is good, it means progress, right?
In theory, yes. In practice, most people aren't entrepreneurs. Try convincing someone living in the Bay Area to walk from their 200K/yr plus stock units from FB/Google/etc. and you'll find out the reality: what really separates capital from labor is risk tolerance.
Labor = show up, do your job, get paid. Capital = put money at risk without any guarantee of getting anything back, let alone earning a profit. Capital makes the rules because half of America is two paychecks away from bankruptcy (I read this somewhere credible) and can't/won't take any risk.
People talk as if everyone can become a creator, which it ain't so (partly because of the personal skills needed, partly because there's no economy where "everybody is a creator" -- for the same reason why not everybody can be a millionaire at the same time (money loses it's value then). Any large scale economy needs few sellers/creators and many buyers).
So what this trend does, it empower a few programmers/creators (ok, more than the past few large capital owners), and makes all the other programmers redundant or devalued.
I'm failing to follow the logic here. The initial point is that "The potential impact of the lone software engineer is soaring". However, the author then takes that to mean that engineers will be less valuable in the future?
This analogy seems apt to me, and I think it makes the logical leap more obvious:
Fifty years ago, if you wanted to design a new model of car, you had to get plywood and clay, and work hundreds of hours of to get even the most basic model completed. Now, you can open up a CAD program and begin making real decisions immediately. What previously took a large team to build, a single designer can now do. (The analogous logical leap to what the OP seems to be saying is) Soon, we won't need designers at all! A company will come out with a new car model simply by snapping design-legos together.
In reality, removing inefficiency made designers more valuable. If you were the best car designer 50 years ago, you had to spend a significant amount of time molding clay. Today, you spend almost all of your time creatively finding solutions to problems. I see the value of individual engineers increasing significantly as we remove cruft in the creation process.
I think the core of the argument is that most software problems are becoming easier, which means the talents of the very best just aren't applicable or especially useful.
If something is easy enough, anybody can do it, and do it nearly as well and as fast as the best. For example, nobody can flip burgers 10x faster than average. Maybe 2x, but not 10x.
If something is difficult, then many people won't be able to do it at all. Most of the rest may only be able to do it very slowly and with great difficulty. The very best will make it look easy, even though it's not.
When something becomes 10 times easier, that doesn't mean the best become 10 times better or faster at it. They may become a little faster, but the real gains are at the low end: The average and worst performers become significantly better. Further, many people who just can't do the thing become able to accomplish it.
Replace "software engineer" with "person building a product" in that first quote.
The value of engineering skills as they are defined today is falling, but the value of a single person with an idea and the drive to implement it is soaring.
Back when I was a physicists every two bit crank used to go on about how ideas were more important than knowledge.
Now I'm hearing the same in programming. And it's just as wrong here. Picking the right algorithm and stack means the difference between a website that can handle 2,000 people and one that can handle 2,000,000. How many websites start off with a bad idea, get popular and then need to hire hundreds of engineers to put ban aid after band aid on a fundamentally terrible idea?
Facebook and google have both had to spend millions of man hours to fix the fact they were optimized for millions of users and billions of web pages and now have to deal with x1000 times those. And the only reason why they were that prepared in the first place was because extraordinary people wrote them. The average developer who bangs out a half thought out colour by numbers website would be lucky to have it not crash before it reaches a thousand users a day.
On the flip-side, the scalability doesn't matter if no one comes. The efficiency of the implementation doesn't matter if the wrong idea is implemented.
Sometimes skimping on the engineering is better than skimping on the idea. That's what MVP and quick-to-market are about, correcting the idea so you can architect a solution to the RIGHT problem.
This article is grossly oversimplifying and conflating things. All the examples cited are skewing the results to fit the agenda the author wants to promote. Specifically, the functionality, audience, and valuation are creating insane $$$-to-engineer ratios, that the author then tries to claim is the future of the industry.
Functionally, imgur is not very complex. Its an order or 2 of magnitude less complex than say, another "software is eating the world" business like a web-based CRM app. That's not to discredit imgur or Instragram. They are great achievements and successful apps. But it doesn't and shouldn't take dozens and dozens of engineers to make those.
The potential audiences for the examples are, essentially, the world. These apps can gain use very rapidly, and it can disappear just as fast. 1 person can make Flappy birds and the entire world can use it. The people to audience ratio here is huge compared to say, a SaaS invoicing product.
Imgur doesn't have a monetization strategy like, say, a SaaS product like a web CRM, or even something as simple as a WordPress backup SaaS product. Instragram had no business model. How much is it worth? This is where the crazy audience ratio comes in and messed up the results some more by effecting valuation. These valuations are not based on real revenue numbers. They are based on the wish and dream of extracting value from eyeballs. I'm sure they are worth something, but taking a Series A or Series B valuation and extrapolating is murky.
So, low functionality apps, with a huge potential audience, and a completely made up valuation number. Of course you are going to get $10B to 10 engineers ratios. That doesn't mean this is a trend in the industry, let alone business/enterprise software.
The article suggests that, as super-human programmers build ever-better platforms/languages/tools, those building apps/whatever on top of those will be less differentiated in terms of programming ability, because it will be so easy to develop stuff anyway.
I'm not sure this is true. Are 10x programmers really 10x because they can grapple with technology stacks, or because they're really smart and experienced at problem solving? Programming will always be about problem solving, no matter how easy you make the final implementation of the design.
Specific skills (e.g. working with a certain framework) will go in and out of style, but problem solving will always be in demand.
Also, maybe I'm naive, but I just don't see the software-as-legos future as feasible. There will always be the need for customization and ever-more-advanced functionality.
If you spend a few years at university (I have two degrees), you learn things like sorts, automata, proofs, and calculus.
On the other hand, most of the junk I deal with day-to-day as a working software developer turns more on my knowledge of tools like Chef/IntelliJ, software libraries (the stdlib of various languages, the Java/.NET BCLs), build systems, git/github, and how to do a proper code review.
For most stuff today, I would highly prefer the person with more tools experience. Granted, there are some problems people who aren't "10x developers" could never solve (e.g. writing linux) but for most stuff industrial software devs today are doing, it just doesn't matter. You just need to write the code, it needs to be maintainable, and it needs to be done as quickly as possible.
You compare two people who have different skills. Presumably either one could learn the skills of the other. Why choose the one with the easiest skills to learn?
This post is making the completely wrong conclusion.
The reason companies like Instagram, Imgur, etc are able to service large #s of users with 7 engineers is mainly because computers and tools are a lot CHEAPER than they used to be, not simpler.
Cheaper. Less expensive. This allows small teams to build really huge things because generally small teams don't have much money (because if they did, they'd have a bigger team!), and the fact that stuff is cheap allows them to build it in the first place. Open source = free. Computers, bandwidth = a lot cheaper than they were 5/10 years ago.
Snapchat couldn't have existed 10 years ago because it would have cost 100x what it costs today. In fact, things are so cheap these companies can overspend and run their service on AWS or Heroku because the price of computing doesn't matter as much [1].
Programming is just as complicated as it was before. In fact, anything where you have a team of 5+ people working on the same thing for years is going to be complicated whether it is software, legos, or plumbing. As tools get simpler, the product will compensate by becoming more complicated (which is why we are seeing more awesome software these days), and thus the level of complexity will be roughy the same.
Today I could build a product as a one-person team that would have been competitive 5/10 years ago, but today the bar is much higher, and of course it will continue to get higher as we software and the trade of building software improves.
1: Though companies like StackOverflow still build their own metal and because of that run their sites incredibly efficiently on a cost basis
I think the tools are not simpler as being simplist, but easier to use, maybe by having clearer objectives and design than it's predecessor.
Also, it was not long ago that put a web system was hardm with lot's of workaround or using not specialized tools for the job. Today is clearer what one need to create and support such system online.
If AWS is so overpriced, why isn't Amazon profitable? Colo is only cheap if you have cheap sysadmins and have a very predictable non-decreasing long-term compute need.
The article makes the error of assuming that a 10x engineer is 10x because he has arcane knowledge about the intricacies of low level programming. This may be true for some layers of development. In my experience, for application programming, this isn't the "10x skill".
10x engineers are the guys that can grasp the complexities of large systems, enabling them to change them with minimal effort. When prompted with the need to implement a new use case, they understand the whole system or most of the system and thus they can plot the minimal change path to achieve the new functionality. They don't code faster. They code less, leveraging work that is already done.
If you have fewer engineers working on a system, because of better abstraction layers, then you only need the 10x guys. These are the ones that can drive the powerful machines.
What we are witnessing isn't the demise of the 10x engineer. It's the demise of the code monkey.
This doesn't pass the sniff test. The demand for high-quality software continues to outstrip supply. Bad software is an imperfect-but-better-than-nothing substitute for good software, which is why so much of it gets written.
The problem here is that there's a sort of Gresham's law effect - bad software drives out good software by infection. Your technology stack is only as good as the worst component, and if you're operating at the level of indirection that Sam Gerstenzang suggests, that's a lot of software that you need to depend on, and your software can only be better than the software it depends at the cost of a lot of effort. Some component in your stack doesn't work the way you need it to? Well, suddenly you're either three layers down the stack trying to fix it, or you're writing a non-trivial piece of software higher up the stack to make up for the deficiencies of the lower level.
I'd also dispute Gerstenzang's basic premise that modern engineers need to know less than yesteryear's. 20 years ago, you could write an entire app in Visual Basic. It could integrate with a database, had a drag-and-drop GUI builder, was scriptable using a language that, for all of its many idiosyncrasies, was fairly easy to learn (and you could write plug-in components in C++ if you wanted to). A statically-compiled binary could be put in a floppy disk and would work on any computer running Windows, where it would look basically the same.
Modern front-end web code needs to navigate a Rube Goldberg machine of preprocessors and build pipelines that would have embarrassed a C developer from 1988, and incompatibilities with particular browsers, operating systems or form factors frequently cause simple tasks to take longer than expected. Because nearly every modern application is a distributed application in some sense, you need to understand concurrency and network protocols, and if you want to work on both the client and server components of your application, you probably need to know more than one programming language, or the subtle differences in the runtimes that your language has on client and server. And that's just fairly simple web applications!
WhatsApp was created by a small team, but that team needed to include skills in everything from Erlang to J2ME. That breadth of knowledge was what gave WhatsApp the key competitive advantage of being able to support almost every mobile device on the market, not just the latest generation of smartphones. You might be able to get a startup off the ground without engineering, but you won't be able to grow it much.
This has always been the case. Software people in the 70s and 80s already had the distinction between "systems" and "application" programmers -- the former understood to be concerned with making the pipes and plumbing, vs. the applications guys concerned with the end users. Applications have always been like legos.
I work as a systems developer but often do consulting projects as an application developer, and it's a totally different mindset. Writing software for end users, I rarely even remotely think about CS, algorithms, pointers, recursion, numerical stability, memory allocators, etc, whereas I do care intimately about the tools I'm using, much like I suspect a carpenter cares about what drill he's got. On the other end of the spectrum, I can hardly go a day writing traditional "systems" software without considering things like what data structure MongoDB/Postgres uses to hold indices, and gigantic, complex consensus problems (distributed systems, e.g. how DNS behaves).
I hope we start to treat these two endeavors differently because they require totally different mindsets and education. A typical app developer need know no more about how OSes work, than a plumber needs to understand the metallurgy used to make pipes.
I see the overall point you're making, but it's not true programming applications is always (or even often) like building with Lego pieces. It's just that many of us have to write mindless web apps which mostly consist of CRUDs. Those definitely are like working with Lego.
CS, algorithms, pointers, recursion, numerical stability, etc, etc, are all alive and well in the world of applications programming. Just not in the world of run-of-the-mill CRUDs.
Sure. And to be clear, I in no way think applications is easier. In fact, applications, as they're closer to end-users, have to deal with all kinds of socially-imposed whimsy like people with four names, tax rules, interest rate calculations, and 100 other things that make my head hurt to consider.
Gerstenzang is watching the latest complexification of small shops into huge industries and projecting it onto the entire field. This is very silly reasoning and it ignores history.
Transported back in time, he could say the same thing watching the small, passionate computer shops of the 1970s becoming huge organizations as customer money and investment capital flow in, as more engineers are hired, as efficiencies and industry-wide standardization set in. He could say that the emergence of low-skill labor in the mature industry was the end of engineering, that there was nothing more to do.
But there will always be a new layer to this game. Once the previous layer has walked its arc from hobbyist to small shops to huge industries and maturity, there emerges a ragtag group that takes advantage of the new environment.
Gerstenzang does identify and appreciate this crucial factor, he just seems to view it as an endpoint and not a beginning.
Of course, an AI that can do real, human-level engineering might throw my own entire line of reasoning out of the window.
It's already happening. There's an incredible difference (in personality, tooling, skills) between your typical full-stack web developer and kernel programmer, as there is between the kernel programmer and chip designer. They're each difficult in their own way, but one definitely has a lower barrier of entry.
One could argue the exact opposite: Engineers are needed now more than ever. A skilled engineer (what does this even mean in a context so wide?!) can run a website like PinBoard, serving millions of users, from the a jungle in Thailand, on a crappy 56k using a 250 USD ChromeBook.
Of course it's possible but how many people do actually manage to do it successfully?
And while having AWS, Github and you-name-it makes this possible (scaling was the big issue apparently), AWS and Github can not be run by a lonely engineer, you need an army of them.
1) Even if we are able to remove all of the inessential complexity of the problem (ceremony and routine drudgery,) we cannot automate the essential complexity. For crud apps, the essential complexity is very low. However, for many apps there is intrinsic complexity to the problem and how it relates to the domain. You might argue that what this leads to is a vast array of cheap specialty modules that can be plugged together. Well, the complexity is then in understanding the impacts of the modules and their interrelationships. The 10x engineers aren't 10x because they can set up logging or automated deployment, they are 10x because they can instantly cut through multiple layers of abstraction to see how their interactions lead to an emergent behavior that is different from the spec (for example.)
2.) Unfortunately, machines (and networks) are not fast enough to compensate for an unconsidered approach to information retrieval and propagation and this is not going to be helped by a "legos" approach to building software. This might be helped by a different set of tools (view-first rendering driving parallel backend requests..) but that brings up its own set of challenges.
This article fails to mention that our expectations for software have increased. We expect more from apps than ever before. This has lead to new specialities such as Front End Engineers and UX staff. Surely we can build more with less with current technologies as compared to the past but we also expect more. If our expectations outpace technological improvements this could lead to spending more time/money on developing apps.
Uniformity isn't so great for people who want to differentiate their products. (Not saying that they should want to differentiate their products visually, but they almost certainly do want this).
This seems to be pretty specious reasoning which has at its heart the principle that acquisitions are the final goal, not a functioning enterprise. I think the acquiring companies would tend to disagree.
Teams are not getting smaller. The original Macintosh team was not much larger than the teams he mentions and probably had much more technological impact.
The web has been around for a quarter century now and still rather trivial websites that mostly could be replaced by tools sell for some $50k. Because even those basic tools we had 25 years to build are not there yet.
Software engineering is actually becoming proper engineering, at least in Europe where much of it is focused more on industry. Crashes are mostly unacceptable now. Embedded software is sometimes formally proved.
Finally, as everyone who actually develops software knows, we need to cut many needed features to ship in time. Think how incredibly long it still takes to implement what you can imagine: It takes many months. This is a good indication of how far we are actually from combining things in a lego-like fashion, from making dreams come true quickly because it is so easy.
The author makes some good observations but I feel he is undervaluing post-foundation engineering skills.
Sometimes the reason why certain companies end up with so many engineers due to the lack of those so-called "10x engineers" -- specifically, those who know how to build robust, scalable systems while coordinating with the needs of product development.
I know some of the engineers who have worked for those lean, highly scaled shops and I know for a fact that they know their shit extremely well - and they've poured hundreds of hours into carefully scaling their infrastructure. They could not be replaced by your average full-stack engineer off the street as they'd immediately be in over their head and would have to spend the next few years learning all the hard-won knowledge of the more experienced engineer.
Let's compare writing software to writing English. (Writing is kinda an individual sport) Almost everyone can write a short story in English, but the 10x writers really really stand out. Writing isn't a perfect analogy - we generally think of programming as a skill (right and wrong, boolean, either you can or can't) and writing as an art (shades of gray, non-boolean skill).
The assumption is that more people being able to make things with code would make the 10x engineer less valuable - but I think it makes them more valuable. In a noisy world - only the truly great can stand out. And to really make our future great, we are going to need a lot more than CRUD web apps.
I could be wrong - but I don't think this is as cut and dry as the author claims.
The implications of this are pretty interesting. People will only use software that solves actual problems they have. Software development is now more accessible to more people because of all these frameworks and tools. It only follows that the people most familiar with actual problems will be able to create the most used software, and those people aren't necessarily going to be software engineers by trade. The question "How long before we have a billion-dollar acquisition offer for a one-engineer startup?" is crazy, but likely not too far in the future, but even more interestingly, "How long before we have a billion-dollar acquisition offer for a zero-engineer startup?" might not be that far off either.
This article is confusing two different things: engineer and the tools used by the engineers. It's not like the tools will ever substitute an engineer or exist one tool to rule all other tools or even one tool that do everything. What we have now better tools than before, which makes possible for few engineers to produce more than before.
Examples from other jobs, graphic designers were not substituted by tools like Photoshop and InDesign, will they ever be? I don't think so - and that's by design, the tool is to easier the job, let the person concentrate where it makes more money, and not to substitute the person.
Part me wants to agree the picture painted here, but bits like the one below make the whole thing out to be an veiled appeal for more naive investment targets without the myriad complications of engineers.
Basically, they'd like a lazy Susan of people who've done a bunch of leg work on a 'lego' app or website and are ready to accept $50K for 60%, Shark Tank style.
>'As software becomes a high-impact, low-skill trade, we decouple the technical ability and experience needed to write tricky software from the ability to solve problems for people.'
Not exactly.
In keeping with the analogies to traditional infrastructure and trades.
This sort of commoditization of labor allows a certain class of problems to be solved - repeatable, fairly well-understood ones for which there are ready instructions and appropriate 'legos' available.
Providers of these services race to the bottom and have to be kept in check with codes and consumer paid inspections to ensure that their snap-together builds meet barest minimums of safety and sensible design.
In this race to the bottom, implementers don't care and won't pay for better components, just cheaper ones that meet the minimum spec.
There's good reason people loathe calling a plumber or HVAC repairman as much as they love certain tech companies known for their design and engineering.
This is probably fine for money men, but neutral to bad for nearly everybody else.
Looks like I'm riding the wrong train, screw learning C++/Scala, Big Data and NoSQL database, looks like engineering is gonna be commoditized.
Not this time, I'm gonna get on the new train early this time! I'm gonna pivot to learn SEO, A/B testing, social media marketing and lean startup principles on how to manage coders to build startup's on the backs of free open source software hosted on Github donated by hobbyists!
A billion dollars for a one engineer social media company? Certainly possible, but then you'd have to ask yourself what you are paying for. Perhaps it's not just (or even mostly) the engineering; in which case I'd say the author slightly misses the mark here.
This is certainly not to say that I believe more engineers/engineering == better product or more value, far from it. Only that devaluation of skill and experience suggests that the problem being addressed might not be primarily focused on engineering tasks. And that perhaps the gargantuan sums we see being paid for some small collection of companies are calculated somewhat differently than we might be used to.
Even if I'm wrong about that, heralding the death of the highly skilled engineer is more than a little premature either way. I tend to think that problems will continue to present themselves to challenge our best tools, regardless of how much engineer quality of life improves. If that's not happening in social media (which I highly, highly doubt), then that can probably be considered a separate issue entirely.
Well, yes, if we restrict our domain to the set of application types for which development can be commoditized, then development becomes a commodity.
Not all development is webapp and mobile app development, though, and some things, like actual engineering, just aren't things that can be commoditized (beyond a certain point).
I happen to think that because the leverage is greater, the reward for hiring highly skilled & talented engineers is much greater than it ever was.
All the products mentioned were functionally simple and I like billyhoffman's explanation of why the low functionality/large audience products don't represent the majority of the industry - https://news.ycombinator.com/item?id=8109434
I'm trying to understand how Sam Gerstenzang becomes a partner at Andreessen Horowitz and I can't figure it out.
When I was in college in the mid-late 90s, my Software Engineer professor said that "In 5 years, programmers won't exist. With the revolution of Object Oriented Programming, creating software will be a matter of plugging blocks together".
The Lego approach has been the holy grail of software development at least back into the 80's with OOP. Yet it still requires programmers who know about data structures, algorithms, etc. to write software. Even if they're just cobbling together libraries, there is still a lot of additional functionality required--not just 'glue' code.
All we have now are improvements in efficiency (great, mind you), with operational improvements like IaaS and development improvements with the great OSS that we have now. It will require a sea change in the field of comp sci. to achieve Lego-like orders of magnitude improvements.
I would like to think there will always be a frontier, and there will always be people driven by their curiosity to seek the frontier out. Some of them will be called 10x engineers, some visionaries, some pioneers.
This article sets up a a few interesting points, but stops at only one. Yes, the ever-increasing level of abstraction available in information technology enables less technically-specialized people to creatively utilize the tools available to them, and that's pretty amazing.
But the plumber analogy fails to capture the breadth of the tech world. What I would call equally exciting to the empowerment of less-technical developers is what highly technical "10x engineers," or whatever you want to call them, are enabled to do in a world of next-level abstractions.
> The same software foundation (open source software, development tools like Github, infrastructure as a service provided by the likes of Digital Ocean, and more) that allowed Whatsapp and Imgur to scale, means that experience and skill writing software become less important.
The author focuses a lot on ease of scaling, but the bigger challenge has always been building applications (experiences) that need scale. If you build it, they will not come.
I love how all the top comments here are all devs in denial about this, but it's an inevitability that programming "becomes legos". Don't become the old men guys!
As said in the article and below, many products will still require very high-level engineers building without "legos", but the basic webapp kind of stuff probably will be full commodity able to be built by anyone with the motivation to see it through.
It's not about being in denial. It's that this argument has been brought up with every advance in the last 40 years.
In the early '70s it was "Programming in C will allow anyone to program and commoditize the industry!".
In the later '70s it was "Object-oriented programming allows modules to be created that snap together like legos, commoditizing the industry!".
In the '80s-90s it was "Scripting languages like ruby and python are so simple, it's almost like writing real english! It's so simple you won't even need a programmer."
In the early '00s it was "Scratch and other visual programming languages will create a platform that everyone can understand! You can learn it yourself, you don't need a developer."
Now, it's "Libraries and tooling are getting so great, what used to take 100 engineers now takes 5. In a few years, it will take zero!"
--
That obviously isn't proof that this time is the same as the others. However, this article gives no evidence that I can see why this simplification and removal of inefficiencies is any different than the others.
Yea...and it has gotten easier. I'd bet the average level of software dev (writ large) today is lower than 20 years ago because it's getting easier. (Though I'd bet there's probably more great devs today ever because the pool of devs itself is growing and becoming more accessible)
Sure, having MS Word doesn't make you a novelist, but Wordpress + plug-ins can let you make your own pretty high-quality news site with very little technical knowledge - that's real improvement.
I'd say to just give it time.
My company is doing this to another set of engineers in the mechanical / manufacturing space, so I hear these types of complaints all the time.
Most Wordpress themes are highly inefficient in regards to load time and bandwidth. There are some great plugins but most are sub par quality in terms of performance and security. All large scale Wordpress deployments are tuned either by in-house engineering staff or http://vip.wordpress.com/
> Software engineering is not yet plumbing — or Legos — because our standards are incomplete, our libraries incompatible, scaling is still not free and our software still buggy.
I think it's more that plumbing has been with us for thousands of years. Imagine if we had computers 5000 years ago - software surely would have been plug-and-play by now as Sam writes about.
Its funny I've always thought that Twitter could really be run by 100 folks (heck I've said 40) and be hugely profitable and it a lot of ways more efficient. Instead they've got 2300 employees to allow folks stream 140 character tweets.
You can't scale without well crafted solutions - scaling your rails app on digital ocean would have a higher burn rate than hiring a few good engineers to build scalable code that needs fewer machines. AWS is pricey!!
This seems to be ignoring entire classes of software. I can't imagine the enterprise development world will be taken over with easy to snap together software packages any time soon.
Because enterprises are infinitely complex and diverse and there will never be a single piece of software that can run all the businesses in the world with a couple configuration setting tweaks.
I had this vision 5 years ago, for sort of an operating system for social apps. Where a regular user could build their own app by just installing some plugins, dropping a chatroom and some other components on a page, paying a developer to wire them up, and paying a designer to design the theme. And there would be a marketplace of components built by more experienced developers.
Like mobile apps, social apps require an operating system that solves common problems in a standard way:
* User signup and secure auth, login
* Should work with emails, mobile #, facebook
* Accessible from desktop, tablets, mobile
* Contacts integration / friendlists
* Real time collaboration from any device
* Offline notifications via fb, sms, app notifications etc.
* Access control and privacy - as expressive as possible
* Native apps on iPhone and iOS via Cordova
* Invitations and one-click account creation
* Encourage best practices for scaling, web standards etc.
* Installer to upgrade apps and plugin versions cleanly
etc etc.
All that stuff should have standard solutions which you can override. The apps and plugins should all interoperate.
And finally, since I really care about decentralization, it would be cool if this whole operating system could be as decentralized as Wordpress is for blogs, as opposed to say facebook. That would also solve all the scaling problems that facebook and other centralized social networks have.
That last part - decentralizing the social network - took the most effort, as I had to solve problems of security, identity, privacy, as well as app and plugin versioning and signing across a distributed network. Just like iOS is for mobile apps, and Bitcoin is for distributed contract apps, the Q Platform is if you want a social layer for your app, from user accounts to everything else, without having to write it from scratch.
How come every time I post about the platform I've spent 3 years building someone here decides to vote it down? I'm just curious. Is it really something you don't want to be spoken about?
I think this got downvoted due to the way you describe it.
My experience with HN is that it's a very hard-edged, no-shit crowd. People here expect things to be described with simple, crisp language, no jargon (especially not computer/science/technology terms with precise meanings), and a minimum of hand-waving. It's a bit hard on the ego, but it'll make you a better communicator.
In the first sentence of your post, you described your thing as a "social operating system" or something like this -- this immediately sets off my bullshit alarm. "Operating system" is a very specific thing to software people, but for whatever reason, non-technical people love using this term to talk about something perceived to be big and complicated.
In general, try to cut the fat. If it's a library of software components you're making, say that. If it's a Ruby gem, or an app, or some kind of a specification, just describe it for what it is. You don't gain much here trying to hype it up.
Other things I notice:
- You say you can "override" -- how? Using what?
- What is a "social app", anyway? Aren't all apps "social" today? This descriptor (social) adds very little value. Github, facebook, twitter, instagram, linkedin, could all be considered "social" apps.
- "Social" is a very overused buzzword in SF. Everyone here is building bullshit "social" companies and all the devs here get like 10 job ads/day for some dumb social network for cats.
- Again, "interoperate". How? Via a REST API? Server-side includes? Be specific.
- Don't say "it's come a long way". Stop telling. Show.
- "Decentralized". Again, what does this mean? It can be hosted anywhere? The software is widely distributed? What is a "distributed network"? Aren't all networks "distributed"?
WhatsApp was able to scale to so many users while keeping the engineering team small precisely because they hired great engineers who wrote an extremely efficient system in Erlang. (Does he think commodity coders know Erlang? Really!?) They heavily tuned their architecture and built efficient paradigms which kept developer productivity & performance from degrading as their user base grew. [1] Commodity engineers don't write that sort of code.
Yes, we increasingly have tools which allow mediocre coders to get stuff done. (For example, Ruby on Rails.) But the inevitable byproduct of letting mediocre coders loose on a stack they only partially understand is a mess of spaghetti code which takes even more developers to maintain. This is already what you see with non-tech corporations who employ dozens/hundreds of developers to maintain Drupal/RoR/Java apps which are significantly simpler in scale & complexity than WhatsApp or Instagram. If you want to use tools well, you need to hire people who understand the tools.
Yes, the tooling has improved dramatically. A 10x engineer of yesteryear might actually be a 100x engineer today. But if you give a bad engineer modern tools, it just allows them to fail faster.
I might be able to build a house with legos, but I expect to be spending a lot more time repairing it than if I'd just hired a carpenter in the first place.
[1] http://highscalability.com/blog/2014/2/26/the-whatsapp-archi...