> Good luck developing anything mission critical with a team of low paid googlers.

IMHO, that's another dark-secret of this trade, most of us don't develop "mission critical" services. By "mission critical" I understand something like "if this code breaks then someone dies or huge sums of money are lost etc.".

> One day even people in our profession might come to actually have some self-respect for what we do

I totally agree with you that "software development is hard", but not because someone happens not to know what methods a standard library class implements (every few days I give almost god-like thanks to the developer that decided to implement the dir() function in Python), but because ~35 years after "The Mythical Man-Month" was first published we repeat at least half of the mistakes presented in there.

Yup, I agree.

Software engineering also requires a lot of mental pain, and struggling and debugging, but the end result? it's equal to what other occupations produce with less struggle.

And it's never the same half of the mistakes, is it? :(

That's the thing that annoys me about software engineering--it's impressive that it works at all, and that isn't a good sort of impressed.

"Good luck developing anything mission critical with a team of low paid googlers."

This is the dirty secret of outsourcing.

And the dirty secret of outsourcing is that it very rarely goes the way people want it to go and the low paid googlers end up being replaced by very expensive experienced software developers.

Sorry, that's what I was implying. You get a bunch of people who work for so cheap because they're not properly trained, they invade forums and such asking for "the codez". The product can end up as a bunch of code snippets pasted in from the Internet, may work, may not.

http://plzsendmethecode.blogspot.com/

If your original team was a bunch of improperly trained people who happen to be nearer to you geographically, at the very least, you have saved some money building a product which, as you say, may or may not work.

Here, let's grep and replace:

"The dirty little secret that I don’t mind saying is that mechanical engineering isn’t hard. There’s lots of handbooks, manuals, blog entries and Stack Overflow Q&A pages out there that very clearly describe how to work with any stress equation or tool. Follow it step by step and you are a mechanical engineer."

This is pretty common in professional engineering--most bridge/pressure vessel/structural engineering is done by effectively filling in the blanks on worksheets and following thick handbooks of best practices. Are they too worth your contempt?

Self-respect doesn't enter into it--computer science might be hard, mathematics might be hard, clever low-level or very abstract programming might be hard...but software development? Especially as practiced nowadays with the toolsets and parts we are encouraged (nay, expected!) to use?

No, software development need not be any harder than any other form of white-collar engineering.

It isn't that much different from other forms of engineering. And in all of those forms, there are the "formula plugger" engineers, and there are the real engineers.

The difference was exemplified by a question another "engineer" asked me once - "what book did you get that formula from?" I replied that I derived it from the boundary conditions, etc. He just looked at me in utter bafflement.

On a related note, it's easy to discover the "formula pluggers" in electrical engineering. They are the ones that carry around a little card with:

   V=IR
   I=V/R
   R=V/I

helpfully written on it.

Show them the PIRE wheel, it will blow their minds: http://www.google.com/search?q=pire+wheel&hl=en&prmd...

The problem here is that in things like chemical or structural engineering, you generally don't just bodge together things by first principles.

In fact, there is a great deal of liability on the line--so much so, for example, that the guild (for what else is something like the ASME, really?) has decided "Here are the equations by which we design pressure vessels. Plug in your material--from this table of guaranteed grades and empirical data--and your dimensions, and lo, here's what you should spec."

As wonderful as it is to derive these things from equations and first principles, and as satisfying as it is to do so, do you really want to stake your career on your own from-scratch solution to a CFD problem? Or on the exact solver that you used to solve those PDEs? Or on your "hunch" as for which forces were truly at play?

Or do you want to follow the guidelines in Shigley's, Roark's, ASME codebooks, NACA airfoil reference, etc., and be able to more easily pass through a forensic engineering post-mortem?

Electrical engineering and computer science may not have had the same checkered history as some of the more established disciplines, but that doesn't mean you can't learn something from their rigor.

>The problem here is that in things like chemical or structural engineering, you generally don't just bodge together things by first principles.

The thing is, formula pluggers are unable to put formulas together from first principles because they simply do not understand them. All the book formulas are based on certain assumptions - and so the formulas will give you wrong results if your real world problem doesn't match those assumptions.

When it doesn't match the book formula assumptions, you have to derive one that does. And that requires understanding. Furthermore, you check your equations backwards and forwards using well known techniques. Lastly, you back up your analytical results by using a test rig.

Yes, I've known a lot of formula pluggers. You argue that they get rigorous correct answers. They don't, they misuse the formulas because they fail to understand them.

Understanding that is what separates a real engineer from a formula plugging technician.

What impact does this make on actual work delivered?

Seriously, we can talk about negligible leaps of quality that a guy who learns things in details has over the one who works with powerful layers of abstractions.

There a lot of workshop guys and foremen, who can repair even the most difficult problems in car engines without understand much about the physics and chemistry of things going on inside an engines. They can also build custom engines by assembling parts. Sure they can't match quality of a guy who can build everything from scratch, but get enough quality of build useful things. A lot of them can build homes.

Surely some centuries back only mechanical and civil engineers could do this. But today we have tools and techniques that make dealing with things very easy.

This is the natural graph of evolution in every discipline. Not just software but in nearly everything.

I worked on airplane design. Getting the design light and strong was worth $$$$. The extra money to pay a real engineer was nothing compared with the cost of keeping heavy objects at 30,000 feet. In electronics, shaving cost off a design you're going to make millions of copies of is worth $$$$.

Repairing a car engine is very different from designing one. There's a reason why, if you want to work on race cars on a professional team, they expect you to have an engineering degree and use it.

Designing a house isn't much engineering. There you can easily afford to way, way overbuild it to compensate for utter lack of finesse. (Funny story - in Florida they found that Habitat for Humanity houses stood up to hurricanes better. The reason? Habitat workers knew nothing about construction, and so used far too many nails.)

You're right that any semi-competent mechanic can build a working tractor from scratch, with doing no calculations at all. But it'll be either excessively heavy, or not strong enough, or cost far too much to build.

I'm not saying you should re-derive book formulas. But you are often going to encounter problems that don't have book solutions - like calculating the moment of inertia of a spinning part that doesn't match the shape of the book formulas. Or the bending strength of a beam with an unusual cross section. Then, you gotta know how those book formulas are derived in order to derive the right formula for your unusual shape.

Your alternative is to overdesign, or to compromise your design into being book shapes. Have you ever noticed that your car doesn't contain many parts with book shapes?

Exceptions only prove the rule.

The number of people working on mission critical stuff are always going to be lesser.

I am not against learning things in detail. I am only saying these days productivity has become more important metric than them in most general cases.

Now please don't tell me about cases like space craft/airplane design. You and I know well, they don't apply to the masses.

And after all experts have got to be good at something.

Heh, when I was in my Electrical Engineering programme, I got funny looks; instead of the formulas listed, I pretty much had to just keep two straight: i = C dv/dt and v = L di/dt. Alternatively, Z = 1/Cs and Z = Ls. While everyone else was trying to keep all of their formulae with exponentials straight, I'd just solve from the base equations (either by solving the diff-eqs or equivalently the inverse Laplace transform).

So what is going to happen when you are asked to design (or dare I say "engineer") a system of some sort from scratch? Say, an engine or a component of an engine or a complete new machine? That is engineering. What you described sounds like the duties of a technician or a mechanic. That is not engineering.

Firstly:

A designer is allowed to come up with whatever they want.

An engineer is subject to constraints, most notably: does it work, is it safe, and is it cheap. That last value does not always mean money--what is the cost if the design goes wrong, or if others have to maintain it, and so forth.

This is the popular notion of "engineering": that a stoic engineer sits down, sliderule or vim in the one hand and coffeepot in the other, and sits up into the wee hours of the night scrawling schematics onto a sheet of paper or typing into a VT101, only to go pound rivets or machine code the following morning with only the aid of his throbbing turgid member--this is absurd!

Secondly:

In your case, for an engine, I would ask for details about the weight, cost, power output, speed, time, maintenance lifecycle, and predicted loadings of the engine. Then, I would sit down and blackbox the inputs and outputs. Then, I would go and figure out mechanisms to accomplish each box. Then, I would look up the values for those materials--honestly, do you expect me to come up with the vibration amplitude of a forged 316 stainless shaft of x length and y rpm myself? You don't want me to not use tables for that! Those things--the "engineering"--done, I'd get to the actual "design" work: how do I shape these mechanisms, how do I get them to fit in a box in the size we decided on, etc.

To hold you to your own standard:

Would you write your own web stack?

Okay, would you write your own protocol?

Your own compiler?

Linker?

OS?

Hardware?

We stand on the shoulders of giants--I am no less an engineer for using reference tables and formula than you are for using Redis/Rails/libc/Ruby/Python/etc.

The lack of invention? That's the sign of maturity.

I have done all of these things, and I know when it's appropriate and when it's not. The trick isn't that I can "come up with the vibration amplitude of a forged 316 stainless shaft", but that I know how I could do that. Over and over, I've seen the effects of failing to understand the abstractions (formula tables) that people are using to develop software. It's that fundamental base understanding that, in my opinion, can propel someone from pretty good to exceptionally awesome.

"low paid googlers" is an interesting phrase. Took me a moment to get it, and disambiguate from "high-paid Googlers"

We, with our very hands destroyed this when we started inventing IDE's that had intelligence to do nearly everything that had to be learn't to be done other wise.

We invented frameworks to cover large number of solutions to problems.

Its something like this. There was a point of time among mechanical engineers when knowing how to assemble and sweating it making things in the workshop was glorious. Today most of those jobs are inevitably automated. And apart from design and actual engineering. Most mechanical engineers can be clearly separated from foremen and workshop guys.

The same is happening with software, IDE's and frameworks are making it damn easy for any idiot with vague ideas to program things into existence. Soon coding in itself will not be a very task in itself.

At this rate its scary to imagine how future generation of programmers will even program. They may just deal with visual interfaces which generate programs in the backend.

This is sad but inevitable in every discipline of engineering.

If any average Joe can make something that works, then the true and valuable skill is to make something that is simple, understandable, and maintainable.

I don't think modern tools help you with any of the constant decisions needed to hit those marks.

I never implied that.

What I said was over the years with the help of tools it takes lesser intellectual work because the more and more skill based intelligent work is automated inside IDE's and frameworks.

You can do a lot of network programming today without actually understand anything about networks. You could not have said the same around 15-20 years back.