Code is a solution, based on a model of the problem.
Understanding a problem is more important than writing a solution that is clean in itself. However, we often don't understand a problem immediately. Writing code (and seeing the results) is one way to quickly gather data about the problem, and so help understand it.
This is where clean code can help: code that clearly communicates your current model of the problem is very helpful part of the process. After a break, it reminds you of the model; someone else inspecting the code can see the model; it is easier to see where the model is incorrect; it is easier to modify the code to correct the model; when the problem itself changes, it is easier to modify the code.
However, there is a trade-off between effort and clarity: it is a mistake to spend too much time clarifying something that might not be correct (or, if the problem will change soon, making it incorrect). Just write code that reflects your understanding of the problem. Regarding efficiency: if you know how to easily make it more efficient without compromising that (e.g. without making it convoluted), then of course do it.
Code is not a final product, but a process. The faster you can iterate, the better it is. (An exception is if you already understand the problem perfectly.)
"...it is a mistake to spend too much time clarifying something that might not be correct.."
I think the problem is that too many people disagree on what is 'too much time'. Not trying to nit-pick but obviously it is a mistake to spend 'too much time' doing anything. That's implied in the too much. That may come off as being pedantic, but I don't mean it to be so. I think defining 'too much time' is right at the heart of the problem. And it's a hard problem to be sure.
I'd also disagree that the possibility of something being incorrect is a reason to spend less time clarifying it...I'd say just the opposite.
By clarifying code you're making it easier to comprehend and thus change in the future. I'd say you should write code with change in mind.
Regarding 'incorrectness' itself...that's the only reason code is ever changed. Whether the code is fundamentally incorrect (the type I believe you were referring to) or temporally incorrect, the incorrectness of the code is always the reason for the change.
I agree the exact trade-off is difficult. I just meant clarity is not an absolute. You shouldn't make a project out of clarifying - it shouldn't be a driver of what you are doing; instead, given different ways of developing quickly, you should choose one that gives clearer code. Write it as you see it, instead of trying to be clever - that is, write in in terms of your understanding of the problem not beyond it. It's bang per buck: "a little effort for a lot of clarity" is worth it.
As for the exact trade-off: when you are iterating quickly, it is not effective to try to nail down these trade-off precisely - you need those cycles for actual coding!
For that matter, it's also difficult to define what is "clear" - or even what is "simple". Have you ever tried applying the "Single Responsibility Principle" thoroughly? It's amazing how many genuine "reasons to change" you can find - and amazing how less clear it can be. Another example is dogmatically writing "short methods". Clarity, like "simplicity" - and like the trade-off you note - is itself extremely hard to pin down precisely.
BTW: reasons to change other than correctness include: making it more efficient; making it clearer (e.g. refactoring); making it reusable (Brooks' says it's x3 the work). And, in practice, there's a continuum of correctness, not true/false, for "better" (more accurate, more reliable) results: e.g. more accurate search ranking; better collision detection; better webpage rendering; OS that crashes less often.
I'd agree that refactoring for clarity can't reasonably fall under a wide 'correctness' umbrella. Definitely more reasons to change code than correctness.
I put the word in quotes to try to express a much wider than usual connotation for the word (air quotes, if you will..). It's a shame that language is all we have to communicate on the internet.... :)
I also agree that we shouldn't precisely define tradeoffs before continuing on a single, specific, problem.
I just think that, in the macro sense, we can and should get better at defining the factors that would inform such a precise weighing of the tradeoffs for clarify, efficiency, expediency, etc... Doing so would give us better instincts for single instances where doing a precise comparison would be absurd.
I agree it would be good to improve at weighing trade-offs. This is the standard method of progress in many fields, systemization of data, the finding of rules, the development of models - theories, in short. A nice practical example is Deming's quality management. They used to think you could improve quality up to a certain point, but beyond that you just have to put up with some random amount of defects. He showed that with measurement and accurate models, you can increase quality as much as you like. Is that more or less your thinking?
But I think there's a fundamental barrier for software, because programs can differ from one another arbitrarily. This makes it hard to generalize - and generalization is the essence of theory, rules, systemization, modeling etc. I assume you'll accept that generalization is crucial, but you'll disagree that software can vary arbitrarily.
The reason I say this is because if you find regularity in software... you can automate it. Factor out a common method. Extract a "pattern" into a framework (or better, in functional programming). Derive one thing from another. It's like compression: as soon as you notice something that you can generalize, and construct a theory to model it... you can use that theory to eliminate the redundancy.
In a deeper sense, programming itself represents theories. And theories (e.g. of physics) are not generalizable: we don't have a theory of theories, that enables us to automatically generate the next correct theory. Oh, we can automatically generate theories, just not correct ones. For that, we need data to test them, to find out which one is correct. And so the unknown is the ultimate cause of the problem: we can't generalize what we don't know.
The best you can do is come up with rules of thumb, that help in specific domains - but they only work there. And if they are really accurate, someone will encapsulate them in a method, a framework, a library, a language or operating system, so that programmers don't need to deal with them, and that pattern disappears from their programs.
And to finish off with one of my favourite quotes, by Alan Turing: There need be no real danger of [programming] ever becoming a drudge, for any processes that are quite mechanical may be turned over to the machine itself.
NB: in practice, there is an enormous amount of repetition in programs (and parts that are common between different programs). This amounts to ongoing demand for libraries, frameworks, languages, operating systems, and extra features for them.
So, it's hard to obtain a precise weighing of the tradeoffs for clarify, efficiency, expediency when the playing field keeps changing. We can get better, but not much. It's different in fields like civil engineering, where materials and techniques don't change often. BTW Another fuel for change in software is Moore's Law - which enables different techniques to become favoured that weren't plausible before (like dynamic languages, VMs, etc).
sorry, such a long and self-indulgent response. Maybe it doesn't apply so strongly to trade-offs (though I think clarity is closely related to theory). At any rate, it doesn't stop us improving, which is what you suggest.
Understanding a problem is more important than writing a solution that is clean in itself. However, we often don't understand a problem immediately. Writing code (and seeing the results) is one way to quickly gather data about the problem, and so help understand it.
This is where clean code can help: code that clearly communicates your current model of the problem is very helpful part of the process. After a break, it reminds you of the model; someone else inspecting the code can see the model; it is easier to see where the model is incorrect; it is easier to modify the code to correct the model; when the problem itself changes, it is easier to modify the code.
However, there is a trade-off between effort and clarity: it is a mistake to spend too much time clarifying something that might not be correct (or, if the problem will change soon, making it incorrect). Just write code that reflects your understanding of the problem. Regarding efficiency: if you know how to easily make it more efficient without compromising that (e.g. without making it convoluted), then of course do it.
Code is not a final product, but a process. The faster you can iterate, the better it is. (An exception is if you already understand the problem perfectly.)