We (or at least I) may base our estimates on the happy-path.
That is, I can see how to solve a problem in the common case. But as I code, I encounter the myriad of error cases, exceptions to the common problems, and especially the interactions of all these. Then add in performance considerations and the complexity can multiply quickly. The long-tail of those errors and exceptions can be very time-consuming to solve, so the mean is likely to be way off.
It's obviously much easier to estimate when you're experienced with a problem space. This should mean that you know how to solve a particular problem as well as the likely exceptions (non happy-path) you'll encounter.
If you're experienced with the domain, you can fit the relevant partial solutions in your head and more accurately judge what could go wrong with your code. But more generally I think this is similar to solving problems in other domains as well, such as architecture, building a car, or anything with many moving parts. These domains do not usually have loosely coupled sub systems.
Cars and buildings are probably more loosely coupled than most HN readers imagine. Up through the engine in the 1995 Mustang (I think that was the cutoff), the Ford small block engine line would all bolt into Mustangs all the way back to 1965. I've got a 1989 block, a modern 5-speed transmission with overdrive, hydraulic clutch, disk brakes, dual circuit braking, limited slip differential, and a semi-modern radio in my 1965. All of those were either direct bolt-on or straightforward upgrades with minimal field engineering needed. Many of the VW and Audi engines also have common engine/trans mounts and interchange across years and brands. You can often bolt heavier duty suspension or brake packages onto base model cars/trucks. "Parts bin engineering" is a phrase to search on to learn more. It's done aftermarket, but it's also done by the manufacturers.
There's a similar story in the mechanical engineering for buildings. Sure, some architectural choices will make it harder or easier, but most buildings can be retrofitted system-by-system over the decades a building is in service.
Part of that is to allow configuration pre-sale, but a lot of it is just "it's too damn complex if everything affects everything else", which is not that different from software.
That is, I can see how to solve a problem in the common case. But as I code, I encounter the myriad of error cases, exceptions to the common problems, and especially the interactions of all these. Then add in performance considerations and the complexity can multiply quickly. The long-tail of those errors and exceptions can be very time-consuming to solve, so the mean is likely to be way off.
It's obviously much easier to estimate when you're experienced with a problem space. This should mean that you know how to solve a particular problem as well as the likely exceptions (non happy-path) you'll encounter.
If you're experienced with the domain, you can fit the relevant partial solutions in your head and more accurately judge what could go wrong with your code. But more generally I think this is similar to solving problems in other domains as well, such as architecture, building a car, or anything with many moving parts. These domains do not usually have loosely coupled sub systems.