Funny anecdote: where I work, one of our departments had a model we used to predict the working characteristics of our most popular product. Over time, we noticed that real-world data deviated from our calculations, so the model was regularly adjusted to bring the predictions in line.
Recently we had to model a new system, and none of our numbers looked remotely right. Looking at the code, the whole program had essentially been turned into a lookup table that was completely useless at actually 'predicting' anything we hadn't seen before. Since then, the system's been completely overhauled, and now it much more accurately predicts general characteristics even if it appears less precise than before.
My point's kind of gotten away from me, but I guess it's just easy to see how people can fall into the trap of leaning on perceived reliability and using it to avoid making radical changes.
Recently we had to model a new system, and none of our numbers looked remotely right. Looking at the code, the whole program had essentially been turned into a lookup table that was completely useless at actually 'predicting' anything we hadn't seen before. Since then, the system's been completely overhauled, and now it much more accurately predicts general characteristics even if it appears less precise than before.
My point's kind of gotten away from me, but I guess it's just easy to see how people can fall into the trap of leaning on perceived reliability and using it to avoid making radical changes.