This debate has been going on for years in advertising. There is a cultural gulf between direct response advertising people and general advertising people. Direct response people know to the penny how well different adverts work because it is intrinsic to their medium. As a result, there exists a set of near-inviolable laws on how to run a direct response campaign that works. Long TV slots always work better than short ones, long copy in print works better than short copy, simple illustrations work better than clever art and so on. These rules almost directly contradict what is considered good by general advertising people, who rarely ever try to measure the effect of their advertising on the bottom line.
As business people, we have two choices. We can trust the taste and intuition of designers and hope that their vision is what the market demands. Or, we can test and measure relentlessly, seeking always to measure what matters - whether a decision will be profitable or not. It is entirely plausible that a test-driven environment may miss out on sparks of creative genius that happen to be utterly untestable. It is also possible that there is an invisible teapot orbiting the sun. I factor neither possibility into my decisions because I cannot possibly make rational decisions on that basis.
It has never been cheaper or easier to test. Rapid prototyping means that even complex hardware can be produced for testing at a cost which pales in comparison to the tooling for a mass-production run. Just In Time techniques and electronic point of sale systems allow brick-and-mortar retailers to test endless permutations of merchandising. A business without systematic testing is a ship with no compass. Gut instinct is valuable and laudable, but to fail to confirm it empirically is utterly foolish when the costs of testing are so small relative to the costs of even the most minor failure.
Changes in sales? Surveys of brand awareness? I'm not a traditional marketing guy but I doubt there aren't techniques that can be used to at least gauge the response to a campaign. They won't be direct necessarily but they must be there!
As business people, we have two choices. We can trust the taste and intuition of designers and hope that their vision is what the market demands. Or, we can test and measure relentlessly, seeking always to measure what matters - whether a decision will be profitable or not. It is entirely plausible that a test-driven environment may miss out on sparks of creative genius that happen to be utterly untestable. It is also possible that there is an invisible teapot orbiting the sun. I factor neither possibility into my decisions because I cannot possibly make rational decisions on that basis.
It has never been cheaper or easier to test. Rapid prototyping means that even complex hardware can be produced for testing at a cost which pales in comparison to the tooling for a mass-production run. Just In Time techniques and electronic point of sale systems allow brick-and-mortar retailers to test endless permutations of merchandising. A business without systematic testing is a ship with no compass. Gut instinct is valuable and laudable, but to fail to confirm it empirically is utterly foolish when the costs of testing are so small relative to the costs of even the most minor failure.