I've done some reading on split testing, where you show two different designs and analyze the results over a period of time to determine which design performs the best. Has anyone here done anything like this and if so can you elaborate on your findings? If not, I'm going to try it with a new product we're launching in about 2 weeks (hopefully!)
Anyone who considers themselves a professional designer or user experience expert has done this. Google Website Optimizer makes it easy for anyone to do, and there are more advanced systems like Omniture's offerings that provide more options and more data on the results.
This is how a lot of "best practices" are determined. I know I've found out changing button colors or icons can increase conversion rates on email newsletters and changing a sign up process can vastly decrease the number of people who bailout at some point. Testing designs is very important.
I am guessing if 37signals conversion rate dropped significantly, they'd redesign the page again - it's kind of like a rudimentary way of split testing :)
