Thanks for the interest, and sorry for the delayed response.
We certainly hope that the surveys are not annoying. We know that some people will prefer to pay, but others (like me) would rather take a few seconds to complete a survey than pay. Our hope is that the surveywalls will let users access content that otherwise would have been beyond their reach. And at the same time, quality content publishers will be able to make money off their work.
As to the researcher side, we agree with you that nothing beats revealed user behavior when optimizing web sites or apps. But it's not cheap or even possible to A/B test in other situations.
Suppose that you're a restaurant owner and want a new sign for your building. It's not practical to purchase two signs and see how alternating the signs affects business on different days. And it's also not in budget to spend $10k or more on a traditional market research survey.
Or if you're a politician, you can not wait until voting day to see which of your various ad campaigns worked in different districts. You need proxy measurements.
Companies already ask these types of questions using traditional approaches like panels and phone polling. We provide a cheaper way for them to do it online. All approaches have built in biases, and we're working to account for the biases and quality issues in online polls. As you point out, we'll have to do that to succeed.
Thanks for responding to my, as noted by others, rather "rude" critique. I think you guys are building something distinctive from the surveys on AmazonTurk, SocialSci, because your recruitment poll is not chosen. Turkers went to amazon with the idea to properly do survey, your survey might took the user by surprise.
A few suggestions:
1. Hire someone with experience on experimental psychology, I mean, by trade. Not necessarily a PhD but at least with some experience and has knowledge of the textbook experiment errors. Just one of those would help the design a long way.
2. Throw in some free analytics tools/result for the organization behind the survey. As you mentioned, restaurant owners and politicians might just want to run the survey for some straightforward result, well, not all surveys have straightforward result, even when designed well. Some analytics/visualization tool, e.g., manyeyes from IBM, doesn't really take a lot of dev time but would be quite impressive in the eyes of your customers (i.e., the politicians or restaurant owners).
Thanks again, Alex. I appreciate the comments and great ideas.
We do plan to add polling and survey design experts to our team (I have stats but not polling background). And an analytics platform is coming.
What do you mean by our survey "might take the user by surprise"? Do you mean that it might be surprising to the user to have a survey launch when clicking one of our links? We're testing different "teaser" text for the links. And hopefully we can make it clear that a survey will be coming when you click.
If you're willing to run a test survey, fill out the contact us form on our site, and we'll give you a discount code. Appreciate the feedback.
We certainly hope that the surveys are not annoying. We know that some people will prefer to pay, but others (like me) would rather take a few seconds to complete a survey than pay. Our hope is that the surveywalls will let users access content that otherwise would have been beyond their reach. And at the same time, quality content publishers will be able to make money off their work.
As to the researcher side, we agree with you that nothing beats revealed user behavior when optimizing web sites or apps. But it's not cheap or even possible to A/B test in other situations.
Suppose that you're a restaurant owner and want a new sign for your building. It's not practical to purchase two signs and see how alternating the signs affects business on different days. And it's also not in budget to spend $10k or more on a traditional market research survey.
Or if you're a politician, you can not wait until voting day to see which of your various ad campaigns worked in different districts. You need proxy measurements.
Companies already ask these types of questions using traditional approaches like panels and phone polling. We provide a cheaper way for them to do it online. All approaches have built in biases, and we're working to account for the biases and quality issues in online polls. As you point out, we'll have to do that to succeed.