1. Do you expect the opt-in nature of these studies to impact their findings?
2. To compensate for the voluntary nature of the studies, do you think researchers in general will still be incentivized to find data sources that are less respectful of people's privacy and don't require an opt-in to the study?
> 1. Do you expect the opt-in nature of these studies to impact their findings?
The Rally participant population is not representative of the U.S. population—these are users who run Firefox (other browsers coming soon), choose to join Rally, and choose to join a study. In research jargon, there's significant sampling bias.
For some studies, that's OK, because the research doesn't depend on a representative sample. For other studies, researchers can approximate U.S. population demographics. When a user joins Rally, they can optionally provide demographic information. Researchers can then use the demographics with reweighting, matching, subsampling, and similar methods to approximate a representative population. Those methods already appear throughout social science; whether they're sufficient also depends on the study.
> 2. To compensate for the voluntary nature of the studies, do you think researchers in general will still be incentivized to find data sources that are less respectful of people's privacy and don't require an opt-in to the study?
Rally is designed to provide a new research capability that didn't exist before. I don't expect a substitution effect like that.
Regarding 2. that would run afoul of many ethics boards at universities. Generally they require that (informed) consent has been given to take part in the study.
A few follow up questions:
1. Do you expect the opt-in nature of these studies to impact their findings?
2. To compensate for the voluntary nature of the studies, do you think researchers in general will still be incentivized to find data sources that are less respectful of people's privacy and don't require an opt-in to the study?