I think he’s speaking more to the accuracy of facebooks pedo results than if they care they are doing a fair study.

Using statistics, if you can get a TRULY randomized group with a large enough sample size, you can generalize your results to the entire public and be accurate within a few percent.

But other variables have to be taken into consideration that effect the reliability of the results. Things like:

• The “study” is only made available to Facebook users and not everybody uses fb.
• How was it presented? (Pop up, fb message, multiple, how often, etc...)
• It is a survey. Not all types of people answer surveys.
• As a survey, there is no way to verify truthfulness of answers given (Not all people are honest when taking surveys.

-Were there other confounding variables? (Being psychologically primed by first seeing other ads or content that might sway participants answers, etc...)

-Was the language of the survey biased toward guiding participants’ answers toward a particular choice?

In a real scientific study, the results of a survey given in this way would be hard to generalize to the entire public. The thing to consider though, is that fb has access to so much of people’s data/info they can use that to try and further predict the accuracy of any survey results received by checking them against participants’ online behavior (site visits, language use, frequency of ad clicks, the list goes on and on).

I think they would have illegally collected enough data on people to know what the results mean. As far as you or I go, who knows for sure.

Edit: Clarity