Last week Nate Silver ranked Gallup as the worst polling firm when it came to accurately predicting the 2012 election results. Gallup’s editor fired back on Friday.
We have a reverse law of the commons with polls. It’s not easy nor cheap to conduct traditional random sample polls. It’s much easier, cheaper, and mostly less risky to focus on aggregating and analyzing others’ polls. Organizations that traditionally go to the expense and effort to conduct individual polls could, in theory, decide to put their efforts into aggregation and statistical analyses of other people’s polls in the next election cycle and cut out their own polling. If many organizations make this seemingly rational decision, we could quickly be in a situation in which there are fewer and fewer polls left to aggregate and put into statistical models. Many individual rational decisions could result in a loss for the collective interest of those interested in public opinion. This will develop into a significant issue for the industry going forward.
Silver responded on Twitter by saying that Gallup “needs to hire better statisticians.”