by Project for Excellence in Journalism
- Percent versus percentage: A drop in company profit from 15% to 10% is not a 5% change. It's a five percentage point change. The percent change between those two numbers is 33.3%. This mistake is among the most common and embarrassing.
- Margin of Error: Poll stories should mention the margin of error but too often bury it and even then fail to use it properly. The margin of error influences each number, not just one. Thus if two candidates are 5 points apart, and the margin of error is three, statistically they are tied, and the candidate trailing in the poll could be leading. How is that? The number for the leading candidate could be overstated by three points, and the number for the trailing candidate understated by three points. The guy who is "behind" could actually be ahead by one point, using the very numbers in that poll.
- Poll Momentum Swings: Sometimes in the quest for a good horserace story journalists can make too much of even small changes in the gap between candidates. Just because a candidate has picked up a point or lost two points doesn't mean he or she is "making a move" or "closing the gap" or "losing ground." Often when the point shift is small, the pollster will be nervous about ascribing a reason to the change. It could even be chance.
- Zero Sum Game: If Republicans hold an eight seat advantage in the state legislature, Democrats don't need to win nine seats to gain control. They need to win five seats. It is a zero sum game. A loss for one is a gain for the other. Think of two teams in a baseball pennant race when they are playing each other. If a team is one game behind, and defeats the team immediately ahead of it, it is now tied, since one team's victory is by definition the other team's loss.
- Interpretation is Everything: Sometimes, the same numbers can be used to make apparently opposing arguments and both sides are right. The key is watching your words and knowing what the numbers mean. A pollution report comes out saying 100 million Americans breathe polluted air. Scary. Another report using the same numbers says on most days, the air in every city in the U.S. is healthy. The two statements are based on the same numbers, and are both statistically correct. One hundred million Americans do breathe polluted air, but only occasionally. Read the wording around numbers carefully, and consider the source and the bias they might be pushing. As Bill Clinton said, everything can depend on what the meaning of is is.
