Avoiding Bad Data (Baseball Analogy)

When you are already working with a somewhat small sample, you need to be careful about the data you receive. The purpose of these data is to draw conclusions that will help you make decisions about your business. But when the sample is small, the conclusions you draw may not be accurate.

Baseball is a sport of statistics. How a player hits, fields, and pitches are all closely monitored, and in addition to basic stats (ERA, RBI, etc.) there are also a variety of highly advanced statistics designed to provide accurate measurements of a player’s true skill.

But like all data, baseball statistics is highly prone to sample size fluctuations, which is why one month of data, for example, is simply not enough to draw any conclusions at all. While fans can be concerned (or pleased) with how a player is performing, the sample size for the data itself is so meaningless that it becomes useless – literally useless. If teams made decisions about one month of data, the following would have occurred: David Ortiz would have been dropped from the Boston Red Sox roster.

Matt Holliday would have been dropped from the Oakland Athletics roster. Troy Tulowitzki would have been dropped from the Colorado Rockies roster. Josh Hamilton would have been dropped from the Texas Rangers roster.

If you haven’t followed baseball, the above 4 players are arguably some of the best hitters in the game today, and were just as skilled in 2009. But like all baseball players, they are prone to sample size variations, and those variations cause the data itself to be almost completely meaningless – with some data being quite literally useless. People like to believe that all data can be an indication of something, but there are simply times that the data has no meaning, and in baseball, that occurs often when you are looking at only small samples.

Baseball is one of the most common places where people misread information within the data, but it is certainly not the only one. This occurs within the market research/employee satisfaction/customer satisfaction realm too, sometimes without the researcher even realizing that they are over-analyzing the data. In the next article, we’ll look at how misreading the data occurs, and what can be done to prevent it from happening to your company. Related Blog Part 2