Tuesday, April 28, 2015

Making sense of the data mess

I was researching my son's local middle school after we moved to get an idea of how well it measured up to area schools. When I clicked on the link that led to the school report card, I gasped. Up came that mediocre "B-" grade glaring at me. I was horrified, and felt like a complete failure for moving him from an "A" grade school environment to a substandard one.

After more digging, I realized that the middle schools were not using the same criteria for rating. The "A" schools were naturally reporting their top scores, and the new school was including all its tests - the best and worst. I was to later come across another report that ranked the entire state's schools with a very low score, and then read today that the state wasn't fairing as bad  at all.

Same goes for just about any other type of measurement I come across, especially those ranked by surveys taken from well-known groups that try to make sense of the world through statistics and percentages. Just Google something as simple as "retirement fund" and you receive a host of equally favorable and unfavorable recommendations in one search. Heck, I've come across conflicting data in a single article on the same topic.

It's frustrating, especially with the limited time and energy the average person has these days. Now imagine trying to stay updated on career trends when you're exposed to so much contradiction.

How do you make decisions when the data you're using doesn't tie up?

  • Credible sources. First, be sure the material you're reading is reliable. Also try to understand the purpose of any study or research work you're looking at. In the example of the retirement fund, be sure the material you're reading isn't distributed by a company with a vested interest in any type of investment. With studies that quote percentages, dig further to determine who and how many made up the population researched.
  • Read the fine print. Before relying on any type of data, you want to understand what makes up the body of information. What was the purpose of the study, were the numbers quoted right, and did the sources you're reading interpret the results right.
  • Compare the info. If you come across two different sources of the same topic giving different ranks, compare the numbers and the topic being ranked. Chances are, you'll find that although the subjects sound similar, there is a difference in either the way the ranking was determined or how the subject was being rated.
  • Drill down to the original source. If you're reading an article that is quoting a survey, click on the link to read the material yourself. All you need to do is read the summary that accompanies most research to interpret the material on your own. Interpretation plays a large part in how results are conveyed.
  • Take the middle road. Research is not a perfect science and if you scroll down to the footnotes, you'll always find a margin of error listed there. The purpose of any study is to make sense of the world and its numerous issues through percentages, but those ratios do rely on the population being interviewed and how honest the researcher and respondents are. A recent report just exposed a slew of scientific surveys for unethical results, although they were quoted by just about every media and academic outlet available. To make a good decision, do your research and select two to three sources to rely on. Use your common sense, experience, and reasoning, and then trust yourself enough to make the right decision.

No comments:

Post a Comment