Which research can you believe?

Indeed there are a great many so called "studies" being referenced in print,  television, and radio, supporting just about every viewpoint and agenda imaginable. One way to increase confidence in a study's findings is to consider only those which are published in a "refereed" journal. These are publications whose articles are reviewed by professionals in the same field before the study is permitted to be published. They judge whether the study was conducted, and the results evaluated, using known and trusted processes. The most frequently referenced type of study is known as a correlation study.  Let us take a look at just what one of these can tell us, and just as important, what it cannot.


Generally, correlation studies attempt to show how frequent event "A" occurs in the presence of event "B" within a specified group or population. To make this more clear, here are some sample questions that might be answered with a correlation study.



Research tells us that for the first question, there is a high degree of negative correlation between a high school senior's grades and his or her use of illegal drugs. That is to say, the higher the scholastic performance of a senior, the less likely he or she is to also use drugs.


A good guess for the other question would be that there is a high positive correlation between the number of years a U.S. citizen drives and the age at death. In other words, the older a person is when he or she dies, the more likely it is that the person has a long history of driving.


Neither of these results come as a surprise. What is surprising, however, is the use to which they are put by undiscerning teachers, counselors, authors, and media personalities.


For the first question, it is quite common for advocates of better education to exclaim: "See, I told you so! If we could get are kids to do better in school, they would not use drugs as much." For those campaigning for stronger drug enforcement, they would exclaim: "See, I told you so! If we could get drugs out of the hands of our youth, their grades would improve." From a scientific standpoint, that correlation study does not tell us much at all as to  if or how one event "causes" the other.


To show how ridiculous those kinds of conclusions from a correlation study can be, take a look at the second question.  Here we see that the older a person is when he or she dies, the more likely he or she is to have many years of experience driving an automobile. The obvious(?) conclusion is either that driving a great many years results in you dying at an older age or, even more ridiculous, that dying at an older age makes you a more experienced driver.


Counselors with weak education backgrounds in psychology or professional counseling are frequently guilty of misusing statistics to persuade a client to alter a behavior. A well meaning counselor might, for example, tell a client not to move in with his or her lover prior to marriage because the statistics show that doing so will put them at higher risk of divorce. It is true that those who live together prior to marriage have a significantly higher divorce rate than those who do not, but to say that living together "causes" a higher risk of divorce is not only questionable, it directs attention away from the factor that is responsible for the higher risk so that the client is not helped by confronting the root issue.


Despite the error of misusing correlation studies to prove one variable causes another, the race goes on to inundate the public with quick and easy correlation studies to get the consumer to support one thing or another, to vote one way or another, to buy a certain product or food, to make a change in lifestyle, or just plain get alarmed over something. The more we learn about the science of research, the less we will be emotionally whipped about by zealous folks with an agenda. To that end I suggest reading a serious book on statistical research such as Fundamentals of Educational Research by Thomas K Crowl or, on the lighter side, Statistics for Dummies.