Which research can you believe?
Indeed there are a great many so called "studies" being referenced in print, television, and radio, supporting just about every viewpoint and agenda imaginable. One way to increase confidence in a study's findings is to consider only those which are published in a "refereed" journal. These are publications whose articles are reviewed by professionals in the same field before the study is permitted to be published. They judge whether the study was conducted, and the results evaluated, using known and trusted processes. The most commonly referenced type of studies are known as correlation studies. Let us take a look at just what these can tell us, and just as important, what they cannot.
Generally, correlation studies attempt to show how frequent event "A" occurs in the presence of event "B" within a specified group or population. To make this more clear, here are some sample questions that might be answered with a correlation study.
- How does the scholastic achievement level (event "A") correlate with the use of drugs (event "B") among high school seniors (group or population)?
or- How does the number of years driving an automobile (event "A:) correlate to the age of death (event "B") among U.S. citizens (group or population)?
Research tells us that for question #1, there is a high degree of negative correlation between a high school senior's grades and his or her use of illegal drugs. That is to say, the higher the scholastic performance of a senior, the less likely he or she is to also use drugs.
A good guess for question #2 would be that there is a high positive correlation between the number of years a U.S. citizen drives and the age at death. In other words, the older a person is when he or she dies, the more likely it is that the person has a long history of driving.
Neither of these results come as a surprise. What is surprising, however, is the use to which they are put by unknowledgeable parents, counselors, authors, and media personalities.
For #1, it is quite common for advocates of better education to exclaim: "See, I told you so! If we could get are kids to do better in school, they would not use drugs as much." For those campaigning for stronger drug enforcement, they would exclaim: "See, I told you so! If we could get drugs out of the hands of our youth, their grades would improve." But from a scientific standpoint, that correlation study does not tell us much at all as to how one event "causes" the other.
To show how ridiculous those kinds of conclusions from a correlation study can be, take a look at #2. Here we show that the older a person is when he or she dies, the more likely he or she is to have many years of experience driving an automobile. The obvious(?) conclusion is either that driving a great many years results in your dying at an old age or, even more ridiculous, that dying at an old age makes you a more experienced driver.
Counselors with weak education backgrounds in psychology or professional counseling are frequently guilty of misusing statistics to persuade a client to alter a behavior. A well meaning counselor might, for example, tell a client not to move in with his or her lover prior to marriage because the statistics show doing so will put them at higher risk of divorce. It is true that those who live together prior to marriage have a higher divorce rate than those who do not, but to say that living together "causes" or "will result in" a higher risk of divorce is not only untrue, it directs attention away from the factor causing the higher risk so that the client is not helped by confronting the root issue. In this instance, the root issue is likely to be a weak regard for marriage in general, or it could be that the client does not see him or herself as the determiner of success of a marriage but thinks the other person, or some other factor, is more influential.
Despite the error of using correlation studies to prove one measured variable causes another, the race goes on to inundate the public with quick and easy correlation studies to get the consumer to support one thing or another, to vote one way or another, to buy a certain product or food, to make a change in lifestyle, or just plain get alarmed over something and thus "stay tuned" to the media for further information. The more we learn about the science of research, the less we will be emotionally whipped about through the misuse of its results by zealous folks with an agenda. To that end I recommend a book Fundamentals of Educational Research by Thomas K Crowl. Parts of it are very simply stated and, for those more serious inquirers, parts are quite comprehensive.
by Charles Jennings, M.C., LPC