How to Verify the Science You Read

3 Steps to Take to Confirm the Claims

Man reading tablet at work

A colleague asked for my assessment of an alleged study of veganism. The “alleged” is important. What he found was a report on a media website with reference to a study that does not exist.

The report made all sorts of brash claims about the study, which purportedly demonstrated harms of veganism over the long term, but importantly never named the study, or the journal in which it was published (the report did say, explicitly, that it was a “published study”).

The report did name the “doctor” responsible for the study, but when I did a Google search of his name, it produced just one URL: the site on which I started. No real person, with the possible exception of inveterate hermits in the Himalayas, are that invisible to Google. The name was obviously just made up; a fiction.

A search for the study was just as fruitless as the search for the author. There is no such person; there was no such study. And, just in case you are wondering, there certainly are robust studies of veganism, both the short and long-term effects, notably the famous Adventist Health Studies out of Loma Linda University. Google that, and you retrieve 445,000 sites, give or take, including links to publications in prestigious peer-reviewed journals, and highly accomplished researchers. These and other studies on the topic show overwhelmingly, by the way, benefits, not harms, of veganism.

But this is not a column about veganism; it is a column about science and sense; truth and lies; trust, and verification.

Confirming the Claims

Ronald Reagan famously said: “trust, but verify.” I channeled that laudable sentiment, in a language of my own invention, in a fantasy adventure novel I wrote as: “ahpval nis miftanib fdalu.” That translates to: “keep the faith, but get the data.”

These days, I am thinking we should skip the faith, and look for the data right away.

Why? Because these are days of a post-truth era. Permission is being granted by example from the most powerful office in the land to refute established fact about everything from climate change to crowd size- and simply lie. Let’s be clear, there are no alternative facts, there are only alternatives to facts. There is distortion, deceit, and maybe delusion; but facts are facts, and alternatives to them are false.

There is nothing partisan about objective reality; it simply is what it is. Vaccines do not cause autism, and do save lives. So says epidemiology, not ideology. Reality, and the measles, don’t care what we believe.

If by any chance you are among those who harbor doubts about the scientific method, I invite you to reflect on how these thoughts of mine have reached you. I thought of these words, and relayed them to the keypad of a computer on my desk here in Connecticut, a beneficiary of both computer hardware and software, products of engineering. Then, the column was uploaded to a website and reached you via the practical magic of the web—the transmission of organized electrons through cyberspace.

The reliable power of science is on display to us all every day in the most mundane of our activities—like reading on-line, or texting, or sending pictures out on Instagram.

Remember the adage, “familiarity breeds contempt?” That seems to capture our modern relationship to science. We make such routine use of the incredible, consistent power of science that we take it entirely for granted, and allow ourselves to be dismissive of it.

That’s a grave mistake. The very same reliable methods that allow us to communicate this way, send one another selfies, or fly from New York to LA, are responsible for what we know about vaccines and veganism, climate change, and cleanses.

Science is reliable.

3 Steps to Verifying What You Read

But there is a fly in this ointment. The same practical magic that gives you access to information on the web gives you access to misinformation, and now more than ever. Deceit is in vogue, and anyone can feign expertise.

That places the burden on each of us to differentiate baby from bathwater. Here are my tips for doing just that.

1. Find the study. Any legitimate report invoking a study should name, and preferably link to, the study in question. If you can’t find the study, it’s a red flag.

2. If you find the study, identify the journal and authors. If the journal is among the more reputable, there is a good chance you will have heard of it before. If not, check the journal site to see if it is peer-reviewed, a requirement for legitimate science. The authors should have relevant credentials, and generally, links to universities. The absence of any of these quality indicators is cause for doubt.

3. Check for replication. Contrary to the insinuations of our every-study-is-a-breakthrough culture, science is incremental. That doesn’t suit the interests of our 24/7 news cycles, but it’s the truth. Any one study makes only a modest contribution to what we know reliably, and we are not sure about findings until they prove consistent. If you have particular interest in a study finding, do a search of that general topic to see if it has been replicated. Value the overall weight of evidence over any one study. An isolated finding is a curiosity; reserve your convictions for findings that recur, and stand the test of time.

You obviously don’t have to go through the above steps every time you read about science. Sometimes the topic won’t be of sufficient interest, and sometimes it will be clear right away that the story is legitimate. But when the topic resonates, and the validity is uncertain, don’t trust—go ahead and verify.

There is no reason to go it alone; you can get the help of trusted friends in knowing what to trust. That’s the very purpose of a site like Verywell, where content is reviewed for accuracy. You may find some differences of opinion here, but no differences of fact. If you see mention of a published study here, you can be sure the study is real.

Science has taken us from a flat earth to the surface of the moon and the outer reaches of our solar system. The practical manifestations of science empower us all every day.  It would be a shame, and a mistake, to allow that familiarity to foster contempt rather than respect.

Respect science by challenging it. Verify, don’t just trust. Real facts will withstand the scrutiny; alternatives to facts will wither under it.

Continue Reading