Can You Tell if a Health Story Is Total BS?

If you feel like you've been reading about more "miracle cures" lately, it's because you probably are—fake news, after all, isn't just restricted to politics.

But fake health news is a different beast from the likes of Breitbart and US Uncut—its authors don't always intend to deceive. The stories tend to be exaggerated and uncritical, which can put unwitting readers at risk.

"The biggest concern is not so much stories that are out-and-out faked," says Ivan Oransky, co-founder of Retraction Watch, a blog that covers retracted scientific papers, and a writer in residence at New York University's Carter Journalism Institute. "Those exist and we should be concerned. But the larger problem is the stories that have a kernel of truth that is misinterpreted or blown out of proportion."

There's no question that fake news in general is popping up more than it used to. A Pew Research Center survey from December estimated that 23 percent of Americans had shared a fake news story.

But the problem might be even more rampant when it comes to health news. Sixty percent of the 2,319 stories reviewed on HealthNewsReview.org, a nonprofit site that assesses health coverage across a variety of outlets, have received a ranking of three stars or lower out of five stars, suggesting that there are significant flaws in the reporting. The same sort of technology that enables fake news and pressures journalists to do more work with less time for research and reporting are likely influencing health journalism, too.

There are also strong incentives to create misleading headlines, whether it's because it could make the story more popular, or because it makes ridiculous claims that prey on desperate people, Oransky says. Health news stories can also serve as marketing tools for new products and treatments, showing unsubstantiated bias.

And of course, fake health news can directly affect patients. Misleading stories can cause people to make up their minds about treatments before they even see a doctor, so doctors must spend valuable time debunking the misinformation, says Gary Schwitzer, the publisher of HealthNewsReview. Or patients might endanger their health by seeking alternative treatments and avoid going to the doctor at all.

Detecting which stories are exaggerated can be daunting if you're no expert. Here are the four most important things to consider when gauging whether or not you should believe the latest "breakthrough."

Check the label
Some publications are more likely than others to exaggerate health news than others (Health News Review has rated many of the major daily news outlets like the New York Times and Wall Street Journal). The scientific journal where the research was published matters, too. You want to make sure it's a highly-rated peer-reviewed journal (here's a list of them), and that it's the appropriate setting for the subject matter. A study published in an improper journal, or one you haven't heard of, might have ended up there because it didn't meet the stricter criteria of more elite journals.

For stories about a recent scientific study, it's ideal for a story to have at least two sources, one of whom participated in the research and one of whom is in the same field and can give context. You should also look at the credentials of who is quoted, and how. Did the quotes come from a press release? Is the piece about cervical cancer and the story quotes a podiatrist (sounds crazy, but it happens all the time)? Those could be indications that the implications of the study might have been misinterpreted.

Beware the animal study
Studies done in mice and in cell cultures in the lab are really important for research. But the outcomes of those studies shouldn't shape your health decisions. That's because—shocking, I know—you're not a mouse. And you're more complicated than a few hundred cells in a dish. Even if a drug looks really promising in a mouse, it's more likely than not to never make it to market (only 6 percent of drugs pass all three phases of clinical trials).

Control the spin
No treatment is perfect—there are always risks and side effects to any medication or intervention. If a story doesn't go into those, or the cost of that treatment, then it's not a complete discussion, Schwitzer says.

There are some buzzwords that might tip you off, Oransky adds. Things like "cure" or "breakthrough" or "paradigm shift" should only be used for true exceptions. "It's so rare those actually happen. If you see that [language], I wouldn't trust the story," he says.

Look at the numbers
How many people were in the study—19 or 19,000? A treatment that works on 19 carefully selected participants might not work for a huge cohort of people with different severities of disease or coexisting medical issues.

Schwitzer notes that it's also important to know the difference between absolute and relative risk. Absolute risk is going to be the number that is most useful; a pair of Dartmouth health policy professors have said that relative risk is like having a 50 percent off coupon for a department store, but you don't know if you can use it on diamonds or chewing gum.

Unfortunately, you'll probably have to calculate absolute risk yourself. (I know, I know—but sometimes crunching the numbers is important.) Consider this example from Health News Review: A study says that a new drug reduces the risk of heart attack by 50 percent. Sounds pretty good, right? But that's relative risk. To understand how good the drug really is, you need to consider the absolute risk.

If only two people out of an untreated control group of 100 people had heart attacks, and one of the 100 people medicated did, the absolute risk drops from 2 percent to 1 percent. "[The relative risk is] not a lie, it's just terribly misleading," Schwitzer says. Now that you know that only 1 percent of the entire cohort benefitted from the treatment, and that any drug is going to have costs—both monetary and in terms of side effects—it suddenly seems a lot less desirable.