You can now read 10 articles in a month for free on BostonGlobe.com. Read as much as you want anywhere and anytime for just 99¢.

The Boston Globe

Health & wellness

Daily Dose

How much can you trust dramatic study findings?

An analysis published last week in the Journal of the American Medical Association examined more than 228,000 clinical trials and found that about 9 percent of them had “very large” findings — showing that a particular treatment was five times or 500 percent more likely to help someone or hurt them compared with a placebo.

These were mostly from small studies with very few measured outcomes such as heart attacks or deaths. If 10 heart attacks occurred in one group and five heart attacks in another, for example, the researchers would have found a five-fold difference between the two groups. But the laws of statistics dictate that smaller samples have larger margins of error.

Continue reading below

What’s more, these small studies with dramatic findings were often the first one s of their kind to be published, and repeat studies typically found more modest effects. And almost none of the studies found large differences when comparing the number of deaths between two groups — that outcome is considered the most important measure of how well a medical treatment works or how much something harms.

Bottom line: “Genuine large effects with extensive support from substantial evidence appear to be rare in medicine,” wrote the study authors. That means you should take any study showing something for the first time with a grain of salt — especially those that have dramatic findings. D.K.

Loading comments...

You have reached the limit of 10 free articles in a month

Stay informed with unlimited access to Boston’s trusted news source.

  • High-quality journalism from the region’s largest newsroom
  • Convenient access across all of your devices
  • Today’s Headlines daily newsletter
  • Subscriber-only access to exclusive offers, events, contests, eBooks, and more
  • Less than 25¢ a week