Science health news verification: a practical guide

What to take away

Science coverage is a chain of translations. A researcher turns observations into measurements and a paper. An institution turns the paper into a release.

A reporter turns both into a sentence a reader will repeat at dinner. Each handoff can drop a condition or widen a claim.

Verification walks the chain backward, from the public sentence to the raw evidence.

Write the claim as five fields first

Before you judge whether a claim is true, write down what it actually asserts. Pin down each field:

Field	What to pin down

"A supplement protects the heart" usually collapses into something far narrower: adults already at high cardiovascular risk, given a stated dose, showed a lower level of one laboratory marker than a control group after eight weeks. That is a real finding about a marker.

For a claim without a paper attached, map each phrase to one of those fields: identify the population, exposure, comparison and measured outcome, then write the reported direction and size. Mark anything the claim adds that the study did not test—for example, a change in a laboratory marker does not establish protection from a heart attack. The claim mapping guide can serve as a companion to this check.

Assemble the source packet

Pull the fullest research record that exists, in this order:

For an approval claim, add the regulator's product record after the research records. Then read the institutional release and independent expert comment against the paper: a release is evidence of what an institution chose to say, not a substitute for the study. An abstract can hide changed outcomes, adverse events, missing data and sensitivity analyses.

Match the design to the question

The National Library of Medicine's chapter on formal trials versus observational studies sets the designs side by side: randomized trials assign an intervention, while observational studies examine exposures as they occur. Randomized placebo-controlled trials remain the standard for judging whether a therapy is safe and effective, but they answer narrow questions.

Observational work can overestimate treatment effects. It is also the only option where randomizing people is impossible, and a well-built cohort can answer questions no trial will. Judge the design against the question, not against a slogan about evidence rank.

Ask how participants were selected, how comparison groups formed, whether investigators and participants were blinded, and how missing observations were handled. Randomization does not repair a weak outcome measure, heavy attrition or a sample too small for the question.

Compare the plan with the report

Registration records let you see whether the primary outcome, analysis window, eligibility rules or subgroup plan changed between design and publication. Compare the planned and reported versions of each item, along with the analysis method, using the registry history, paper and supplement. A change is not automatically improper. It becomes a reporting problem when a later choice is described as if it had been fixed in advance, or when an unfavorable prespecified result quietly disappears.

Item	Planned	Reported	Explanation found?

Ask the authors or sponsor about material differences and report a documented amendment accurately. An unexplained discrepancy is a question, not proof of misconduct.

Read the result in absolute terms

Relative measures flatter small differences. Move an outcome from 2 in 1,000 to 1 in 1,000 and the relative reduction is 50 percent while the absolute reduction is 1 in 1,000. Both numbers are correct. Only the second shows the scale a patient or a policymaker is deciding on.

For each main outcome, record the events and totals in each group. Record the absolute risk or mean value. Record the absolute and relative difference. Record the confidence interval. Record the follow-up period. Record losses to follow-up. Record adverse events.

"Not statistically significant" does not mean "no effect." It means the chosen analysis did not separate an effect from noise under its own assumptions. Significance, in the other direction, does not establish clinical importance.

Test the alternative explanations

For observational findings, work through confounding, reverse causation, selection, and recall. Also consider measurement error, missing data, and multiple testing. For trials, inspect allocation, blinding, adherence, and attrition. Also check outcome switching, stopping rules, and whether analysis followed the groups as assigned.

Then ask whether anyone has replicated it, whether other studies disagree, and whether a systematic review covers the same population and outcome. One new study can shift a body of evidence.

It rarely replaces one by itself.

A case report describes an individual patient's experience; an observational study examines exposures without assigning them; and a randomized trial estimates outcomes after assigning an intervention. A systematic review synthesizes studies that meet defined criteria, while a meta-analysis statistically combines results from multiple studies. These designs answer different questions, so a case report and a meta-analysis should not be presented as if they carry the same weight. The evidence types comparison offers a companion overview.

Check the regulatory status word by word

"FDA registered," "FDA cleared," "FDA authorized" and "FDA approved" are four different things. Each carries its own product record, intended use, population, dose and date. A trial listing on ClinicalTrials.gov does not mean a treatment works, and publication in a journal does not create an approval.

Go to the regulator's decision document rather than the company's press page. For drugs, check Drugs@FDA; for devices, look for the applicable 510(k), De Novo or Premarket Approval (PMA) record, and check the FDA's Emergency Use Authorization records for an EUA. If the study tests an unapproved use of an approved product, say both things in the same sentence. The clinical study story audit covers how to keep the trial, the label and the marketing claim from blending together.

Report funding without treating it as a verdict

Funding, employment, patents, consulting, data access and sponsor control all bear on the risk of bias. None of them proves a result false. Establish who designed the study, who held the data, who ran the analysis, who approved the manuscript, and who paid.

Then evaluate the methods on their own terms, and let readers see the evidence rather than the scoreboard. A Journalist's Resource summary of research on framing scientific errors and trust in science discusses how presenting errors as part of self-correction can support public trust, and notes that American science reporting has leaned toward a broken-or-in-crisis frame.

Write to the strongest supported level

Before publication, read the headline, the deck, the chart caption and the body copy as one document. A qualified body cannot repair an absolute headline, and a chart caption is often the only text a social reader sees. Match the wording to what the evidence supports:

Common questions

Does peer review prove a study is correct?

No. It is one round of scrutiny by a few people, done before the data are public. Check the methods, the conflicts, the correction record and whether other studies agree before you treat a finding as settled.

Can an observational study justify causal language?

Sometimes. Some cohort and natural-experiment designs support causal inference when the analysis addresses confounding, selection, timing and measurement, and rules out credible alternatives. Association alone never gets you there.

Should a preprint be ignored?

No, but label it. Say it has not been peer reviewed, check which version you are reading, and look for independent comment. Do not present unsettled findings as established practice.

When does a health story need a safety note?

When a reader could change treatment, delay care or buy a product because of the story. Point individual decisions to a licensed clinician and the current regulator guidance, and keep the reporting on the evidence itself.