Science and health news verification guide

What to take away

• Translate the public claim back into the exact research question, population, comparison, outcome, and period.
• Read the paper, protocol or registration, supplement, and correction status, not only a press release.
• Match causal language to the study design and analysis.
• Report absolute numbers, uncertainty, harms, and study limits alongside headline findings.
• Verify product approval, indication, dose, and jurisdiction through the relevant regulator.

Science reporting is a chain of translations. Researchers turn observations into measurements, analyses, and a paper. Institutions may turn that paper into a release. Reporters turn both into public language. Each translation can drop a condition or widen a claim. Verification rebuilds the chain from the audience-facing sentence back to the evidence.

Start with a claim specification

Write the proposed claim as five fields before deciding whether it is true:

Field	Verification question
Population	Who or what was studied, and who does the story address?
Exposure or intervention	What treatment, behavior, condition, or measurement is at issue?
Comparison	Compared with what group, baseline, dose, or period?
Outcome	What was measured, by whom, and at what time?
Direction and size	Was the result higher, lower, associated, predictive, or causal, and by how much?

"A supplement protects the heart" may reduce to a much narrower result: adults already at high cardiovascular risk who received a specified dose had a lower level of one laboratory marker than a control group after eight weeks. The public sentence cannot outrun those fields.

Build a source packet

Obtain the most complete research record available:

1. the paper or preprint and its current version;
2. protocol, registration, and prespecified analysis plan;
3. supplementary tables, appendices, and underlying data when accessible;
4. funding and conflict disclosures;
5. journal corrections, expressions of concern, or retractions;
6. regulator documents for product-status claims; and
7. institutional release, author comments, and independent expert review.

A press release is useful evidence of what an institution says. It is not a substitute for the study. An abstract can omit subgroup rules, outcome changes, adverse events, missing data, and sensitivity analyses.

Identify what the study can answer

The National Institutes of Health guide to understanding clinical studies distinguishes randomized controlled trials, which can support cause-and-effect conclusions when well designed, from observational research, which is useful for identifying associations but cannot by itself prove cause. Ethical and practical limits sometimes make random assignment impossible, so the design must be judged against the question rather than a slogan about evidence rank.

Ask how participants or samples were selected, how comparison groups formed, whether investigators and participants were blinded, what was measured, and how missing observations were handled. Randomization does not repair a poor outcome measure, high attrition, selective reporting, or a sample too small for the question.

Compare the plan with the report

Registration and protocols can show whether the primary outcome, analysis window, eligibility criteria, or subgroup plan changed. A change is not automatically improper. It becomes a reporting problem when a later choice is presented as if it had been fixed in advance or when unfavorable prespecified results disappear.

Create a plan-versus-publication table:

Item	Planned	Reported	Explanation found?
Primary outcome
Measurement time
Sample size
Exclusions
Main model
Subgroups

Ask the authors or sponsor about material differences. Reflect a documented amendment accurately. Do not imply misconduct from an unexplained discrepancy alone.

Read results in absolute terms

Relative measures can make a small baseline difference sound large. If an outcome moves from 2 in 1,000 to 1 in 1,000, the relative reduction is 50 percent and the absolute reduction is 1 in 1,000. Both are mathematically correct; the second shows the scale needed for a decision.

For each main outcome, record:

• events and total participants in each group;
• absolute risk or mean value;
• absolute and relative difference;
• confidence interval or other uncertainty;
• follow-up duration;
• losses to follow-up; and
• adverse events or tradeoffs.

Do not turn "not statistically significant" into "no effect." It means the chosen analysis did not distinguish an effect under its data and assumptions. Likewise, statistical significance does not establish clinical importance.

Test alternative explanations

For observational findings, consider confounding, reverse causation, selection, recall, measurement error, missing data, and multiple testing. For experiments, inspect allocation, blinding, adherence, attrition, outcome switching, stopping rules, and whether analysis followed assigned groups.

Then ask whether the result has been replicated, whether other studies disagree, and whether a systematic review addresses the same population and outcome. One new study can change a body of evidence, but it rarely replaces that body by itself.

Verify the product and regulatory claim

"FDA registered," "FDA cleared," "FDA authorized," and "FDA approved" are not interchangeable. The relevant category, specific product, intended use, population, dose, and date matter. A clinical trial listing does not mean a treatment works, and publication does not create regulatory approval.

Use the regulator's product record or decision document. If the study examines an unapproved use of an approved product, state both facts. Avoid medical advice and distinguish reporting about evidence from advice for an individual patient.

Report conflicts without using them as a verdict

Funding, employment, patents, consulting, data access, and sponsor control can affect the risk of bias. They do not prove a result false. Explain who designed the study, controlled the data, performed the analysis, approved publication, and funded the work. Then evaluate the methods and evidence directly.

Write to the strongest supported level

Use a language ladder:

Evidence supports	Suitable wording
Descriptive difference	"was higher" or "was lower"
Statistical association	"was associated with"
Predictive model	"predicted within this sample"
Credible causal estimate	"reduced" or "increased," with scope
Early or uncertain evidence	"suggests," followed by the material limit

Before publication, compare the headline, deck, chart, caption, alt text, body, and social copy. A qualified body cannot repair an absolute headline.

Common questions

Does peer review prove a study is correct?

No. It is one form of scrutiny. Reporters still need to check methods, data, conflicts, corrections, and consistency with other evidence.

Can an observational study ever justify causal language?

Some rigorous observational designs can support causal inference, but the argument must address confounding, selection, timing, measurement, and credible alternatives. Do not infer cause from association alone.

Should a preprint be ignored?

No. Label its status, check its version and methods, seek independent review, and avoid presenting unsettled findings as established practice.

When should a health story include a medical warning?

Add a clear safety note when readers could change treatment, delay care, or use a product based on the story. Direct individual decisions to qualified clinicians and current regulator guidance.