How to audit a clinical study story

What to take away

• Freeze the public claim before reading supporting publicity.
• Match the paper with its protocol, trial registration, results record, supplement, and current journal status.
• Reconcile planned and reported outcomes, enrollment, exclusions, dates, and analysis groups.
• Recalculate the main effect in absolute as well as relative terms.
• Rewrite every claim that exceeds the design, data, or regulatory record.

This workflow is for a story about a clinical study involving people. It can be scaled for a short brief or a long investigation, but its order matters. Start with the claim, then reconstruct the study. Starting with promotional language makes it harder to see what has been omitted.

Step 1: capture the claim exactly

Save the headline, release, paper version, charts, author posts, and product page with dates. Copy the strongest sentence into a worksheet. Mark every verb and quantity that needs proof.

Example claim:

A daily tablet cut infections by 40 percent and is safe for older adults.

That sentence contains at least five claims: daily dosing, a defined tablet, a 40 percent reduction, a safety conclusion, and applicability to older adults.

Step 2: identify the study and version

Record the title, authors, journal or repository, DOI, publication status, publication date, and correction status. Check whether the version is a conference abstract, preprint, accepted manuscript, final article, correction, or retraction.

Do not merge separate studies that share a product name. A pilot, phase 2 trial, phase 3 trial, extension study, and pooled analysis answer different questions.

Step 3: locate the study record and protocol

Search the paper for a registration number. In the record, review sponsor, study type, phase, allocation, masking, enrollment, eligibility, intervention arms, primary outcomes, secondary outcomes, dates, locations, results, and changes.

ClinicalTrials.gov's guide to reading a study record identifies separate Study Details, Researcher View, Results, and Record History areas. A record can describe a study even when no results have been posted, so registration alone is not evidence that the intervention worked.

Save the earliest available registration and later versions. Note when each change occurred relative to enrollment, analysis, and publication.

Step 4: create a design card

Summarize the study without conclusions:

Item	What to record
Research question	Population, intervention, comparator, outcome, time
Design	Randomized, observational, crossover, single-arm, other
Recruitment	Source population and selection process
Assignment	Random method, concealment, or observed exposure
Masking	Participants, clinicians, assessors, analysts
Sample	Planned, enrolled, randomized, analyzed
Follow-up	Scheduled duration and actual completion
Primary outcome	Definition, measurement, and time point
Analysis	Main model and analysis population

If you cannot explain the design in a short paragraph, stop drafting the story and resolve the gaps.

Step 5: reconcile the participant flow

Follow people from screening through analysis. Ask why participants were excluded after assignment, whether losses differed between groups, and which safety population was used. A result among only participants who completed treatment may differ from the result under the original group assignments.

Create a flow table with screened, eligible, assigned, treated, followed, and analyzed counts. Check that totals match across the abstract, figure, supplement, and registration.

Step 6: compare planned and reported outcomes

For every primary and material secondary outcome, record:

• exact definition;
• measurement instrument;
• planned time point;
• registered analysis;
• reported result; and
• any amendment date and explanation.

Watch for a primary outcome that became secondary, a new time window, a favorable subgroup elevated after analysis, or a composite outcome described through only its most dramatic component. Changes can be justified, but the story must not erase them.

Step 7: reproduce the central number

For a binary outcome, capture events and participants in both groups. Calculate risks, absolute difference, relative difference, and number needed to treat or harm when appropriate. Keep the confidence interval.

Suppose infections occurred in 6 of 500 treatment participants and 10 of 500 controls:

Measure	Result
Treatment risk	1.2%
Control risk	2.0%
Absolute reduction	0.8 percentage points
Relative reduction	40%
Approximate number needed to treat	125

The 40 percent claim is arithmetically plausible, but the absolute result gives needed scale. The confidence interval determines how precisely the effect is known.

Step 8: audit harms and stopping

Record serious adverse events, withdrawals due to adverse events, common side effects, deaths, and the time available to observe delayed harm. Check whether the study stopped early and who made that decision. A trial designed for efficacy may be too small or brief to establish broad safety.

Replace "safe" with the observed safety result and its limits unless a larger evidence record supports the stronger word.

Step 9: test generalizability

Compare the study sample with the people named in the story. Age alone does not establish applicability to all older adults. Frailty, diagnoses, medications, pregnancy, geography, care setting, baseline risk, and access can matter.

Ask whether the trial measured a patient outcome, laboratory marker, imaging result, symptom score, or surrogate. A change in a marker is not automatically proof of longer life or fewer hospitalizations.

Step 10: verify product status separately

Check the relevant regulator's record for the exact product, manufacturer, formulation, intended use, age group, dose, route, and jurisdiction. Trial registration, an investigational authorization, publication, or a patent does not equal marketing approval.

If the study concerns an approved product used outside its approved indication, describe that distinction plainly. Do not give individualized treatment directions.

Step 11: seek independent review

Ask an expert without a close tie to the study to review the design, effect size, and main limitations. Give the expert the paper and protocol, not only the press release. Ask focused questions:

1. What can this design establish?
2. Is the primary result clinically meaningful?
3. Which bias or assumption most threatens the conclusion?
4. Does the result fit the wider evidence?
5. What should a patient not infer from it?

Step 12: run a claim-to-evidence audit

Create one row per public statement:

Draft claim	Best evidence	Supported level	Required revision
Cut infections by 40%	Outcome table	Relative result only	Add absolute values and interval
Is safe	Short trial safety table	Limited observed harms	Narrow and state follow-up
Works for older adults	Subgroup or eligibility	Depends on sample	Name included ages and exclusions

Repeat the audit for the headline, chart, caption, alt text, newsletter, and social post. Publish only after the strongest framing matches the narrowest relevant limitation.

Common questions

What if the paper has no registration number?

Ask the authors and journal. State the absence if registration was expected, and do not assume that reported outcomes were prespecified.

Is a primary endpoint always more important to patients?

No. It is the outcome designated for the study's main analysis. It may be a surrogate or scale that needs translation into patient relevance.

Can a reporter calculate a number the paper omitted?

Yes, if the inputs and method are sound. Label the newsroom calculation, preserve the worksheet, and seek review for material or complex results.

Does a listed study have government endorsement?

No. A registry record documents information submitted about a study. It does not by itself verify results, approval, or endorsement.