How to audit a clinical study story before it runs

What to take away

This workflow is for a story about a clinical study involving people. It scales from a short brief to a long investigation, but the order matters. Start with the claim, then rebuild the study underneath it. Promotional language read first makes the omissions harder to see.

Capture the claim exactly

Save these items, each with its capture date:

Take this sentence:

A daily tablet cut infections by 40 percent and is safe for older adults.

It carries five separate claims: daily dosing, a defined tablet, a 40 percent reduction, a safety conclusion, and applicability to older adults. Each one gets its own row.

Identify the study and its version

Record the title, authors, journal or repository, DOI, and publication status. Also record the publication date and correction status. Then decide which version you hold: conference abstract, preprint, accepted manuscript, final article, correction or retraction.

Do not merge studies that share a product name. A pilot, a phase 2 trial, a phase 3 trial, an extension study and a pooled analysis answer different questions.

Find the registry record and protocol

Search the paper for a registration number. In the record, review the sponsor, study type, phase, and allocation. Also review masking, enrollment, eligibility, and arms. Then review outcomes, dates, locations, and results.

ClinicalTrials.gov's guide to reading a study record separates Study Details, Researcher View, Results and Record History. A record can describe a study with no results posted, so registration alone proves nothing about whether the intervention worked.

Save the earliest registration you can retrieve and each later version. Note when each change landed relative to enrollment, analysis and publication.

Build a design card

Summarize the study with no conclusions in it.

If you cannot explain the design in a short paragraph, stop drafting and resolve the gaps first.

Reconcile the participant flow

Follow people from screening to analysis. Ask why anyone was excluded after assignment, whether losses differed between arms, and which safety population was used. A result among only those who completed treatment can differ from the result under the original group assignments.

Build a flow table with screened, eligible, assigned, treated, followed and analyzed counts. Check that the totals agree across the abstract, the flow figure, the supplement and the registry.

Compare planned with reported outcomes

For each primary outcome and each material secondary outcome, record the exact definition, the measurement instrument, the planned time point, the registered analysis and the reported result. Add any amendment date and its stated reason.

Watch for a primary outcome that became secondary, a new time window, a favorable subgroup promoted after analysis, or a composite outcome described through only its most dramatic component. Changes can be legitimate. The story still has to show them.

Reproduce the central number

For a binary outcome, capture the events and the participants in both arms. Calculate the risk in each arm, the absolute difference, the relative difference, and the number needed to treat or harm where that measure fits. Keep the confidence interval beside the point estimate.

Worked example

Suppose infections occurred in 6 of 500 treatment participants and 10 of 500 controls.

The 40 percent figure is arithmetically sound. The absolute result tells the reader what it means in a population. The interval tells them how precisely the effect is known.

Audit harms and early stopping

Record serious adverse events, withdrawals for adverse events, common side effects and deaths. Note how long the study could observe delayed harm. Check whether the trial stopped early and who made that call. A trial powered for efficacy is often too small or too brief to settle safety.

Unless a wider body of evidence supports the stronger word, replace "safe" with the observed safety result and its limits.

Test generalizability

Compare the study sample with the people named in the story. Age alone does not carry a result to all older adults. Frailty, diagnoses, other medications, and pregnancy all move the answer. Geography, care setting, baseline risk, and access also move the answer.

Ask what the trial actually measured: a patient outcome, a laboratory marker, an imaging result, a symptom score or a surrogate. A marker that improves is not automatically fewer hospitalizations or longer life.

Verify the product status separately

Check the regulator's record for the exact product and formulation. Also check the manufacturer, the intended use, and the dose. Then check the route, the age group, and the jurisdiction. Registration, an investigational authorization, a publication and a patent are four different things, and none of them is marketing approval.

If the study covers an approved product used outside its approved indication, say so plainly in the copy.

Get an independent read

Ask someone with no close tie to the study to review the design, the effect size and the main limitations. Give them the paper and the protocol, not the press release. Ask five focused questions.

A University of Michigan guide to publication errors sets out the routes open to a reader with a concern: write to the corresponding author, raise it with the journal editor as a letter, or use the publisher's comment format. It separates honest error from research misconduct. Knowing which route exists keeps a query from reading as an accusation.

Run the claim-to-evidence audit

Give every public statement its own row.

Repeat the audit for the headline, the chart, the caption, the alt text, the newsletter and the social post. Publish only when the strongest framing matches the narrowest relevant limitation.

Common questions

What if the paper has no registration number?

Ask the authors and the journal. If registration was expected for that design, say in the copy that it is absent. Do not assume the reported outcomes were prespecified.

Is a primary endpoint always the one patients care about?

No. It is the outcome designated for the study's main analysis, and it may be a surrogate or a scale score. Translating it into patient relevance is a separate step.

Can a reporter calculate a number the paper omitted?

Yes, when the inputs and the method are sound. Label it as a newsroom calculation, keep the worksheet, and get it reviewed if the result is material or complex.

Does a registry listing mean the government endorses the study?

No. A registry record documents what was submitted about a study. It does not by itself verify the results or confer approval.