Statistics and chart verification checklist

What to take away

• Record the exact dataset, release, table, variable, and retrieval time.
• Check whether the data are accurate, complete, and suitable for this question.
• Reproduce every material calculation and comparison.
• Carry uncertainty and definitions into charts and headlines.
• Preserve the source file, formulas, chart data, and final version.

Use this checklist before publishing data-driven text, tables, maps, polls, dashboards, and charts. Add a reviewer, date, result, and evidence path. Mark a source insufficient for purpose when the question exceeds what the data can answer.

Provenance and version

• Producer, dataset, release, table, variable, and file identified
• Specific source page or download saved
• Retrieval time, revision status, and update schedule recorded
• Raw, cleaned, modeled, imputed, adjusted, and derived values distinguished
• Data dictionary, technical notes, and release notes reviewed
• Corrections, breaks in series, and back revisions checked
• Query, code, filters, or manual export steps preserved

Fitness for purpose

• Reporting question and required unit defined
• Data population, coverage, geography, and period match the claim
• Record-creation process and institutional purpose understood
• Accuracy, completeness, timeliness, and applicability assessed
• Duplicate, missing, suppressed, and invalid records handled explicitly
• Policy or system changes that alter recording considered
• Corroborating records or spot checks used where risk warrants

The U.S. Government Accountability Office's data reliability guide frames reliability around accuracy, completeness, and applicability for the intended purpose, using a risk-based assessment. A dataset can be reliable enough for one narrow claim and inadequate for another. Record that judgment instead of applying a universal quality label.

Definitions and units

• Numerator, denominator, unit, scale factor, and period named
• People, events, visits, reports, cases, and transactions distinguished
• Nominal and inflation-adjusted money distinguished
• Adjusted and unadjusted series labeled
• Count, proportion, percentage, rate, ratio, index, mean, and median used correctly
• Overlapping categories not summed
• Comparable definitions used across places and years

Survey and model checks

• Target population and sampling or recruitment method recorded
• Sample and subgroup sizes checked
• Question wording, order, choices, field dates, and mode reviewed
• Weighting, design effect, response, and coverage examined
• Model inputs, assumptions, validation, and revision identified
• Sampling and nonsampling errors distinguished
• Published uncertainty measure matched to the design

Calculations and comparisons

• Material totals, percentages, changes, rates, and indexes reproduced
• Percentage change distinguished from percentage-point change
• Rounding occurs after calculation and does not alter the claim
• Start and end values retained with change language
• Statistical test accounts for relevant uncertainty and dependence
• Significant difference distinguished from practical importance
• Correlation not rewritten as causation

Chart construction

• Chart type matches the analytical question
• Title states the measure, group, place, and period accurately
• Axes, units, source, denominator, and notes visible
• Bar and area baselines start at zero
• Nonzero line-chart axis is clear and justified
• Comparable panels use consistent scales
• Category order, color, area, and perspective do not distort magnitude

The UK Office for National Statistics axes and gridlines guidance calls for zero baselines on bar and area charts, consistent scales across comparable charts, and clear treatment of nonzero or nonlinear axes. Those rules follow from how visual length and position encode numerical differences.

Uncertainty and missing data

• Confidence or uncertainty ranges shown when they affect interpretation
• Suppressed and unavailable values not drawn as zeros
• Breaks in series displayed rather than connected silently
• Small subgroup estimates flagged or omitted under source rules
• Model and forecast ranges labeled apart from observed values
• Data labels do not imply false precision

Publication language and review

• Headline, deck, chart, caption, alt text, and social copy agree
• Highest, lowest, majority, average, typical, risk, and record claims tested
• Estimate language used when point values are uncertain
• Method and material limits placed near the result
• Sponsor and conflicts disclosed where relevant
• Independent reviewer reproduces the central number and chart
• Correction plan includes data files and distributed graphics

Common questions

Must a journalist inspect every row?

No. The depth should match risk, but definitions, quality controls, anomalies, and material calculations always need review.

Is a dashboard screenshot enough as a source?

No. Preserve the underlying table, filters, update time, and method when possible.

Can a line chart start above zero?

Sometimes, if the axis break is clear and the range does not exaggerate the claim. Bars should start at zero.

What if the data are only partly reliable?

Narrow the question, corroborate, disclose the limit, or omit the claim if the remaining risk is unacceptable.