The Data Checkup: A Framework for Assessing the Health of Federal Datasets

Why It Matters

By quantifying dataset vulnerability, the Data Checkup helps researchers, policymakers and journalists anticipate data loss, ensuring more reliable evidence‑based decisions and stronger data governance across the federal landscape.

Key Takeaways

• Six dimensions assess comprehensive dataset health
• Risk statuses range from Gone to No Known Issue
• Built on feedback from 30+ data experts
• Captures risks beyond simple URL checks
• Serves research, policy, journalism, and litigation

Pulse Analysis

Federal data repositories have long suffered from invisible decay—broken links, funding cuts, and shifting policy priorities that silently erode the utility of critical statistics. Early 2025 saw thousands of government webpages and data files go dark, exposing the limits of traditional URL trackers. Stakeholders realized that merely knowing whether a file is online does not reveal deeper threats such as reduced sample sizes, discontinued variables, or legal constraints that could render the data unusable for future analysis.

The Data Checkup addresses this gap with a six‑dimensional framework: Historical Data Availability, Future Data Availability, Data Quality, Statutory Context, Staffing and Funding, and Policy. Each dimension receives a status—Gone, High Risk, Moderate Risk, or No Known Issue—creating a nuanced health score for each dataset. Applied to flagship collections like the American Community Survey, Consumer Price Index and National Crime Victimization Survey, the tool surfaces hidden vulnerabilities, such as funding shortfalls for the Medicare Current Beneficiary Survey or policy‑driven variable removals in the National Assessment of Educational Progress. This granular view enables users to prioritize monitoring efforts and advocate for resources where risk is highest.

For the broader data ecosystem, the Data Checkup offers a common language to discuss dataset resilience, fostering collaboration between data producers, users, and watchdog groups. By highlighting risk before datasets disappear, it supports more robust research pipelines, informs legislative oversight, and strengthens journalistic investigations. As agencies adopt the framework, it could become a standard metric in federal data management, encouraging proactive investment in data continuity and enhancing public trust in government statistics.