Real-World Data: Assessing Relevance and Reliability

Every FDA RWE guidance — from the 2018 Framework through the 2024 final guidances on data standards and study design — reduces to two questions: is the data relevant to the regulatory question, and is it reliable enough to support a regulatory decision?

Relevance is about fit: does the source capture the target population, exposures, outcomes, follow-up duration, and covariates the protocol requires? Reliability is about provenance: were the data accrued, transformed, and curated in a way that can be audited and reproduced?

A defensible memo includes: data source description and licensing, accrual and update cadence, ETL and curation steps, code lists and value sets for every variable, missingness analysis by variable and by patient-time, linkage methodology if multiple sources are combined, and a documented audit trail.

For EHR sources, address structured-versus-unstructured capture explicitly. For claims, document the difference between paid, adjudicated, and open claims, and the implications for outcome ascertainment. For registries, document case definition, ascertainment, and loss to follow-up.

When RWD is used in a regulatory submission, the systems that produced and transformed it fall within Part 11 scope. Sponsors should expect to demonstrate validation, audit trails, access controls, and electronic signature provenance for the curation pipeline — not only for the analysis environment.