HHS recently proposed eliminating the“real world testing” (RWT) requirement for certified health IT vendors under the ONC Health IT Certification Program. RWT was originally introduced as part of the 21st Century Cures Act to ensure that certified systems performed safely and effectively in the environments where they were deployed.
The proposed rule aims to reduce burden and accelerate innovation by scrapping what HHS now deems a largely redundant reporting exercise. And if you’ve ever read a vendor’s real world testing plan, you might find yourself asking: Are we really losing anything of value?
Was Real World Testing Actually Real?
According to a 2024 review published inJAMIAby Kellermann et al., real world testing never came close to fulfilling its intended promise. Instead of surfacing safety risks or informing improvements, the reports were largely vendor-generated artifacts, divorced from clinical workflows and often lacking in usable detail. As the authors write:
“Despite certification, many EHRs deployed in hospitals and physician offices still exhibit serious usability and safety issues… Certification alone is insufficient to ensure safety.”(Kellermann et al., 2024)
RWT was, in theory, a good idea. But in practice, it became a checkbox. No site-specific configuration. No clinician input. No meaningful evidence of actual system performance in context.
The Responsibility Was Already Ours
If anything, this rule change simply acknowledges what’s been true all along: the burden of ensuring health IT safety lies squarely with health systems—not vendors. Certification was never a substitute for local validation.
Every hospital that’s ever had a system go down mid-shift, or discovered a misconfigured lab interface days after go-live, knows the truth: compliance documents don’t catch errors. Testing does.
And that testing has to be done where the system is used, how it’s used, by the people who use it.
So, What Are We Losing?
Not certification. Not safety oversight. Just the illusion that once-a-year, vendor-authored “real world” reports gave us anything meaningful.
In fact, you could argue that removing RWT is an opportunity. It clears the regulatory underbrush and puts attention back where it belongs—on rigorous, accountable,site-level validation.
But there’s a catch: you can’t fall back on spreadsheets, file shares, and hallway approvals. The pace of change is accelerating. And now, without even the thin layer of RWT,you must be able to prove that your systems are safe, testable, and traceable—every time they change.
A Better Way Forward
What should modern, real RWT validation look like? Systems like Cymetryc offer a practical answer. Purpose-built for health IT, Cymetryc brings:
- Centralized test planning, execution, and documentation
- Real-time dashboards to track tester activity and approval progress
- Seamless paperless review by clinical approvers, from anywhere
- Instant generation of audit-ready compliance reports
When TriCore Reference Labs had to validate two major instrument integration projects, they skipped the "stone age tools" and adopted Cymetryc instead. The result? Over 850 test scripts executed with full traceability, 100% coverage under a tight 4 week timeline, and smoother inspections thanks to one-click reporting.
(You can download the details here: https://landing.cymetryc.com/instrumentation-validation-whitepaper/)
The Real Test Is Yours
RWT may be going away, but real-world validation isn’t. In fact, it’s more important than ever—because now there’s no one else pretending to do it for you.
This shift isn’t a loss. It’s a wake-up call. And if your team is ready to modernize validation—not just as a compliance task, but as a clinical safety discipline—you won’t miss RWT at all.
P.S.If your validation process still relies on binders, email chains, or memory, it might be time for a rethink. Modern tools exist—and they make real-world validation not only faster, but better.
References
- Becker’s Hospital Review.HHS eases regulations on EHR vendors.June 20, 2024.https://www.beckershospitalreview.com/healthcare-information-technology/ehrs/hhs-eases-regulations-on-ehr-vendors
- Kellermann, A. L., Jones, S. S., et al.Oversight of health information technology: a proposal for a national-level approach.J Am Med Inform Assoc. 2024; PMC12153719
