Why Health Systems Have QA Teams and Still Run UAT in Spreadsheets

Validated, part one of two

Epic held its Users Group Meeting in Verona last week, and most of the coverage went where you would expect: Ergo, Cosmos Curiosity, an Agent Factory with 120 out-of-the-box AI capabilities, AI-assisted coding.

The statistic that stayed with me was less exciting than any of that.

Epic noted that it used to release every eighteen months and now releases quarterly. If you work in this space, that part is not news. Quarterly releases have been the norm for years. What is new is what is inside them. A single quarterly release now carries roughly the enhancement volume of one of those old eighteen-month releases, and the most recent one contained about 27% more code than the same version a year earlier.

The calendar did not change. The payload did.

That is a health system problem before it is anyone else's, because the upgrade cadence is a health system decision. Enterprise IT weighs every application, every team and every competing priority, then picks a schedule. In most organizations that lands on semi-annual, with some staying on the full quarterly cycle. Whatever comes out of that process is what every downstream team lives with.

Which means a semi-annual organization is now absorbing two releases' worth of change per window, and each of those releases is denser than it was last year.

Nothing on the calendar looks different. The same upgrades appear in the same months. The plan carries forward from last year because last year's plan worked.

And the EHR is only the largest item on the list. I will come back to that, because it turns out to be the more important half of the problem.

What an upgrade actually is

Ask someone outside the process to describe an Epic upgrade and you will usually hear something about testing the new version.

That description is not wrong, but it badly understates the shape of the work.

An enterprise upgrade means dozens of application teams working in parallel. It means analysts testing across every module on the same deadline, with domain specialists in clinical, billing, revenue cycle and ancillary areas pulled in to validate outputs and review results before going back to their own work. It means thousands of scenarios that are interdependent, so a failure in one area invalidates assumptions in three others. It means a fixed date that everything has to converge on.

The individual tests are rarely the hard part. Someone opens a workflow, follows the steps and confirms it behaves.

The difficulty is everything around the tests. Who is testing what. Which scenarios are actually finished versus marked finished. Which failures are real defects versus environment problems versus someone testing the wrong thing. What has been fixed and retested, and what has been fixed and forgotten. Where the remaining effort sits, and whether there is enough time and enough people to get through it.

Coordinating that is a different problem from executing it. And it is the one that determines whether the upgrade goes smoothly.

The testing function is not the problem

Here is where I think a lot of organizations end up stuck, and it is not where people usually assume.

The easy explanation is that hospital testing struggles because there is no real testing function. That is not what is happening. Health systems have IT testers, and many have dedicated testing and quality assurance leadership with that responsibility in the job title. Look at how the analyst role is written and you will find testing sitting inside it too: developing system test plans, executing testing of upgrades and patches, maintaining a record of test progress and results, tracking problem reports. One industry description puts the analyst role well: a hybrid business analyst, system analyst and QA coordinator all at once.

So the function exists. The discipline exists.

What is missing is an instrument.

Most software test tooling was built for a continuous engineering process: automated execution against a build, professional testers embedded in the pipeline, a clean boundary between the people writing code and the people verifying it. Those tools are good, and test management inside DevOps is a genuinely solved problem.

An enterprise upgrade is a different shape of work. It is a burst of time-boxed manual execution across dozens of application teams at once, run by analysts who are also building, with domain specialists rotating in to validate outputs and then returning to their own jobs. Thousands of steps, executed by people, on a fixed date.

Ask the person accountable for that where testing actually stands on a Tuesday afternoon, and in most organizations the honest answer requires assembling it: pulling from spreadsheets, a chat channel per application, a defect list that lives somewhere else and a round of asking people. The QA lead is not missing a process. They are missing a place where the process runs and reports itself.

Nordic, who have no product to sell here, name the symptoms in their guidance on upgrade projects. The areas where clients struggle, they say, tend to center on testing scripts and, as a consequence, testing itself. They also note that organizations short on resources often have the upgrade project manager doubling as the testing coordinator.

I would take both of those seriously, and I would connect them.

Test scripts are a problem because they have nowhere to live. Written in documents, revised by whoever touched them last, rebuilt at the start of each cycle because finding and trusting the previous version costs more than starting over. Years of accumulated work that does not accumulate.

And staffing looks short partly because so much of the available capacity is not going into testing. It is going into coordination: chasing status, reconciling trackers, working out whether a thing marked complete is complete, rebuilding the same picture every week for the same meeting. A team can be adequately staffed for the testing and badly understaffed for the administration around it, and from the inside those feel identical.

Neither of those is a maturity problem. They are what a capable team looks like when it is doing skilled work without the right equipment.

What the upgrade team actually needs

When I talk to enterprise teams about this, the things they want are consistent, and none of them are exotic.

Visibility that reflects reality. Not a status meeting summarizing what people said. An actual view of which scenarios have been executed, by whom and with what result, available the Friday before go-live without anyone assembling it by hand. Rolled up across every application team, with the ability to drill into whichever one looks wrong.

Control over the work. Assignment, sequencing and blocking. Knowing which teams are behind and which areas nobody has touched. Knowing whether execution is concentrated in a handful of testers, and whether someone who has logged nothing in two weeks is blocked rather than idle.

Issue tracking tied to the test. A defect attached to the specific step that produced it, with severity and status, not a screenshot in an email thread. Above all, the ability to see what has been fixed and is genuinely ready for retest.

The part that gets underestimated: analytics

Visibility tells you where this upgrade stands today. Useful, and most teams would settle for it.

But the more valuable thing is what you can only see across upgrades, and it is the thing almost no organization has, because the data does not survive. Each upgrade gets its own spreadsheets. At the end, they get archived, and the next cycle starts from an empty file. Whatever the last one could have taught you is sitting in a folder nobody opens.

Run every cycle in the same place and something different becomes possible. Not a status report. A baseline.

Is testing taking longer than it used to? Over the past two quarters, four quarters or two years, is the same scope of work consuming more days, more testers or more elapsed calendar? This is the question the release-density problem actually turns on. Everyone I talk to has a feeling about it. Almost nobody has a number. And a feeling does not get you headcount, while a two-year trend line might.

Are we finding more issues, or fewer? Issue discovery rate per cycle, tracked over time. Rising discovery can mean the build is getting riskier, or it can mean the testing got better and was previously missing things. Either interpretation matters, and you cannot have the conversation at all without the history.

Are we resolving faster? Days from issue raised to ready-for-retest, cycle over cycle. This is where real operational improvement shows up, and it is measurable in a way that "the team is getting better at upgrades" is not. It is also the number that lets you demonstrate the effect of a process change instead of asserting it.

Which applications are consistently the buggiest? Not in this upgrade, but across all of them. Some modules generate defects every single cycle. Others have been stable for two years. That pattern is genuinely actionable in both directions: it tells you where to concentrate testing effort and scrutiny, and it tells you where you may be over-testing something that has not surprised anyone since 2024. Risk-based scoping needs evidence, and this is the evidence.

Where does the effort actually go? Which applications, workflows and teams consume the testing capacity? That is what capacity planning for the next cycle should be built on, rather than last time's plan plus a guess.

Put those together and the conversation with leadership changes. Instead of asking for more testers because this upgrade feels big, you can show that the same scope now takes 30% longer than it did eighteen months ago, that discovery has risen while resolution has improved, and that three applications account for most of the defect volume. That is a resourcing argument that survives contact with a CFO.

None of this requires sophisticated measurement. It requires that the data from one cycle still exists in a comparable form when the next one starts, which is precisely what running the work in spreadsheets guarantees it will not.

And it was never only Epic

I opened with Epic because the UGM statistic is what prompted this, and because the EHR upgrade is the largest single testing event most health systems run.

But look at what else your organization validates.

The laboratory information system. Pharmacy. Radiology and PACS. Blood bank. Anesthesia and cardiology systems. The middleware and interface layer holding it together. Revenue cycle and claims. ERP systems for workforce, supply chain and financials. Credentialing. Patient engagement. Analyzer software in the lab. Whatever came along with the last hospital you acquired and has not been consolidated yet. A long tail of departmental applications that are heavily customized, clinically important and never mentioned in an enterprise project plan.

Most of these are not on Epic's release schedule. They are on their own, set by their own vendors and increasingly delivered as hosted software that updates when the vendor decides. Which means the testing burden is not one large annual event. It is a large event plus a continuous stream of smaller ones, distributed across teams that have no reason to be coordinating with each other.

If each of those systems is validated its own way, with this team's spreadsheet, that team's ticketing system and another team's shared drive, then the rollup does not exist and the trend data cannot be built. You cannot compare defect rates across applications when every application counts defects differently. You cannot tell whether testing is taking longer overall when no two teams measure "longer" the same way.

The analytics I described are not possible because someone builds a prettier dashboard. They become possible because the underlying work is captured consistently, cycle after cycle, in a form that can actually be compared.

That is the argument for a single platform, and it is not really an argument about convenience. Consistency of method is what makes measurement possible. One tool across every system you validate is what turns thirty disconnected efforts into a program you can actually see.

It also means the investment does not sit idle between EHR upgrades. The same platform running your Epic cycle runs the LIS validation in March and the ERP rollout in September, and each one adds to the same baseline.

The part I would like to hear about

So here is my actual question, and I am asking it rather than answering it.

What are you using to manage UAT across your upgrades, and does it give you a real picture on the Friday before go-live, or a picture assembled from what people told you in a status meeting?

I already know what most of the answers are. Spreadsheets. Shared drives. A tracker somebody built years ago that everyone has learned to work around.

What I am more interested in is why.

In my experience, it is almost never a decision. Nobody sat down, evaluated the options and concluded that a spreadsheet was the right instrument for coordinating thousands of test executions across dozens of teams. It is what the last upgrade used, so it is what this one uses. The person who set it up has probably left. It gets copied forward because copying it forward is faster than questioning it, and because the upgrade is already underway and nobody has time to change the method mid-flight.

And underneath that, a reasonable assumption that this is simply how it is done. That if something purpose-built for this existed, you would have heard about it by now.

So the question I would actually like answered is not whether your current approach works. It is whether anyone has ever gone looking for something better, and what they found when they did.

Why I am writing about this now

Cymetryc started in laboratory validation, which is where most people know us from. Over the last year, we have been used for enterprise EHR upgrade validation as well, and the interest arrived from an unexpected direction: one of our customers posted on an Epic community board about how they run their EHR validation with us, and several other health systems reached out after seeing it.

Two things about that stayed with me. A health system decided this was worth telling its peers about, unprompted. And the people who reached out were not organizations missing a testing function. Some of them hold testing and quality assurance in their job titles. They had the discipline. They were looking for the instrument.

That usually means the need was there before anyone went looking for a solution.