Why federation can't give us a single patient record
Networking the existing EHR systems together will not make them behave like one record. The obstacle isn't funding, effort or goodwill. It is a set of limits that apply to every distributed computer system, and they bite hardest on old systems that were never designed to share.
The short version
- Federation leaves each patient's record split across several suppliers' systems, with no master copy, and tries to assemble it on demand over a network.
- Networks fail, slow down and change. A design that assumes otherwise breaks in ways that are well documented: the fallacies of distributed computing.
- When part of the network is unreachable, a distributed system must either refuse to answer or risk a wrong answer. It can't avoid both. This is the CAP theorem, and it is a mathematical proof, not an opinion.
- The more systems a record depends on, the more often that choice comes up.
- The organisations that would have to make federation work are the ones whose business depends on holding the record.
What federation promises
The idea is attractive because nobody has to give anything up. Each record stays where it is, inside the system of whichever supplier the GP practice, hospital or community service happens to use. Interfaces are added between them. When a clinician opens a patient's record, the network gathers the pieces and presents them as one.
For that to be safe to practise from, three things have to be true. The record must be complete. It must be current. And it must be there when it's needed. Federation can't guarantee all three at once, for the reasons below.
Eight assumptions that don't hold
In the 1990s, engineers at Sun Microsystems wrote down the false assumptions that people new to networked systems reliably make. The list is credited to L. Peter Deutsch, with the eighth added by James Gosling, and is known as the fallacies of distributed computing. Each one is something a designer takes for granted and later pays for.
A federated national record depends on every one of them.
| The assumption | What it means for a federated record |
|---|---|
| 1. The network is reliable | Every time a record is opened, each system holding part of it must answer. Sooner or later one won't, and the clinician sees a record with a hole in it. |
| 2. Latency is zero | A record assembled from several systems is only as fast as the slowest of them. A ten-minute consultation can't wait for it. |
| 3. Bandwidth is infinite | Whole histories, documents and images have to be fetched across the network on demand, again and again, instead of being read from one place. |
| 4. The network is secure | Every connection between systems is another way in. Each supplier has its own sign-in, its own permissions and its own audit log, so there is no single account of who saw what. |
| 5. Topology doesn't change | Practices merge, providers change supplier, systems are upgraded or retired. Every change can break connections that others rely on. |
| 6. There is one administrator | There are many: competing suppliers, each with its own priorities, release schedule and commercial interests. Nobody can fix a fault that spans two of them. |
| 7. Transport cost is zero | Every interface has to be built, tested, assured and maintained, by each party, for as long as the federation lasts. |
| 8. The network is homogeneous | Each system has its own data model, built up over decades. Translating between them loses meaning, and a record that has been translated is not the record that was written. |
The CAP theorem
The fallacies are hard-won experience. The CAP theorem is stricter: it is a proof. Eric Brewer proposed it in 2000, and Seth Gilbert and Nancy Lynch of MIT proved it in 2002.
It concerns three properties of a system that keeps data in more than one place:
- Consistency. Everyone who looks sees the same, most recent data.
- Availability. Every request gets an answer.
- Partition tolerance. The system keeps working when some of its parts can't talk to each other.
The theorem says that when the parts can't talk to each other, you cannot have both of the first two. And over a real network, sooner or later, some parts won't be able to talk to each other. So the choice can't be designed away. It can only be made.
Put that in a consulting room. A clinician opens a record, and one of the systems holding part of it can't be reached. The federation has two options:
There is no third option, and no amount of engineering produces one. Neither of these is something you'd choose for a clinical record.
Even when everything is connected, a related trade-off remains. Confirming that every copy agrees takes time, so a system must still choose between answering quickly and answering with certainty.
The arithmetic of many systems
A single well-run system that is available 99.5% of the time is out of action for about 44 hours a year. A federated record needs every contributing system to be up at the same moment, so the odds multiply.
| Systems that must all answer | Chance the full record is available |
|---|---|
| 1 | 99.5% |
| 5 | 97.5% |
| 10 | 95.1% |
| 20 | 90.5% |
These figures are an illustration, not a measurement: they assume each system is up 99.5% of the time and that they fail independently. The pattern is what matters. With twenty systems in the chain, the complete record is unavailable roughly one time in ten, even though each system on its own is performing well.
Writing is harder than reading
Reading a scattered record is the easy part. A record is also written to, and in a federation there is no master copy to write to.
Suppose a hospital stops a medication on the same afternoon that the GP practice changes its dose. Each system has recorded a true event. Which one is the current medication list? Inside a single system, the database settles the order and one answer comes out. Across independent systems owned by different companies, nothing settles it. The disagreement either sits there unnoticed or lands on a clinician to sort out by hand.
You can add a layer on top to decide. But a layer that holds the authoritative answer for every patient is a single system of record. At that point you have built the thing federation was meant to avoid, with all the old systems still attached to it.
Old systems, built to stand alone
The systems being federated were mostly designed years ago to be the only record-keeper for one organisation. They assume that they are the sole writer of their own data and that it only changes through their own screens. Being one participant among many in a live national network is a different job, and it has to be retrofitted onto software, data models and codings that have each grown in their own direction for decades.
The incentive problem
The sixth fallacy, that there is one administrator, is where the engineering meets the economics.
Federation only works if every supplier builds, maintains and promptly fixes high-quality interfaces to its competitors, indefinitely. But a supplier's commercial position rests on being the place where the record lives. The harder the data is to move or to use from elsewhere, the more secure the contract. Good interoperability weakens exactly that.
This doesn't require bad faith from anyone. It is simply that federation asks each supplier to invest, year after year, in something that works against its own interests, and then depends on all of them doing so at once. A design that only works if the participants act against their incentives is not a sound design.
Why pilots look fine
Federation tends to work on paper and in small pilots. That is what the theory predicts. A pilot has a few carefully chosen sites, a few systems, little traffic, and engineers watching it. Every problem described above grows with the number of systems, the number of connections between them and the length of time they have to keep working unattended. A pilot shows that the interfaces can work. It can't show that they will all work, together, every day, across the country.
What federation is good for
None of this means that systems shouldn't talk to each other. Sending a referral, a discharge letter or a test result from one organisation to another works well, because a message has one sender, one receiver and no need for instant agreement. Looking something up in another system as a convenience is useful too, so long as nobody relies on it being complete.
What can't be done is to turn a collection of independent record systems into something clinicians can treat as one complete, current record. That needs a system of record.
What works instead
The dependable way to get a single patient record is to have one: a single system of record, with one route in for every read and every write.
This doesn't repeal the CAP theorem, which applies to every system. It changes who is dealing with it. A single platform under one administrator can keep copies of its data in several places and keep them in step, using well-understood techniques, because it controls all of them. The trade-offs are made once, deliberately, by people accountable for the result, instead of emerging by accident from the gaps between competing products.
It also makes the first stage of the Trust's plan sound. Keeping a copy of the record up to date from a practice's existing system is a one-way flow from a single master, which is a far simpler problem than many masters trying to agree.
A single record concentrates responsibility, so the question that matters is who holds it. That is why the Medical Records Trust is a non-profit controlled by clinician trustees, holding the record on an open standard, with one door to the data that checks and logs every request.
References
- L. Peter Deutsch and others, Sun Microsystems. The Eight Fallacies of Distributed Computing, 1994; eighth fallacy added by James Gosling, 1997.
- Eric A. Brewer. Towards Robust Distributed Systems. Keynote, ACM Symposium on Principles of Distributed Computing (PODC), 2000.
- Seth Gilbert and Nancy Lynch. Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services. ACM SIGACT News 33(2), 2002.
- Daniel J. Abadi. Consistency Tradeoffs in Modern Distributed Database System Design. IEEE Computer 45(2), 2012. (The trade-off between speed and consistency when the network is healthy.)
