The Medical Records Trust

Why federation can't give us a single patient record

Networking the existing EHR systems together will not make them behave like one record. The obstacle isn't funding, effort or goodwill. It is a set of limits that apply to every distributed computer system, and they bite hardest on old systems that were never designed to share.

The short version

What federation promises

The idea is attractive because nobody has to give anything up. Each record stays where it is, inside the system of whichever supplier the GP practice, hospital or community service happens to use. Interfaces are added between them. When a clinician opens a patient's record, the network gathers the pieces and presents them as one.

For that to be safe to practise from, three things have to be true. The record must be complete. It must be current. And it must be there when it's needed. Federation can't guarantee all three at once, for the reasons below.

Eight assumptions that don't hold

In the 1990s, engineers at Sun Microsystems wrote down the false assumptions that people new to networked systems reliably make. The list is credited to L. Peter Deutsch, with the eighth added by James Gosling, and is known as the fallacies of distributed computing. Each one is something a designer takes for granted and later pays for.

A federated national record depends on every one of them.

The assumptionWhat it means for a federated record
1. The network is reliableEvery time a record is opened, each system holding part of it must answer. Sooner or later one won't, and the clinician sees a record with a hole in it.
2. Latency is zeroA record assembled from several systems is only as fast as the slowest of them. A ten-minute consultation can't wait for it.
3. Bandwidth is infiniteWhole histories, documents and images have to be fetched across the network on demand, again and again, instead of being read from one place.
4. The network is secureEvery connection between systems is another way in. Each supplier has its own sign-in, its own permissions and its own audit log, so there is no single account of who saw what.
5. Topology doesn't changePractices merge, providers change supplier, systems are upgraded or retired. Every change can break connections that others rely on.
6. There is one administratorThere are many: competing suppliers, each with its own priorities, release schedule and commercial interests. Nobody can fix a fault that spans two of them.
7. Transport cost is zeroEvery interface has to be built, tested, assured and maintained, by each party, for as long as the federation lasts.
8. The network is homogeneousEach system has its own data model, built up over decades. Translating between them loses meaning, and a record that has been translated is not the record that was written.

The CAP theorem

The fallacies are hard-won experience. The CAP theorem is stricter: it is a proof. Eric Brewer proposed it in 2000, and Seth Gilbert and Nancy Lynch of MIT proved it in 2002.

It concerns three properties of a system that keeps data in more than one place:

The theorem says that when the parts can't talk to each other, you cannot have both of the first two. And over a real network, sooner or later, some parts won't be able to talk to each other. So the choice can't be designed away. It can only be made.

Put that in a consulting room. A clinician opens a record, and one of the systems holding part of it can't be reached. The federation has two options:

Stay consistentRefuse to show the record until every system has answered. The clinician has nothing to work from.
Stay availableShow what could be found. The clinician works from a record that may be missing an allergy, a recent result or a change of medication.

There is no third option, and no amount of engineering produces one. Neither of these is something you'd choose for a clinical record.

Even when everything is connected, a related trade-off remains. Confirming that every copy agrees takes time, so a system must still choose between answering quickly and answering with certainty.

The arithmetic of many systems

A single well-run system that is available 99.5% of the time is out of action for about 44 hours a year. A federated record needs every contributing system to be up at the same moment, so the odds multiply.

Systems that must all answerChance the full record is available
199.5%
597.5%
1095.1%
2090.5%

These figures are an illustration, not a measurement: they assume each system is up 99.5% of the time and that they fail independently. The pattern is what matters. With twenty systems in the chain, the complete record is unavailable roughly one time in ten, even though each system on its own is performing well.

Writing is harder than reading

Reading a scattered record is the easy part. A record is also written to, and in a federation there is no master copy to write to.

Suppose a hospital stops a medication on the same afternoon that the GP practice changes its dose. Each system has recorded a true event. Which one is the current medication list? Inside a single system, the database settles the order and one answer comes out. Across independent systems owned by different companies, nothing settles it. The disagreement either sits there unnoticed or lands on a clinician to sort out by hand.

You can add a layer on top to decide. But a layer that holds the authoritative answer for every patient is a single system of record. At that point you have built the thing federation was meant to avoid, with all the old systems still attached to it.

Old systems, built to stand alone

The systems being federated were mostly designed years ago to be the only record-keeper for one organisation. They assume that they are the sole writer of their own data and that it only changes through their own screens. Being one participant among many in a live national network is a different job, and it has to be retrofitted onto software, data models and codings that have each grown in their own direction for decades.

The incentive problem

The sixth fallacy, that there is one administrator, is where the engineering meets the economics.

Federation only works if every supplier builds, maintains and promptly fixes high-quality interfaces to its competitors, indefinitely. But a supplier's commercial position rests on being the place where the record lives. The harder the data is to move or to use from elsewhere, the more secure the contract. Good interoperability weakens exactly that.

This doesn't require bad faith from anyone. It is simply that federation asks each supplier to invest, year after year, in something that works against its own interests, and then depends on all of them doing so at once. A design that only works if the participants act against their incentives is not a sound design.

Why pilots look fine

Federation tends to work on paper and in small pilots. That is what the theory predicts. A pilot has a few carefully chosen sites, a few systems, little traffic, and engineers watching it. Every problem described above grows with the number of systems, the number of connections between them and the length of time they have to keep working unattended. A pilot shows that the interfaces can work. It can't show that they will all work, together, every day, across the country.

What federation is good for

None of this means that systems shouldn't talk to each other. Sending a referral, a discharge letter or a test result from one organisation to another works well, because a message has one sender, one receiver and no need for instant agreement. Looking something up in another system as a convenience is useful too, so long as nobody relies on it being complete.

What can't be done is to turn a collection of independent record systems into something clinicians can treat as one complete, current record. That needs a system of record.

What works instead

The dependable way to get a single patient record is to have one: a single system of record, with one route in for every read and every write.

This doesn't repeal the CAP theorem, which applies to every system. It changes who is dealing with it. A single platform under one administrator can keep copies of its data in several places and keep them in step, using well-understood techniques, because it controls all of them. The trade-offs are made once, deliberately, by people accountable for the result, instead of emerging by accident from the gaps between competing products.

It also makes the first stage of the Trust's plan sound. Keeping a copy of the record up to date from a practice's existing system is a one-way flow from a single master, which is a far simpler problem than many masters trying to agree.

A single record concentrates responsibility, so the question that matters is who holds it. That is why the Medical Records Trust is a non-profit controlled by clinician trustees, holding the record on an open standard, with one door to the data that checks and logs every request.

Register your interest

References

  1. L. Peter Deutsch and others, Sun Microsystems. The Eight Fallacies of Distributed Computing, 1994; eighth fallacy added by James Gosling, 1997.
  2. Eric A. Brewer. Towards Robust Distributed Systems. Keynote, ACM Symposium on Principles of Distributed Computing (PODC), 2000.
  3. Seth Gilbert and Nancy Lynch. Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services. ACM SIGACT News 33(2), 2002.
  4. Daniel J. Abadi. Consistency Tradeoffs in Modern Distributed Database System Design. IEEE Computer 45(2), 2012. (The trade-off between speed and consistency when the network is healthy.)