Have you ever watched a FHIR API run flawlessly for hours and then hit a wall at the top of the hour? The wall is usually cache expiry. Caches make everything feel fast; they also hide the true latency of the backend until the cache empties and the real number returns. Deployments that never look under the cache carry a latency problem they cannot see.
The trick is to measure both the cached case and the uncached case, on purpose. For related walkthroughs, deeper FHIR walkthroughs collects the surrounding material.
What a Warm Cache Looks Like
A warm cache serves reads out of memory. The latency for a Patient read might be one millisecond during the warm window. The latency for a search-heavy Bundle might be five. These numbers look great and match the dashboard glowing green.
The mistake is treating those numbers as the truth about the backend. They are the truth about the cache, which is a different subsystem.
The Cliff at TTL Expiry
The moment the cache TTL expires, the next request hits the backend. For a well-tuned FHIR server that request completes in fifty milliseconds; for a poorly tuned one it might take two seconds. Whichever number it is, that is the actual backend latency.
Caches with fixed TTLs produce a repeating cliff pattern: latency drops during the TTL window, spikes at expiry, and returns to the low value once the cache warms again. A pass through the site's FHIR p99 latency lookup surfaces the uncached number and gives a clean bracket for what the cliff will look like.
Cache Stampede as a Failure Mode
Cache expiry gets worse when multiple requests hit the backend at the same moment. If a hundred concurrent users all try to read a freshly-expired resource, a hundred requests fire at the backend in the same fifty milliseconds. The backend contends with all of them and the tail latency for that expiry window climbs into the seconds.
The classic fix is a stampede lock: the first request to miss the cache locks the key, fetches the resource, populates the cache, and releases the lock. The other ninety-nine wait for the cache to populate rather than firing their own backend calls.
Sliding TTLs Move the Cliff
Sliding TTLs push the expiry moment forward every time the cache is accessed. Frequently-read resources never expire; rarely-read ones expire on schedule. The cliff moves to the boundary of the usage pattern instead of the boundary of the TTL.
Sliding TTLs interact with the network-vs-server split in subtle ways. For that split as its own topic, network latency vs server latency in FHIR requests covers the diagnostic method.
Measuring the Uncached Number Explicitly
The single move that changes the conversation is to measure the uncached latency on purpose. Add a small percentage of cache-bypass traffic that always hits the backend, and report its latency separately. That number is the truth about the FHIR server's behavior under load.
Deployments that only report warm-cache latency defend numbers that will fail during expiry events. Deployments that report both hold both accountable.
Bundle Reads Hide the Most
Bundle reads are the case where caches hide the most latency. A Bundle assembled from a hundred entries touches many cache keys, and even a partial cache miss produces a noticeable slowdown. Warm-cache latency for the Bundle might be sixty milliseconds; a partial-miss cache latency might be six hundred.
For the size-vs-latency curve underneath that shift, how Bundle size drives tail latency in FHIR is the accompanying reference.
The Honest Dashboard
The honest latency dashboard shows warm, uncached, and expiry-cliff percentiles side by side. Every latency conversation starts easier when the reader can see the split. Caches are a legitimate optimization; hiding the backend behind them is not.
Sources
- HL7 FHIR core specification of HTTP interactions including - HL7 FHIR core specification of HTTP interactions including Cache-Control semantics that shape cache TTL behavior