How often does the FHIR community get a real performance comparison across vendors, run on the same machine, with the test scripts in public view? Almost never. That changed on June 29, when Health Samurai released a public benchmark that puts HAPI FHIR, Medplum, the Microsoft FHIR Server, and Aidbox on identical hardware and reruns the whole thing daily.
The benchmark was put together by Marat Surmashev, VP of Engineering at Health Samurai, and his team. For US emr-development shops that have to size FHIR backends without much shared ground truth, the report fills a long-running gap. The numbers below are from the snapshot on the day the dashboard went public.
The Four Servers Under Test
The lineup is intentional. HAPI FHIR is the workhorse open-source server most teams in the US already know. Medplum is the developer-focused open-core option with a hosted offering. The Microsoft FHIR Server is the Azure-native choice for organizations already sitting in that cloud. Aidbox is the commercial server built by Health Samurai. Putting all four on one box was the part nobody had done in public.
For the US digital health hub, the full coverage of FHIR vendor decisions sits alongside this release.
How the Test Box Is Set Up
The host is one bare-metal machine with 64 CPU cores and 500 GB of RAM. Each server got 8 vCPU and 24 GB of RAM. Medplum runs as 8 single-core replicas of 3 GB each, which matches the way the platform scales in production rather than forcing it into a single-process container.
Aidbox, HAPI, and Medplum each run against PostgreSQL 18. The Microsoft FHIR Server runs against SQL Server 2022 Developer Edition. Synthea generated the data: 1,000 synthetic patients, around 2 million FHIR resources. Grafana k6 drove the load.
What the CRUD Numbers Look Like
The CRUD test runs create, read, update, and delete across nine FHIR resource types under sustained load. The June 29 snapshot:
- Aidbox: about 5,212 RPS
- HAPI FHIR: about 3,058 RPS
- Medplum: about 1,420 RPS
- Microsoft FHIR Server: about 440 RPS
The 11x spread between top and bottom is the headline. For a US emr-development team trying to estimate AWS or Azure costs against a real production load, that gap shapes the bill in ways no marketing deck can hide. The benchmark also reports P99 latency per operation per server, which is the data most architects actually want when they think about tail behavior.
The Storage Range Tells Its Own Story
After loading the same 2 million resources, the storage footprint per server lands all over the map. Microsoft uses 4.24 GB, Aidbox 6.83 GB, Medplum 11.8 GB, and HAPI 22.6 GB. The benchmark notes that HAPI, Medplum, and Microsoft pre-build search indexes on write, while Aidbox ships without default search indexes and leaves that choice to the operator. The trade-off is real on both sides. Pre-built indexes cost storage and write time; on-demand indexes shift the work to whoever runs the server.
For US emr-development teams thinking about a Synthea-shaped migration, top 5 HL7 v2 parser libraries for FHIR conversion in 2026 covers the upstream piece that feeds into a benchmark like this.
Daily Rerun Is the Part That Sticks
Most vendor benchmarks land as a static PDF, and the conversation moves on. This one runs every day on a public CI box and posts the snapshot to a live dashboard. If any vendor pushes an optimization, the next day's number shows it. If a test configuration is off, anyone can open an issue against the open repo. The Medplum CTO has already forked the project, which is the kind of healthy open-source response that a public benchmark is supposed to produce.
For teams pairing a FHIR server with a terminology pipeline, the trade-off discussion in self-hosted vs SaaS terminology servers for US health networks covers how operational ownership maps onto a similar shape of decision. The benchmark numbers help frame the choice; they do not make it.
What This Does Not Cover Yet
The team behind the benchmark is upfront about scope. The dataset is 1,000 Synthea patients with around 2 million resources, which fits in 500 GB of RAM, so the current snapshot is an in-memory test rather than a disk-bound one. A US hospital network with twenty million patients in its consolidated record store will see numbers that look different once the working set spills out of RAM. The benchmark team has flagged a larger-scale follow-up, which is the run that emr-development teams sizing a real production footprint will want to wait for. In the meantime, the daily snapshot is a useful sanity check on vendor claims.