FHIR Bulk Data Export: What Developers Should Know

FHIR bulk data export copies many resources out of a FHIR server in one asynchronous job. A client calls $export on the whole server, a Group or all patients, polls a status URL until the job finishes, then downloads NDJSON files listed in a manifest. It suits population-scale copies, not quick single-patient lookups.

This guide describes the HL7 standard only. Schema Care does not export FHIR, has no API, SDK, webhooks, FHIR, X12 or 275 output of any kind, and does not read insurance cards. Every request, header, manifest and NDJSON line below is a made-up example against fhir.example.com, with fictional patients.

What FHIR bulk data export does

FHIR bulk data export is the $export operation defined in the HL7 Bulk Data Access implementation guide. The HL7 Bulk Data Access IG export page, version 3.0.0 (STU 3) and the current published release, defines three kickoff endpoints: one for the whole server, one for all patients and one for a Group of patients.

The three levels answer different questions:

The IG says how Groups are defined is specific to each FHIR system. A payer might send a roster that an EHR imports as a Group, or membership might follow a condition or a recent discharge. Managing Groups through FHIR is out of scope for the current version, so a developer usually learns the Group id from the server's documentation or by searching Group resources, which the IG says a server supporting Group export SHOULD allow.

For patient and Group exports, the IG says the Patient compartment SHOULD be the reference point for what is returned, and resources outside it that help interpret the data, such as Organization and Practitioner, MAY be included. It also says binary content tied to one patient SHALL be serialized as a DocumentReference with the attachment populated. That is how scanned source documents travel in an export. Storing a scan that way in the first place is covered in the section on keeping the source file as a FHIR DocumentReference.

The server also decides what the client may see. The IG says a server SHALL limit the data returned to the FHIR resources the client is authorized to access. Two clients running the same kickoff can therefore receive different files.

The bulk data export kickoff and status polling flow

The bulk data export flow has three steps. The client sends a kickoff request, the server answers 202 Accepted with a Content-Location header, and the client polls that URL. While the job runs, polls return 202. When it finishes, a poll returns 200 with a JSON manifest listing the output files.

The IG says a server SHALL support GET for the kickoff and MAY support POST with a Parameters resource. The client SHOULD send Accept: application/fhir+json, which sets the format of any OperationOutcome error, and SHOULD send Prefer: respond-async. If either is missing, the server may reject the request or proceed as if it had been sent.

A group-level export, as an example

The sequence below is a fictional example. The server, Group id, job id and timestamps are invented.

GET https://fhir.example.com/r4/Group/ex-clinic-roster/$export?_type=Patient,Coverage,DocumentReference
Accept: application/fhir+json
Prefer: respond-async
Authorization: Bearer <example access token>

HTTP/1.1 202 Accepted
Content-Location: https://fhir.example.com/r4/bulkstatus/ex-job-0042

The first poll arrives while the job is still running:

GET https://fhir.example.com/r4/bulkstatus/ex-job-0042
Authorization: Bearer <example access token>

HTTP/1.1 202 Accepted
X-Progress: in progress
Retry-After: 120

The second poll, two minutes later, returns the manifest:

GET https://fhir.example.com/r4/bulkstatus/ex-job-0042
Authorization: Bearer <example access token>

HTTP/1.1 200 OK
Content-Type: application/json
Expires: Sat, 17 Oct 2026 23:59:59 GMT

{
  "transactionTime": "2026-10-16T14:00:00Z",
  "request": "https://fhir.example.com/r4/Group/ex-clinic-roster/$export?_type=Patient,Coverage,DocumentReference",
  "requiresAccessToken": true,
  "output": [
    { "type": "Patient", "url": "https://files.fhir.example.com/ex-job-0042/patient_1.ndjson", "count": 412 },
    { "type": "Coverage", "url": "https://files.fhir.example.com/ex-job-0042/coverage_1.ndjson", "count": 455 },
    { "type": "DocumentReference", "url": "https://files.fhir.example.com/ex-job-0042/docref_1.ndjson", "count": 1290 }
  ],
  "error": []
}

Rules for polling well

The IG says clients SHOULD back off exponentially and SHOULD honor a Retry-After header, which can be a number of seconds or an HTTP date. A server SHOULD answer a client that polls too often with 429 Too Many Requests, and it may end the session if the client keeps going. X-Progress is optional text under 100 characters whose format is up to the server: it may be a percentage or just "in progress", so log it or show it rather than relying on a number.

Three other details matter in code. A failed job returns a 4XX or 5XX status, and the body SHALL be an OperationOutcome. A partly successful job still returns 200, with OperationOutcome resources in NDJSON files listed under error. A client can cancel with DELETE on the status URL, after which the server SHALL return 404 to further polls.

Async job tracking is a common pattern outside FHIR too. The guide to job states in async document processing APIs covers the same queued, running and finished states for extraction jobs.

Reading bulk export NDJSON files

Bulk export output is NDJSON, newline-delimited JSON: one complete FHIR resource on each line. The IG says that when the organizeOutputBy parameter is absent, each file SHALL hold resources of one type, and a server MAY split one type across several files. Read each file line by line rather than loading it whole.

The three lines below are fictional examples, one from each file in the manifest above. Real resources carry many more elements.

{"resourceType":"Patient","id":"ex-p-1","name":[{"family":"Example","given":["Rosa"]}],"birthDate":"1984-03-09","gender":"female"}
{"resourceType":"Coverage","id":"ex-cov-1","status":"active","beneficiary":{"reference":"Patient/ex-p-1"},"subscriberId":"EXM445566001","payor":[{"reference":"Organization/ex-org-plan"}]}
{"resourceType":"DocumentReference","id":"ex-doc-1","status":"current","subject":{"reference":"Patient/ex-p-1"},"content":[{"attachment":{"contentType":"application/pdf","url":"https://files.fhir.example.com/ex-job-0042/attach/ex-doc-1.pdf"}}]}

The IG says references in exported resources MAY be relative, like Patient/ex-p-1, or absolute URLs rooted at the server's base. Your loader should accept both. The Coverage line shows how card fields appear once they are stored as FHIR; the element-by-element mapping is in the guide to the FHIR Coverage resource from insurance cards.

Downloading the files has its own rules. The manifest's requiresAccessToken field says whether each file request needs the same bearer token as the kickoff. When it is false, the URLs work without a token, and the IG says a client SHALL NOT send one. A client SHOULD send Accept-Encoding with gzip, and SHALL accept files that are uncompressed or gzip-compressed. Attachments inside DocumentReference may be inline in data or behind a url, which also follows the token rule.

Plan for size from the start. Stream each line into a parser, write to your store in batches, and record the file URL and line number so a bad record can be found again. Watch the Expires header in the manifest, because files can disappear after that time.

How does a client authenticate for FHIR bulk data export?

A bulk data client usually authenticates with SMART Backend Services. The client registers a public key with the server once, then signs a short JWT with its private key and trades it for an access token through the OAuth 2.0 client credentials grant. No person logs in during the export.

The Bulk Data IG says servers SHOULD implement OAuth 2.0 following the SMART Backend Services profile, which now lives in the SMART App Launch guide. The SMART App Launch Backend Services page, version 2.2.0 (STU 2.2), lays out four steps: register the client, read .well-known/smart-configuration, obtain an access token and call the FHIR API.

The token request is an HTTP POST with these form parameters, per that page:

The signed JWT follows the SMART confidential client with asymmetric keys page. Its iss and sub claims both hold the client_id, aud holds the token URL, jti is a one-time nonce, and exp SHALL be no more than five minutes ahead. Clients SHALL support both RS384 and ES384 signatures.

Access tokens are short-lived on purpose. The Backend Services page says expires_in SHOULD NOT exceed 300 seconds, so a long export will outlast its first token. Request a new token before each poll or download batch instead of caching one for the whole job. The IG also says status and file requests with requiresAccessToken set to true SHALL be protected like the kickoff, with scopes covering every exported resource type.

How do _type and _since filter a bulk data export?

The _type parameter limits a bulk data export to the resource types you name, and _since limits it to resources changed after a given instant. Together they let a scheduled job pull only the types it needs that changed since the last run, instead of copying everything each night.

The IG lists _since as required for servers to support and _type as optional. A server that cannot handle _type SHOULD return an error with an OperationOutcome so the client can retry without it. A Prefer: handling=lenient header lets the server skip unsupported parameters instead of failing.

The manifest supplies the timestamp for the next run. Its transactionTime is the server's time when the query ran, and the IG says the export SHALL include matching resources changed up to that instant. The IG's own example retains transactionTime from one run and sends it as _since on the next:

GET https://fhir.example.com/r4/Group/ex-clinic-roster/$export?_type=Coverage&_since=2026-10-16T14:00:00Z

Use the server's transactionTime, not your own clock. Clocks differ, and your job starts later than the server's snapshot.

Expect some overlap. The IG says a server MAY return resources referenced by the changed ones regardless of their update time, and in a Group export MAY return older resources for patients added to the Group after the _since time. Your loader should upsert by resource id rather than insert blindly. When _since is used, the manifest's deleted array SHOULD list transaction Bundles naming resources deleted since then, each entry with a request method of DELETE.

_since is not a clinical date filter. The IG says filtering by the date of an event, such as encounters in a period, should use _typeFilter, because a resource's modification date may differ from when the event happened.

When not to use FHIR bulk data export

FHIR bulk data export is the wrong tool for one patient, for anything a person is waiting on, and for servers that do not offer it. Each job takes minutes or hours, needs backend credentials, and returns whole files. A plain FHIR read or search answers a single-patient question in one round trip.

Some cases point elsewhere:

• A single-patient lookup at the front desk. A search such as Coverage?patient=ex-p-1 returns in seconds. A Patient-level export would copy every patient's data.
• Real-time feeds. If a downstream system must react within seconds to a new result or admission, an export loop is too slow. A message feed fits better; the guide to HL7 v2 vs FHIR compares the options.
• Servers without support. Not every FHIR server implements $export, and those that do may support only some levels or parameters. Check the server's documentation and CapabilityStatement before you design around it.
• Writing data in. Export only reads. Loading many records into a server is a separate problem.

The opposite direction is a separate pattern. Sending a backlog of scanned documents into an extraction service means many uploads, retries and job tracking, which the guide to batch processing medical documents via API covers.

Bulk export moves data that already exists as FHIR. It does not turn paper into fields. That earlier step is where Schema Care works: it reads intake forms, claims, lab reports and the other document types listed on the homepage, scores every field and routes uncertain ones to a reviewer. It produces no FHIR resources, so an export pipeline would never call it. Field shapes for one paper source, the insurance card, are described in the guide to an insurance card OCR API.

Mistakes that break bulk export clients

• Polling in a tight loop. Honor Retry-After and back off, or expect 429 responses.
• Reusing an expired token. A five-minute token will not last a two-hour export.
• Parsing a file as one JSON document. NDJSON is one resource per line.
• Using local time for _since. Store the manifest's transactionTime instead.
• Ignoring the error array. A 200 can still mean some resources failed.
• Letting files expire. Download before the Expires time, then send DELETE so the server can clean up.