Unstructured healthcare data extraction is the work of reading free-text documents, such as progress notes, dictated letters and narrative summaries, and turning the facts in them into named fields. The hard part is meaning: a phrase can be denied, belong to a relative, or describe the past, so every value needs its context and a link back to the sentence.
What makes healthcare data unstructured?
Healthcare data is unstructured when the facts sit inside sentences instead of labelled boxes, so nothing on the page says which words are a value and which field they belong to. A narrative note can mention a medication, a symptom and a family member in one line, in any order, in the writer's own phrasing.
The terms themselves are defined in the post on structured vs unstructured healthcare data, and the wider background on reading medical pages sits in the medical OCR basics category. This post starts where that one stops: how fields get pulled out of the text.
Narrative is a recognized kind of clinical data, not a failure to fill in a form. The ONC USCDI page has a Clinical Notes data class, described as "Narrative patient data relevant to the context identified by note types", with note types such as Consultation Note, Discharge Summary Note, History & Physical, Procedure Note and Progress Note. Even in fully electronic records, much of what clinicians write is prose.
Three features make narrative harder than forms:
- No fixed position. On a claim form, the date of service is always in the same box. In a note, a date can appear anywhere, and several dates can appear in one paragraph.
- Variable wording. The same fact can be written many ways: "no known allergies", "NKA", "allergies: none", "denies allergies".
- Context changes meaning. "Chest pain" in a sentence that begins "denies" is the opposite of a finding. A word can be present on the page and absent from the patient's record.
That last feature is what separates extraction from search. Finding the words is easy. Deciding what they mean for a field is the job.
Templates, rules and language models for unstructured healthcare text
There are three broad approaches to extracting fields from unstructured healthcare text: templates that read fixed positions, rules that match patterns of words, and language models that read whole sentences. They differ in what they need to set up, what kind of text they handle, and how their mistakes look when they happen.
Templates map a region of the page to a field. They work when the text sits in a known place, such as a typed header or a labelled line. They fail on true narrative, because there is no region to map. A template is the right tool for the header of a note and the wrong one for its body.
Rules match word patterns. A rule might say: a drug name followed by a number and "mg" is a medication with a strength. Rules are transparent, since a person can read exactly why a value was pulled. They are also brittle. Every new phrasing needs a new rule, and a long rule set becomes hard to maintain. The widely studied NegEx method is a rule approach of this kind: a paper in the Journal of Biomedical Informatics, DEEPEN, a negation detection system for clinical text, describes NegEx as processing one sentence at a time by finding negation and termination terms.
Language models read the whole sentence and predict the fields. They handle varied wording far better than rules. Their errors are less predictable, and a model can produce a value that is not in the text at all. That makes a link to the exact source sentence essential for every value a model returns.
A short comparison:
| Approach | Setup | Handles varied wording | Typical failure |
|---|---|---|---|
| Template | Map regions per layout | No | Value missing when layout shifts |
| Rules | Write and test patterns | Partly | New phrasing not matched |
| Language model | Define fields and examples | Yes | Plausible value with no source in the text |
Scanned narrative adds a layer of risk under all three. A note that arrives as a faxed image first has to be turned into text by optical character recognition, and a misread letter can change a word. "No" read as "Mo", or a dose of "1.5" read as "15", passes straight into whichever approach comes next. Rules miss the misread trigger word. A model may guess around it. Keeping the source location for each value is what lets a person catch these errors at review, because the reviewer sees the page, not only the text the scanner produced.
Many working systems combine them: a template for the header, rules for tightly formatted lines such as medication doses, and a model for the narrative.
Negation and context in unstructured clinical text
Negation and context in unstructured clinical text decide whether a phrase becomes a field value at all, and if so, which one. A condition named in a note can be denied ("denies chest pain"), belong to someone else ("family history of"), lie in the past, or be only a possibility. Each case changes the extracted field.
Published research describes methods for it. The BMC Bioinformatics paper ContextD, on contextual properties of medical terms, describes the ConText algorithm, which uses regular expressions and lists of trigger terms to decide whether a condition is negated, whether it is recent, historical or hypothetical, and whether the patient or someone else, such as a family member, experienced it. That same paper explains that termination triggers limit how far a trigger's effect reaches in a sentence.
For a paperwork team, the lesson is practical. A field for "conditions mentioned" needs more than the condition name. It needs at least three companion values: present or absent, patient or other person, and current or past. Without them, a list of extracted conditions mixes things the patient has with things the patient was asked about.
Five fictional sentences, read two ways
These sentences are invented for the example. They are not from any real patient, and the right-hand column describes how the text should be recorded, not any clinical judgment about it.
| Note sentence (fictional) | Naive reading | Correct extracted field |
|---|---|---|
| "Patient denies chest pain or shortness of breath." | Symptoms: chest pain, shortness of breath | Symptom: chest pain, status absent; symptom: shortness of breath, status absent |
| "Family history of colon cancer in father." | Problem: colon cancer | Family history entry: colon cancer, relative father; patient problem list unchanged |
| "History of appendectomy in 2009." | Procedure this visit: appendectomy | Past procedure: appendectomy, year 2009 |
| "Return if fever develops." | Symptom: fever | Instruction text: "Return if fever develops"; no symptom recorded |
| "No longer taking example drug B since March." | Current medication: example drug B | Medication: example drug B, status stopped, stop date "since March" as written |
Every naive reading in the middle column contains the right words. Every one would put a wrong value in a record. The fix is not a better text scanner. It is a field design that captures the context, plus a person who checks the values the software is unsure of.
How do you check fields pulled from unstructured healthcare text?
Fields pulled from unstructured healthcare text are checked by tying each value to the exact sentence it came from, then having a person compare the two when confidence is low. On a form, the box location proves where a value came from. In narrative, the only proof is the source sentence, highlighted on the original page.
A good check record for one narrative field holds:
- The value, such as "colon cancer".
- The context values, such as relative father, status present.
- The source sentence, copied word for word.
- The page and position of that sentence in the original document.
- A confidence score, a number estimating how likely the value is right.
With that record, a reviewer reads one sentence, not one note. Approving or correcting takes seconds. Without it, the reviewer has to reread the whole document to find where a value came from, and many will skip the check.
Sampling helps too. Even fields that score well deserve a periodic spot-check, because a consistent error, such as one negation phrase the system never catches, can score high every time. Pull a small sample weekly and read each value against its sentence.
The same principle applies to summaries of whole records: any line produced from narrative needs a pointer back to its page. That argument is made in full in the post on why AI summaries of medical records need source links, and the buying questions for summary tools are in medical record summarization. Schema Care follows this pattern for its document fields: each one arrives with a confidence score and a link to its place on the page, and a reviewer approves or corrects the uncertain ones with the source highlighted, as shown in the human review steps.
When templates are enough for semi-structured healthcare documents
Templates are enough for semi-structured healthcare documents when the facts you need sit in labelled places that stay put from one copy to the next. A semi-structured document mixes fixed labels with some free text: intake forms, lab report headers, medication lists with column headings, and typed note headers all qualify.
Before reaching for a language model, ask what share of your fields are truly narrative. A clinic pulling patient name, date of birth, encounter date, author and allergy list from typed notes may find that every one of those sits on a labelled line. A template or a short set of rules handles them, and narrative extraction would add cost and new error types for no gain.
Signs a template will do:
- Labels repeat. "Allergies:", "Medications:" and "Date of service:" appear on every copy.
- Few layouts. Your documents come from a small number of sources, each with a stable format.
- Short values. The fields are dates, names, codes and short lists, not sentences.
Signs it will not:
- Dictated notes with no printed labels.
- Many senders, each with its own layout and wording.
- Fields that depend on context, such as whether a condition was denied or belongs to a relative.
The common middle ground is a document with a structured header and a narrative body. Use a template for the header and a text approach for the body. Progress notes, which often follow a SOAP layout with headings for each section, are a good example; the guide to clinical notes extraction covers how far those headings get you and where the narrative takes over.