Measuring field level extraction accuracy means comparing each extracted field with a labelled correct value, scoring it right or wrong under a written match rule, and reporting results per field type. Blank fields need their own rule, wrong values and missed values are counted separately, and one overall average is never enough on its own.
What field level extraction accuracy means
Field level extraction accuracy is the share of fields, such as a member ID or a plan payment, that an extraction tool returns with the correct value. A field is one labelled piece of data on a document. Scoring fields matches how a billing team uses the output: a field is either safe to post or it is not.
Two other ways of counting are common, and both answer different questions.
Character-level scoring counts how many characters were read correctly. The OCR-D quality assurance specification defines the character error rate as "the quotient of the edit distance over the length" of the correct text and the OCR text, where edit distance is the minimum number of single-character insertions, deletions and substitutions needed to turn one string into the other. That is the right measure for judging how well text was read. It is the wrong measure for judging whether data can be posted, because one wrong digit in a ten-digit member ID is a small character error and a completely wrong field.
Document-level scoring counts a document as right only when every field on it is right. That is useful for planning review work, since a document with any wrong field needs a person to touch it. It hides which fields fail, though. A claim with forty fields and one bad modifier scores the same as a claim where every amount is wrong.
Field-level scoring sits between the two. It tells you which kinds of fields you can trust, which is what you need to set thresholds and decide where review effort goes. It also maps onto the way labelled test data is usually built: one expected value per field per document. What that labelled set holds, and how to build it, is set out in the guide to what a ground truth test set for medical documents contains.
Exact match, normalized match and partial credit
Exact match, normalized match and partial credit are three rules for deciding whether an extracted field equals its expected value. Exact match compares strings character for character. Normalized match first converts both values to a standard form, so "$1,250.00" and "1250" can count as equal. Partial credit gives a fraction of a point for values that are close.
Exact match
Exact match is the strictest rule and the simplest to apply. It suits fields whose format is fixed and where any difference is a real error: procedure codes, member IDs, claim numbers, NPIs. Exact match punishes harmless differences elsewhere, though. A date written "03/14/2026" on the page and returned as "2026-03-14" fails exact match while carrying the same date.
Normalized match
Normalized match applies a written conversion to both values before comparing them. Typical normalizations, decided per field type before scoring starts:
- Amounts: strip currency symbols and thousands separators, then compare as numbers to the cent. "$1,250.00", "1,250.00" and "1250" all become 1250.00.
- Dates: parse to year, month and day. "03/14/2026" and "2026-03-14" match; "03/14/2025" does not.
- Names: fold case and collapse spaces. "RIVERA, ANA" and "Rivera, Ana" match.
- Codes: trim spaces only. Removing characters from codes changes their meaning, so keep the rule tight.
The danger is normalizing away real errors. If the amount rule drops decimal points instead of parsing them, "125000" and "1250.00" become the same number. Write each rule down, and test it on a few tricky pairs before you score anything.
Partial credit
Partial credit scores a field by how close it is, often using edit distance. It answers "how nearly right was the read?", which helps when you are comparing reading quality. It is rarely right for data you will post. A total charge of $1,140.00 against an expected $1,410.00 is two swapped digits and close by edit distance, and it is wrong for payment. Use partial credit for free-text fields such as a referring provider's name if your use tolerates near-matches, and never for amounts, codes or identifiers.
Worked scoring table (fictional claim)
This table scores ten fields from a fictional professional claim. The patient, member ID, account number and amounts are made-up examples.
| # | Field | Field type | Expected value | Extracted value | Match rule | Score |
|---|---|---|---|---|---|---|
| 1 | Patient name | Descriptive | RIVERA, ANA | Rivera, Ana | Normalized (case, spaces) | 1 |
| 2 | Member ID | Identifier | XKM400271193 | XKM40027I193 | Exact | 0 |
| 3 | Patient account number | Identifier | ACCT-88213 | ACCT-88213 | Exact | 1 |
| 4 | Date of service | Date | 03/14/2026 | 2026-03-14 | Normalized (date) | 1 |
| 5 | Procedure code | Code | 99214 | 99214 | Exact | 1 |
| 6 | Modifier | Code | 25 | (blank) | Exact | 0 |
| 7 | Diagnosis code | Code | J06.9 | J06.9 | Exact | 1 |
| 8 | Line charge | Amount | $1,250.00 | 1250 | Normalized (amount) | 1 |
| 9 | Total charge | Amount | $1,410.00 | $1,140.00 | Normalized (amount) | 0 |
| 10 | Referring provider name | Descriptive | (blank on page) | PATEL, R | Blank rule | 0 |
The per-field-type summary it rolls up to:
| Field type | Fields | Correct | Field accuracy | Precision | Recall |
|---|---|---|---|---|---|
| Identifier | 2 | 1 | 50% | 1 of 2 (50%) | 1 of 2 (50%) |
| Code | 3 | 2 | 67% | 2 of 2 (100%) | 2 of 3 (67%) |
| Date | 1 | 1 | 100% | 1 of 1 (100%) | 1 of 1 (100%) |
| Amount | 2 | 1 | 50% | 1 of 2 (50%) | 1 of 2 (50%) |
| Descriptive | 2 | 1 | 50% | 1 of 2 (50%) | 1 of 1 (100%) |
| All fields | 10 | 6 | 60% | 6 of 9 (67%) | 6 of 9 (67%) |
Ten fields are far too few to judge any tool; the table only shows the mechanics. In a real test the same roll-up runs over thousands of fields.
How should blank fields count toward extraction accuracy?
Blank fields count toward extraction accuracy under a rule you set before scoring: a field empty on the page and empty in the output is correct, a value returned for an empty field is a false value, and an empty output for a filled field is a miss. Keeping these cases apart stops blanks from inflating or hiding errors.
Comparing the page with the output gives five outcomes:
| On the page | In the output | How to score it |
|---|---|---|
| Value | Same value | Correct |
| Value | Different value | Wrong value |
| Value | Blank | Missed value |
| Blank | Blank | Correct blank |
| Blank | Any value | False value |
The fifth row is the one teams forget. In the worked table, the referring provider box was empty and the tool returned "PATEL, R", probably read from a neighbouring box. That is a real error: a posted claim would now name a referring provider who was never on the form.
Correct blanks need care in the other direction. Many forms have far more empty boxes than filled ones. If every correct blank counts as a point, a tool that returns nothing at all can post a high accuracy figure. Report correct blanks as their own count, and compute the headline accuracy on fields that hold a value on the page, or on all fields, but say which.
The expected value must also say why a field is blank. "Blank on the page" and "unreadable on the page" are different labels. An unreadable field should not count against the tool for being blank, but it should count against it for returning a confident guess.
Precision and recall per extracted field type
Precision and recall per extracted field type separate two failures that one accuracy figure blends together: returning wrong values and missing values. Precision is the share of returned values that are correct. Recall is the share of values present on the page that were returned correctly. Each field type gets its own pair.
The definitions come from information retrieval. The JAMIA paper on agreement and the F-measure by Hripcsak and Rothschild defines recall as "the proportion of positive cases in the gold standard that were classified as positive", equivalent to sensitivity, and precision as equivalent to positive predictive value. The F-measure is the harmonic mean of the two.
For extraction, translate them this way. A "returned value" is any non-blank output. A "value present on the page" is any field whose expected value is not blank. A wrong value counts against both: it lowers precision because a returned value is wrong, and it lowers recall because the correct value was not delivered.
In the worked table, codes have perfect precision and weaker recall: every code the tool returned was right, but it missed the modifier. Descriptive fields are the opposite: the tool found the patient name but also invented a referring provider. Those call for different fixes. Low recall points to fields the tool does not locate, such as a box on an unfamiliar layout. Low precision points to fields it fills when it should not, which a stricter threshold or a cross-field rule can catch.
For a billing team, precision on amounts usually matters most, because a wrong amount that passes review gets posted. A missed amount at least shows up as a gap someone has to fill. The error types behind wrong values, such as transposed digits and dropped decimals, are catalogued in the guide to medical record OCR error types: swapped digits, lost decimals and merged lines.
Why does an average extraction accuracy number hide errors?
An average extraction accuracy number hides errors because it weights every field equally, so common, easy fields swamp rare, important ones. A high overall figure can sit on top of a field type that fails often. Reporting per field type, with the field counts beside each rate, is the only way to see it.
A fictional example of masking
Suppose a test covers 1,000 fields: 800 names, dates and identifiers, and 200 dollar amounts. These numbers are invented for the arithmetic. If 792 of the 800 are right (99%) and 170 of the 200 amounts are right (85%), the overall figure is 962 of 1,000, or 96.2%. That looks strong. It also means roughly one amount in seven is wrong, which no billing team would accept on payments.
The same thing happens across document types. An average over EOBs and intake forms can look fine while one payer's EOB layout fails on most of its amounts. Report by document type and by layout as well as by field type when the test set is big enough to support it.
What to report instead
A useful accuracy report has, for each field type: the number of fields scored, field accuracy, precision, recall, and the match rule used. Add the count of correct blanks separately. Put the per-type table first and any overall figure last.
Field counts matter as much as rates. A field type with 12 examples and 100% accuracy tells you little; one wrong value would drop it to 92%. When counts are small, say so rather than presenting the rate as settled.
Agreement between two people is measured the same way when you build the labels in the first place, and registries have long practice with it; the guide to clinical registry abstraction shows how abstractor agreement is checked field by field. When you are ready to apply this scoring to a live evaluation, the guide to running an accuracy test during a pilot turns it into a plan. Schema Care scores its confidence in every field it reads from the document types it handles, which makes a per-field-type report the natural way to judge its output against your own labels.