Schema Care

OCR Confidence Score: What the Number Really Tells You

By Schema CarePublished Updated

An OCR confidence score is a number the software attaches to something it read, saying how sure it is of that reading. Depending on the tool, it scores a character, a word or a whole field. Treat it as a pointer to where to look, not a promise: a low score marks the reading most worth checking.

What an OCR confidence score measures

An OCR confidence score measures how strongly the recognition software preferred the reading it returned over the other readings it considered. OCR, optical character recognition, is the step that turns pixels on a scanned page into text. The score belongs to a unit of text, and the unit changes from one tool to the next.

Most engines score at more than one level. The Tesseract documentation's API examples show the open-source engine returning a confidence value for each word, together with the word's bounding box on the page. The same page shows how to ask for a confidence value per symbol, and how to list the alternative symbols the engine weighed, each with its own confidence. It also shows a mean confidence for the text as a whole.

That gives you three different things that can all be called "the confidence score":

  • Character or symbol confidence. How sure the engine is that one glyph is a "5" and not an "S". This is the finest grain, and it is what reveals a single doubtful digit inside a long member ID.
  • Word confidence. One number for a run of characters between spaces. A word score usually hides which character caused the doubt.
  • Field confidence. One number for a value an extraction system has placed into a named field, such as "allowed amount" or "date of service". This is the score a billing team cares about, because a field is what lands in a patient's account.

Before you compare scores from two systems, or two reports from the same system, ask which unit each one describes. A 95 on a character and a 95 on a field are not the same claim.

Where a document extraction confidence score comes from

A document extraction confidence score is the number an extraction system attaches to a finished field, after it has read the text and matched it to a field. It can combine several signals: how clearly the characters were read, how sure the system is that it found the right spot, and whether the value fits the field's expected shape.

Think of a field as the end of a short chain of decisions. First the system has to find the region of the page that holds the value. Then it reads the characters in that region. Then it decides that this string is the allowed amount and not the billed amount printed one column to the left. Each step can go wrong on its own, and a well-built score reflects all of them, not only the reading step.

Models that produce these numbers are usually classifiers: for each decision they assign a probability-like value to each possible answer and pick the highest. The paper On Calibration of Modern Neural Networks (Guo et al., ICML 2017) describes the goal as producing probability estimates "representative of the true correctness likelihood", and reports that modern neural networks are poorly calibrated, meaning the raw numbers often do not meet that goal. A field score built on top of such numbers inherits the same question.

How a vendor rolls character and location scores into one field number varies, and few publish the exact formula. For developers, the developer integration category covers how scores appear in structured output and how to store them next to each value.

Is a 98% OCR confidence score the same as 98% accuracy?

A 98% OCR confidence score is not automatically the same as 98% accuracy. It equals accuracy only when the system is calibrated, which means that among all fields scored near 98%, about 98 out of 100 turn out to be right. Many systems are not calibrated out of the box, so the score is better read as a ranking.

A ranking is still useful. If fields scored 98% are wrong less often than fields scored 85%, the score is doing its main job: putting the risky values at the front of the queue. What it does not do, on its own, is tell you the error rate you will see in production.

Here is a simple example with made-up numbers. Suppose a test set has 200 fields that the system scored between 97% and 99%. If the scores matched reality, you would expect about 4 of those 200 to be wrong. If you check them against the paper documents and find 14 wrong, the system is overconfident in that band: it claims about 98% and delivers 93%.

Whether a given system's numbers line up with reality is a measurement you make on your own documents, and it has its own guide: what confidence score calibration means walks through the gap between a score and observed accuracy, and how to test for it.

Reading OCR confidence scores on a fictional EOB

Reading OCR confidence scores on one document shows how much they vary by field type. An explanation of benefits (EOB) is the payer's statement of how it decided a claim, and its fields range from free text, such as names, to short codes. The table below uses a fictional EOB, with every value and score made up for the example.

FieldExample valueExample confidenceAt a 90% threshold
Patient nameDana Whitfield97%Accepted
Member IDXKW44819023784%Sent to review
Date of service03/14/202695%Accepted
Allowed amount$160.0092%Accepted
Plan paid$96.0088%Sent to review
Remark codeN13062%Sent to review

Three patterns in the example are worth noticing. The member ID scores lower than the name because it is a long string of letters and digits with no dictionary to lean on, so one smudged character drags the field down. The plan paid amount sits just under the line, which is the band where a reviewer's time is cheapest to spend: one glance at the source page settles it. The remark code scores lowest because it is short, often printed small, and can sit in a footnote area far from the service line it explains.

In this example three of six fields go to a person and three pass straight through. Raise the threshold to 96% and the date of service and allowed amount join the queue too; lower it to 80% and only the remark code is checked. That trade-off is the subject of how to set a confidence threshold for extraction, and choosing the threshold a field must clear starts from a labeled sample rather than a guess.

You can move a threshold yourself on Schema Care's fictional EOB demo with a threshold slider, which runs from 70% to 99% and starts at 90%, and watch which fields change from accepted to review.

What should you do with a low OCR confidence score?

A low OCR confidence score should send the value to a person who can compare it with the source page, not straight into your billing system. Whether that person approves it, corrects it or asks for a new scan depends on why the score is low: a faint image, an unusual layout or a genuinely ambiguous value.

A practical order of checks for each low-scoring field:

  1. Look at the field on the page. A reviewer needs the original image beside the extracted value, ideally with the region highlighted. Without that, review becomes re-keying from memory.
  2. Approve if the value is right. Many low scores turn out to be correct readings of hard-to-read print. Approving them is quick.
  3. Correct if the value is wrong but readable. Type the right value and keep a record of the change.
  4. Re-scan if the page itself is the problem. The Tesseract guide to improving output quality says the engine works best at a resolution of at least 300 dpi and that line segmentation suffers badly when a page is too skewed. When many fields on one page score low together, the scan is the likely cause, and a better image fixes all of them at once.
  5. Escalate if no one can read it. Some values are illegible on the paper too. Those need a call to the payer or the sender, not a guess.

Schema Care follows the first three steps: each field arrives with its score and a link to where it sits on the page, anything under your threshold waits in a review queue, and the reviewer sees the value highlighted on the original before approving or correcting it.

OCR confidence score mistakes: averaging, rounding and hiding

OCR confidence score mistakes usually come from simplifying the number for display. Averaging many field scores into one, rounding scores into broad labels, or replacing them with a green check all remove the information a reviewer needs. Each shortcut can let a single wrong value pass because the summary looked fine.

Averaging. A page average of 0.95 can contain one field at 0.60. The strong fields pull the mean up and the weak one disappears. In the fictional EOB above, the six scores average 86%, which sits under a 90% threshold and would flag the whole page while saying nothing about which three fields need a look. The reverse also happens on longer documents, where dozens of clean fields can lift the average over the line and hide a bad one. The comparison of field level vs document level confidence covers which score should decide review.

Rounding. Showing 89.6% as "90%" moves a field across a 90% threshold on screen while the system still treats it as under. Reviewers then see a queue item that looks like it should have passed and lose trust in the routing. Display the same precision the routing uses, or show the threshold decision itself.

Hiding. Some screens replace the score with a green check or a red flag. That is fine for the routing decision, but the reviewer should still be able to see the number, because a field at 89% and a field at 41% call for different amounts of care.

Comparing across systems. A score from one engine and a score from another are on different scales unless both have been checked against the same labeled documents. Moving to a new tool means setting new thresholds from new measurements, not copying the old numbers across.

Treating the score as permanent. A score reflects how the system behaved on the documents it has seen. When a payer redesigns its EOB layout, the same numbers can start to mean something different. Re-check scores against a small labeled sample after any big change in the documents you receive.

All posts in Human Review and Confidence