Deduplicating pages in medical records means removing repeated copies of the same page from a review set while keeping one complete copy and a log of every page you set aside. Work on a copy, never on the produced original, compare pages for stamps and notes before calling them identical, and never renumber what remains.
Why medical record sets contain so many duplicate pages
A medical record set collects duplicate pages because the same visit is documented by more than one source. A hospital, the treating physician and the patient's insurer may each send the same operative report. A duplicate page, for this post, is any page whose clinical content already appears elsewhere in the set.
Three habits produce most of them. The first is overlap between providers: a specialist's consult note is filed in the specialist's chart and again in the referring doctor's chart, so two requests return it twice. The second is copied-forward notes, where an electronic record pulls the prior visit's history, medication list or plan into each new note. Those pages look alike but carry different dates, so they are not true duplicates. The third is re-sent faxes. A records office that resends a batch after a failed transmission, or sends the same response to two addresses, produces whole runs of repeated pages.
A fourth source is the requester. A firm that asks the same provider twice, once by authorization and once by subpoena, often receives two full copies of the chart with different cover letters.
Exact duplicate pages versus near-duplicate pages
An exact duplicate page is a page that matches another page in the set pixel for pixel or byte for byte, usually because the same file was saved twice. A near-duplicate page shares the same clinical text but differs in something added later, such as a fax header, a received stamp, a page number or a handwritten note.
The difference decides what you may remove. Exact duplicates can usually go after a quick check, because nothing distinguishes one copy from the other. Near duplicates need a person to look at both pages, because the difference may be the evidence. A fax header shows when and from where a page was sent. A received stamp shows when a practice got it. A margin note shows that someone read it.
Software can find both kinds. File hashes catch exact duplicates, and text comparison after OCR flags pages whose words mostly match. How OCR text is produced, and where it misreads a page, is covered across the medical OCR basics category. A flagged pair is a suggestion, not a decision. Treat every near-duplicate flag as a page pair for a human to compare.
Which duplicate page should you keep?
The duplicate page to keep is the most complete copy. When copies are equally complete, keep the one with the earliest Bates number, and when one copy carries annotations that matter to the review, keep the annotated copy as well. A Bates number is the unique stamp, a prefix and a running number, placed on each produced page.
The format of that stamp is covered in the guide to what a Bates number on a medical record looks like. Completeness comes first because a later copy is sometimes better. A clean electronic printout may be legible where an early fax copy cut off the bottom line. Earliest Bates number comes second because it gives everyone the same tiebreak, and the citation in a chronology then points to the first place the page appears. Annotated copies come third because they hold information the clean copy lacks.
Here is a decision table with example situations. The Bates numbers are fictional.
| Situation (example) | Keep | Remove from review set | What to log |
|---|---|---|---|
| Same discharge summary saved twice in one production, MRC-000412 and MRC-000413 | MRC-000412 | MRC-000413 | Both numbers, reason "exact duplicate" |
| Lab report from the hospital (HOSP-001020) and from the physician (DRK-000088), identical text | HOSP-001020 | DRK-000088 | Both numbers, the source of each copy |
| Operative note, one copy with a cut-off signature block | The complete copy | The cut-off copy | Which copy was incomplete and why |
| Office note, one copy with a handwritten "reviewed, call patient" | Both copies | Neither | Note the annotation on the kept copy |
| Copied-forward note with the same history but a new date | Both pages | Neither | Not a duplicate; no entry |
When not to remove a duplicate medical record page
A duplicate medical record page should stay in the review set whenever the two copies differ in a way someone may later rely on. Handwritten annotations, a different signature or co-signature, and fax headers that prove when a page was received are the common reasons to keep both copies, even when the typed text matches.
Handwriting shows that a person read the page and what they thought of it. A different signature can mean a note was amended, co-signed by a supervising physician or signed late, and the date of signing may matter. Fax headers and received stamps may be the only proof of when a practice had a result in hand.
Two further cases deserve caution. Pages from different providers are evidence of what each provider had, even when the text is the same, so log the source of every copy you remove. Pages with a different page count footer, such as "page 2 of 4" in one copy and "page 2 of 5" in another, signal a different version of the document. The method for spotting missing pages from those footers is in the post on finding gaps in a set of medical records.
When a person reviews the set for an attorney, ask the reviewer how they handled duplicates and whether they kept a removal log. What that review should deliver is set out in the guide to medical records review for attorneys.
Is deduplication safe after Bates numbering?
Deduplication after Bates numbering is safe when you remove pages only from a working review copy, log every removed page by its number, keep the original numbers on the pages that remain, and never renumber. The produced set stays whole, so any page cited by its Bates number can still be found.
Keep the produced set exactly as it was received. The Federal Rules of Civil Procedure, Rule 34 says a party must produce documents as they are kept in the usual course of business or organize and label them to correspond to the categories in the request. That describes the production itself. A deduplicated copy is a working tool built from it, and it should never replace it.
Gaps in numbering are expected in a deduplicated copy. If MRC-000413 was removed, the review copy jumps from MRC-000412 to MRC-000414, and the removal log explains the gap. Renumbering would break every citation already made to the original pages. The order of pagination and Bates stamping is explained in the guide to whether pagination comes before Bates numbering.
A removal log needs only a few columns: the removed page's number, the number of the kept copy, the reason, the reviewer's initials and the date. When a team then extracts encounter dates from what remains, Schema Care returns each date with a link to the page it came from, and the review step on the homepage shows how a person confirms a field against that page.