Redacting PII from transcripts: what to remove, what to put in its place, and what over-redaction costs

Decide the policy before the first redaction

Redaction is not one operation with a single right answer. It is a policy question with several defensible answers, and the answer depends on what the transcript is for. A corpus licensed for training a general speech model carries a different obligation from transcripts used for internal research analysis, and both differ from what the speaker agreement may already cover.

Two decisions come first. What counts as identifying in this corpus — a field list, not a principle. And what the replacement has to preserve: whether the redacted text must stay aligned to the audio, whether the same person must map to the same replacement throughout, and whether anything downstream needs to know that a removal happened at all.

The removal list

The structured items are the easy half: phone numbers, email addresses, account and card numbers, government identifiers, license plates, precise street addresses, dates of birth. They have shapes, and shape-based detection catches them reliably.

The hard half is context-dependent, and it is where the misses live.

• Names — not only the speaker's own, but family members, colleagues, and pets mentioned in anecdotes.
• Small-cell identifiers: a job title plus a small town, a company with one office, a school plus a grade. Any combination that narrows to a person.
• Self-disclosures: health conditions, legal matters, financial details, immigration status, relationship facts. Not identifiers in the database sense, and often the most sensitive content in the transcript.
• Third parties: people mentioned but not present, who never consented to anything.
• Location trails: a sequence of place mentions that, taken together, reconstructs a route or a home neighborhood.

What the replacement should do

The replacement is not cosmetic. It decides what the redacted transcript is still good for.

• Consistency: the same person must map to the same replacement everywhere — the same pseudonym, or the same tag. Otherwise the text loses the ability to track who is who, and downstream tasks such as dialogue analysis or coreference become impossible.
• Alignment: a same-length mask keeps character offsets intact, which matters when the transcript is tied to the audio by timestamps or word indices. A bracketed tag is more readable and shifts every offset after it.
• Type information: a tag that says what was removed is more useful than a row of asterisks for most consumers — unless the length of the original is itself sensitive, which for account numbers it can be.
• Reversibility: decide whether a mapping back to the original is kept. If it is, it is a re-identification key and needs its own access control and retention clock. If it is not, that too should be a recorded decision, because it cannot be reversed later.

What over-redaction costs

The failure mode opposite to a leak is a transcript stripped past usefulness, and it is more common than teams expect, because the reviewer who removes too much is never the one blamed when something goes wrong.

• Common nouns caught by name detection: a person whose name is also a verb or an object, and suddenly every instance of that word is gone from the transcript.
• Places that are content: a city mentioned as a business location, a product named after a region, a historical event. Removing them removes information the task needs.
• Domain vocabulary in clinical or legal material: the diagnosis, the charge, the contract term may look like sensitive personal detail and may be the exact content the corpus exists to represent, covered by the consent on file.
• Conversational coherence: aggressive removal of every third party turns a dialogue into a sequence of gaps, and a model trained on it learns to produce gaps.

Run it as a pipeline, and verify with a sample

The working sequence: an automated pass for the structured patterns, a human pass for names and context, then verification — re-scan the output with the same patterns, and read a sample for the context cases that patterns cannot see.

Two quality numbers make the process measurable: the miss rate, from a double-redacted sample, counting how often a second reviewer finds an identifier the first pass left behind; and the over-redaction rate, counting how often a flagged removal turns out to have been ordinary content. Both are sampled and logged the same way as any other QC step, and both feed back into the field list.

One last design note: redaction should leave a trace. A per-file record of how many spans were removed, of which types, is what lets a buyer verify that redaction happened and lets the team see when a batch is anomalous. The trace must not contain the removed values. A log that says one name was removed is an audit trail; a log that says which name is a leak with a file name attached.