All papersLLMs & Language Models

Redacting Personal Data: A Speed and Accuracy Comparison of Regex, Presidio and Hybrid Designs

/ Paper arXiv 2611.07633

Key numbers

88.9%Regex ran at a 0.20 ms median but caught only 88.9% of PII
100%hybrid and fast hybrid each caught 100% of PII and kept 100% of business terms
0.41%even the slowest option adds only 0.41% to the request

From the paperCopied from the abstract, not written by AI.

Abstract

From the paperThe authors' own words.

Many enterprise projects now pass free text to large language models (LLMs), and that text often contains personally identifiable information (PII) such as names, phone numbers, email addresses, home addresses and dates of birth. Once a prompt is sent, this data is copied into provider systems, application logs and observability traces. Redacting PII before the call is an obvious control, yet teams seldom have numbers on what it costs in latency or what it lets through. This paper compares four redaction designs that any LLM pipeline can adopt: a pure regular-expression (regex) redactor, Microsoft Presidio running on a spaCy model, a hybrid in which Presidio finds only person names while regex handles structured identifiers, and a fast hybrid that calls spaCy's named-entity component directly. All four share a known-entity rule, which removes the person the system already knows it is handling, and a consistent placeholder scheme, so the LLM can still tell one person from another after redaction. We evaluate the designs on a sample workflow, which we call ProfileAssist, that turns resumes into structured profiles, using 100 synthetic resumes (523 characters on average, with 900 PII values and 600 business terms that must survive). Regex ran at a 0.20 ms median but caught only 88.9% of PII, because it cannot find names that are not known in advance, such as the references listed in a resume. Presidio, hybrid and fast hybrid each caught 100% of PII and kept 100% of business terms, at median latencies of 37.59 ms, 30.23 ms and 13.48 ms. Measured against a 9,120 ms median LLM call, even the slowest option adds only 0.41% to the request. We therefore recommend the fast hybrid as a default for LLM pipelines that handle personal data, with regex kept as a safety net rather than as the only control.

From the full paper

From the paperCopied from the paper itself by our code.

Reading the paper on arXiv…

Where it fits

The most cited papers it builds on, and the papers that build on it.

/ Builds on 3 most cited

  1. 20164.5k citedNeural Architectures for Named Entity Recognition ↗
  2. 20203.6k citedExtracting Training Data from Large Language Models ↗
  3. 20231.8k citedArtificial Intelligence Risk Management Framework (AI RMF 1.0)

/ Built on by None yet

No papers cite it yet. We check again every week.

From Semantic Scholar, updated weekly.

Ask this paper

/ Ask a questionAnswers come from the full paper and name the section

    / Your notes

    Select text in the abstract to highlight it or add a note. They stay in this browser.

      Read the PDF