De-identification Methods for Enterprise Text Data

1. Introduction#

Before a company can license internal text to an AI developer, someone has to decide what to do about the people named in it. Support tickets contain customer names and order numbers, chat logs contain employee banter and home addresses, emails contain signatures and phone numbers, and source code contains credentials. The standard answer is de-identification, a family of techniques that remove or transform identifying content. The legal paper in this series reviewed what the law expects; this paper reviews the methods, their measured performance, and their documented failures.

Two facts frame the topic. First, text is harder to de-identify than tables. A table has columns, and a column called "name" can be dropped. Free text has no such structure, so identifiers must be found by software before they can be removed, and software misses some of them. Second, removing identifiers does not make data anonymous. Decades of research on re-identification show that combinations of ordinary attributes can single people out, and that language models can both memorise rare strings and infer personal attributes from writing style and context.

This review covers terminology in standards and regulation, the distinction between direct and quasi-identifiers, detection methods for unstructured text (rules, named entity recognition, transformers and large language models), transformation choices (masking versus realistic surrogates), formal methods for structured data (k-anonymity, l-diversity, differential privacy), evaluation metrics and public benchmarks, re-identification case studies, special considerations for four common enterprise data types, and a practical pipeline. It is a literature review, not a security assessment, and it does not certify any tool or method.

2. Methods and sources#

The review draws on official guidance (NIST, the HHS Office for Civil Rights, the European Data Protection Board, the UK Information Commissioner's Office), standards bodies' public descriptions (ISO/IEC), tool documentation (Microsoft Presidio, spaCy, GitHub), and peer-reviewed or archived research papers. Preprints are labelled as such in the text. Sources were opened and read in October 2026. Where a paper's full text was not read, the claim is limited to what its abstract or the cited secondary description states.

Selection favoured work that (a) defines terms used in law or standards, (b) reports measured detection or re-identification performance on named datasets, or (c) documents a real-world failure. The clinical text literature is over-represented because it is the largest body of evaluated work on free-text de-identification, driven by HIPAA; its results are not guaranteed to transfer to business text, a limitation discussed in Section 5. The ISO/IEC 20889 standard was read only through its public catalogue description. Performance figures are reported as published and are not comparable across datasets, annotation schemes or metric definitions unless stated.

3. Findings#

3.1 Terminology: de-identification, pseudonymisation and anonymisation#

The words are used inconsistently. NIST Interagency Report 8053 notes that some authors use "de-identification" and "anonymization" interchangeably, others use de-identification for a process and anonymization for an irreversible subset of it, and that in some healthcare contexts de-identification and pseudonymization are treated as equivalent.¹ The report anchors its usage on an ISO technical specification, which defines de-identification as any process of removing the association between a set of identifying data and the data subject, and pseudonymization as a type of anonymization that also adds an association with one or more pseudonyms; the report itself observes that the standard does not provide unambiguous guidance because some data thought to be anonymous was later re-identified.¹ NIST Special Publication 800-188, published in September 2023, describes de-identification as a general term for any process of removing the association between identifying data and the data subject, and gives specific guidance to government agencies on goals, risks and governance.² ISO/IEC 20889:2018 specifies terminology and a classification of de-identification techniques by their characteristics and their applicability for reducing re-identification risk.³

Legal usage differs. The GDPR defines pseudonymisation as processing so that personal data can no longer be attributed to a specific person without additional information that is kept separately and protected.⁴ Recital 26 says pseudonymised data remains information on an identifiable person, and that the data protection principles do not apply to anonymous information, judged by "all the means reasonably likely to be used."⁵ The EDPB's January 2025 draft guidelines add that pseudonymised data stays personal even where the additional information is held by a different person, and that erasing the additional information does not by itself make the data anonymous.⁶ The guidelines were released as a version for public consultation, and this review did not verify whether a final version exists. The UK ICO makes the same point in a plain statement that pseudonymous data is still personal data because it can be combined with additional information.⁷ Under HIPAA, "de-identified" is a defined status reached by one of two methods (Section 3.2).⁸

Table 1. How the main terms are used in selected sources.

For enterprise text, the practical consequence is that a pipeline that replaces "Jane Smith" with "PERSON_1" and keeps a lookup table is pseudonymisation, and the output is personal data under European law in the hands of anyone with, or able to obtain, the table.

3.2 Direct identifiers, quasi-identifiers and the limits of lists#

NIST IR 8053 describes direct identifiers as data that directly identify a single individual, such as names and Social Security numbers, and notes that the HIPAA Privacy Rule expands the definition to include 18 types of identifiers.¹ Direct identifiers must be removed or transformed during de-identification. Quasi-identifiers, in the privacy-publishing literature, are attributes that are not unique alone but can be combined or linked to external data to single someone out. The k-anonymity model is defined over such attributes: a release is k-anonymous if each record is indistinguishable from at least k minus one other records with respect to the identifying attribute set.⁹¹⁰

HIPAA's Safe Harbor method is the most widely cited list-based approach. It requires removal of 18 categories of identifiers (including names, geographic subdivisions smaller than a state, date elements other than year, contact numbers, Social Security and medical record numbers, account numbers, device identifiers, URLs, IP addresses, biometric identifiers and full-face images) and requires that the covered entity has no actual knowledge that the remainder could identify an individual.⁸ The alternative, Expert Determination, requires a qualified expert to determine that the risk is very small that the information could be used by an anticipated recipient, alone or with other reasonably available information, to identify an individual.⁸ HHS guidance states that there is no explicit numerical level of risk that universally meets the "very small" standard and that the standard makes no distinction between structured fields and free text.¹¹ It also notes that clinical narratives are information rich and may allow identification from context even after listed identifiers are removed.¹¹

The limits of list-based removal are documented. NIST IR 8053 states that even after removal of the 18 Safe Harbor elements, information may remain that allows identification, and that the text de-identification systems it reviewed do not attempt anything beyond removal of those elements.¹ Business text raises the same issue with different content: an order number, a rare product configuration, a job title in a small department, or a description of a distinctive incident can function as a quasi-identifier even though no regulation lists it. A pipeline that removes only named categories will leave these in place.

3.3 Methods for unstructured text#

3.3.1 Rule-based detection

Rule-based detectors use regular expressions, checksums, dictionaries and context words to find identifiers with a regular format: email addresses, phone numbers, payment card numbers (with a Luhn check), national identifiers, IP addresses and URLs. They are fast, deterministic and auditable. NIST IR 8053 summarises the clinical text literature: rule-based systems tend to work well for specific kinds of text but may not work well when applied to new domains, whereas statistical tools generally perform less accurately than rule-based systems and require labeled training data, but are easier to repurpose to new domains.¹ A University of Utah evaluation cited in the same report found that rule-based systems excelled at certain kinds of information such as Social Security numbers and phone numbers, while trainable systems worked better for other kinds, and that no tool tested worked well enough to de-identify the Veterans Health Administration records for public release.¹

3.3.2 Named entity recognition

Named entity recognition (NER) models label spans of text as people, organisations, locations and similar categories. The spaCy documentation describes NER as predicting named entities such as a person, country, product or book title, and cautions that because models are statistical and depend on their training examples, predictions do not always work perfectly and may need tuning for a use case.¹² NER is the standard way to catch names, which cannot be matched by regex. Its weaknesses for business text are domain shift (product names that look like surnames, usernames, ticket-system tokens), casing and spelling variation in chat, and context-dependent decisions about whether a mention is personal.

Microsoft's open-source Presidio combines these techniques. Its documentation describes predefined or custom recognizers that use NER, regular expressions, rule-based logic and checksums with relevant context in multiple languages, an analyzer module that detects entities and an anonymizer module that transforms them with operators such as replace, mask, redact and encrypt.¹³ The same documentation carries an explicit warning that, because it uses automated detection, there is no guarantee that Presidio will find all sensitive information, and that additional systems and protections should be employed.¹³ The documentation also states that the project is transitioning to a community-owned project and is now hosted under the name Data Privacy Stack.¹³

3.3.3 Neural and transformer-based detectors

Supervised neural models trained on annotated text have driven most of the reported gains. Dernoncourt and colleagues presented what they described as the first de-identification system based on artificial neural networks, requiring no handcrafted features, and reported an F1 score of 97.85 on the i2b2 2014 dataset and 99.23 on a MIMIC-based dataset they assembled.¹⁴ Johnson, Bulgarelli and Pollard published a method for free-text medical records using pre-trained bidirectional transformers at the ACM Conference on Health, Inference, and Learning in 2020.¹⁵ A 2025 study in Scientific Reports on clinical notes reported that RoBERTa-large models achieved accuracy over 99% and precision and recall of 96.7% on its test subset.¹⁶ The BigCode project's StarCoder paper describes an encoder model fine-tuned for PII detection in source code, discussed in Section 3.8.¹⁷ All of these are measured on the data they were trained or tested on; the studies do not show equivalent performance on unrelated corporate text.

3.3.4 Large language model redaction

Prompting a general-purpose large language model (LLM) to find and remove identifiers needs no labelled training set and can handle varied formats, and early studies report mixed results. The DeID-GPT preprint, which is not peer reviewed, reported that a GPT-4 based framework showed the highest accuracy and remarkable reliability in masking private information compared with existing methods it tested.¹⁸ A peer-reviewed 2025 study in Scientific Reports evaluating zero-shot GPT-3.5 and GPT-4 on clinical notes from one cancer centre reported GPT-4 precision of 0.9925 but recall of 0.8318 and F1 of 0.8973, with GPT-3.5 at recall 0.6690, precision 0.3448 and F1 0.3905.¹⁶ The high-precision, lower-recall profile is the critical one for privacy: recall measures how many identifiers are actually caught, and each miss is a leak. LLM redaction also sends the raw text to a model, which may itself be a disclosure that needs a legal basis and contract, and it is non-deterministic, so repeated runs can differ. Taken together, current evidence supports LLMs as an additional reviewer or a second pass over output from deterministic and trained detectors, not as a sole control, though the evidence base is small and fast-moving.

3.3.5 Masking, tokenisation and surrogates

Once identifiers are found, the pipeline must decide what replaces them. Masking with a category label ("[PERSON]") preserves structure and makes leaks visible but can reduce readability and model utility. Encryption or keyed hashing supports consistent pseudonyms, so the same customer maps to the same token across tickets, at the cost of creating linkable pseudonymous data and requiring key management. Presidio exposes replace, mask, redact and encrypt operators for these choices.¹³

The "hiding in plain sight" strategy replaces identifiers with realistic synthetic surrogates, so that any identifiers the detector missed are camouflaged among the fakes. Carrell and colleagues tested the idea in a pilot with 31 oncology and 50 family practice progress notes and reported that approximately 90% of residual identifiers could be concealed in text with average and high densities of identifying information, while stating that the result is a feasibility pilot needing replication on larger, more diverse corpora.¹⁹ NIST IR 8053 cites the same study for the finding that reviewers could not distinguish leaked identifiers from surrogates.¹ Surrogates have a cost: they make the output look like real data, they can create accidental real-world matches (a surrogate name that is a real customer), and they can propagate inconsistently if the same entity gets different replacements within a conversation. The i2b2 2014 dataset replaced all annotated authentic identifiers with realistic surrogates before release to researchers.²⁰

3.4 Methods for structured and semi-structured fields#

Enterprise data often pairs free text with structured fields such as account tier, region, product, timestamps and agent ID. The structured side has a mature formal literature.

k-anonymity. Sweeney's 2002 paper frames the problem as releasing person-specific data so that subjects cannot be re-identified while the data remain useful, and defines a release as k-anonymous when each combination of quasi-identifier values appears at least k times; generalisation and suppression are the usual means.¹⁰

l-diversity. Machanavajjhala and colleagues showed with two simple attacks that k-anonymised data can still leak: an attacker can discover sensitive values when there is little diversity in the sensitive attribute within a group, and background knowledge can defeat k-anonymity. They proposed l-diversity as a criterion that defends against such attacks.⁹ For text, the lesson is that a ticket category or product line shared by all members of a small group can reveal what an individual did even without names.

Differential privacy. Dwork, McSherry, Nissim and Smith's 2006 paper sets the noise added to query results according to the function's sensitivity, meaning how much a single record can change the output.²¹ NIST Special Publication 800-226, published in March 2025, describes differential privacy as a mathematical framework that quantifies privacy loss to entities when their data appears in a dataset, and offers a "differential privacy pyramid" and a list of privacy hazards that arise in practice.²² Differential privacy suits aggregate statistics and model training more than the release of individual records. For language models, Yu and colleagues report that differentially private fine-tuning can approach non-private accuracy on some tasks (87.8% versus 90.2% on MNLI with RoBERTa-Large at a privacy budget of 6.7).²³ Brown and colleagues argue, however, that data sanitization and differential privacy rest on narrow assumptions that do not fit the breadth of natural language and privacy as a social norm, and that existing protections cannot guarantee a generic and meaningful notion of privacy for language models.²⁴ Lukas and colleagues found that sentence-level differential privacy reduced PII leakage from fine-tuned models but still leaked about 3% of PII sequences, and that scrubbing reduces but does not prevent leakage.²⁵

3.5 Evaluating de-identification#

3.5.1 Metrics

Detection quality is normally reported as precision, recall and F1. Precision is the fraction of flagged spans that are truly identifiers; recall is the fraction of true identifiers that were flagged; F1 is their harmonic mean. For privacy, recall matters most because each miss is a potential disclosure, and a high F1 can conceal weak recall on rare categories. Evaluation can be token-level or entity-level, with entity-level scoring the stricter, and results can be reported micro-averaged (pooled over all instances) or macro-averaged (averaged over categories). The i2b2 2014 organisers ranked teams on micro-averaged entity-level F1 and noted that this had the strictest rules for a true positive, so those numbers compare less favourably with token-level results elsewhere.²⁰

Span detection is not the same as privacy protection. The Text Anonymization Benchmark (TAB) authors argue that traditional de-identification is limited to predefined semantic categories, and design their corpus to mark which spans ought to be masked to conceal a person's identity, with metrics tailored to both privacy protection and utility preservation.²⁶ Lison and colleagues' ACL 2021 review describes the open challenges as accounting for multiple types of semantic inference, balancing disclosure risk against data utility, and evaluating the quality of anonymised output, and argues for moving beyond sequence labelling models toward explicit measures of disclosure risk.²⁷

3.5.2 Public benchmarks and reported results

Table 2. Selected public benchmarks and reported results for text de-identification.

The pattern is consistent: strong performance on well-defined categories (emails, phone numbers, person names in clinical prose), weaker on ambiguous or sparsely labelled ones (usernames, keys). Benchmarks such as i2b2 use curated, annotated corpora in a single domain, and the studies themselves caution that results vary with training data. Business text has no equivalent public benchmark, so organisations generally need to build a labelled sample of their own data and measure recall on it before relying on a tool.

3.5.3 What human review shows

NIST IR 8053 reports two further findings relevant to evaluation. A study by Meystre and colleagues, summarised by NIST, found that automatically de-identified discharge summaries from a Veterans Health Administration medical center were not recognised by the patient's doctor or treating professional, while the Utah study found that none of the tested tools worked well enough for public release of its records.¹ Results like these show that acceptable average-case scores can coexist with a residual tail of misses, which is one reason HHS frames de-identification as a risk determination rather than a detector score.¹¹

3.6 Re-identification: case studies and measured risk#

3.6.1 Uniqueness of demographics

Sweeney's 2000 working paper, using 1990 US Census summary data, reported that 87% (216 million of 248 million) of the US population had characteristics that likely made them unique on 5-digit ZIP code, gender and date of birth, and that about half were likely unique on place, gender and date of birth.²⁹ Rocher, Hendrickx and de Montjoye (Nature Communications, 2019) built a generative model to estimate the likelihood of correct re-identification in incomplete datasets and reported that 99.98% of Americans would be correctly re-identified in any dataset using 15 demographic attributes, concluding that even heavily sampled anonymised datasets are unlikely to satisfy the GDPR standard.³⁰ De Montjoye and colleagues found in a dataset of fifteen months of mobility for 1.5 million individuals that four spatio-temporal points were enough to uniquely characterise 95% of the traces.³¹

3.6.2 Text and behavioural data

The AOL search log release of 2006 is the standard text example. NIST IR 8053 describes it: AOL removed some identifying information but preserved the search terms, and identified the user with a single numeric code; the code was a random pseudonym and could not itself be reversed, but identifying information appeared in the queries themselves (for instance, people searching for information about their property), and the pseudonym allowed multiple queries from one user to be linked, so journalists identified several users from those terms.¹ The lesson for enterprise text is that the content of free text is itself the quasi-identifier.

Narayanan and Shmatikov demonstrated a de-anonymisation attack on the Netflix Prize dataset of anonymous movie ratings from 500,000 subscribers; using the Internet Movie Database as background knowledge they identified the records of known users, uncovering apparent political preferences and other sensitive information.³² The UK ICO uses the same example when discussing linkability.⁷

Table 3. Documented re-identification results relevant to text and behavioural data.

3.7 Language models: memorisation and inference#

Releasing text to be used for model training adds risks that classical de-identification did not address. Carlini and colleagues showed that an adversary can recover individual training examples from GPT-2 by querying it, extracting hundreds of verbatim sequences including names, phone numbers, email addresses, IRC conversations, code and 128-bit UUIDs, even for sequences present in only one training document, and found that larger models were more vulnerable.³³ Lukas and colleagues described novel attacks that extract up to ten times more PII sequences than existing attacks and noted that the assumption that scrubbing is sufficient is false.²⁵

A second risk is inference rather than memorisation. Staab and colleagues showed that pretrained LLMs can infer personal attributes such as location, income and sex from text, reaching up to 85% top-1 and 95% top-3 accuracy on real Reddit profiles at a fraction of the cost and time of human analysts, and that text anonymisation was currently ineffective at protecting against this inference.³⁴ The study does not measure enterprise text, but it indicates that removing named entities may not remove the stylistic and contextual cues from which attributes can be inferred. A de-identified corpus should therefore be assessed as a corpus: what can be learned from it in aggregate, by whom, and under what contractual limits.

3.8 Special considerations by data type#

Support tickets. Tickets mix a customer's free-form narrative with structured fields and agent notes. Identifiers appear in subject lines, quoted email chains, pasted logs and screenshots-as-text, and customers often disclose health, financial or family details unrelated to the product. Order numbers, serial numbers and account identifiers are quasi-identifiers that regexes should be written for, because default libraries do not know a company's formats. Quoted threads duplicate content, so a miss in one copy can leak in many. The HHS statement that identifiers must be removed wherever recognizable, regardless of whether they are in a structured or free-text field, is a useful design principle even outside healthcare.¹¹

Chat and messaging. Chat is short, informal and multi-party, with nicknames, misspellings, emojis, handles and mentions. NER models trained on formal text degrade on these conditions, and entity consistency across a conversation matters if surrogates are used. Participants are often employees as well as customers, so employee identifiers and internal project code names need their own dictionaries. The inference risk in Section 3.7 is highest for long conversations by one person, where style alone can be distinguishing.

Email. Email adds headers (addresses, routing information, message IDs), signatures with titles and phone numbers, quoted replies, disclaimers and attachments. Rules handle headers and signature blocks well; names in the body need NER. The HIPAA and NIST guidance on quasi-identifiers applies: a thread about a specific, memorable incident can identify participants without any listed identifier.

Source code and configuration. Code contains a distinct class of sensitive content: credentials and keys. Meli, McNiece and Reaves (NDSS 2019) conducted what they described as the first large-scale, longitudinal analysis of secret leakage on GitHub, scanning real-time public commits for nearly six months across a snapshot covering 13% of open-source repositories, and reported more than 100,000 affected repositories and thousands of new unique secrets leaking each day.³⁵ GitHub documents that secret scanning scans the entire Git history on all branches for hardcoded credentials such as API keys, passwords and tokens.³⁶ The BigCode community built a PII dataset of 12,000 files with 22,950 annotated entities using 1,399 crowd workers and fine-tuned an encoder model for PII detection; it reported F1 above 90% for names, emails and IP addresses but 73.39% for passwords and 56.66% for keys and 59.39% for usernames, attributing the weakness on keys to limited labels (308 instances) and noting that the detect-secrets tool with default plugins produced many false positives, with overall precision below 4% on its key benchmark.¹⁷ Secrets scanners and PII detectors are therefore complementary, and any key found should be treated as compromised and rotated, because removing it from a dataset does not revoke it. Code also carries names in commit metadata and comments, and licence obligations that are outside the scope of this paper.

3.9 A practical pipeline#

The sources suggest layering controls instead of relying on one detector. The table below is a synthesis of the reviewed material, not a standard.

Table 4. Layered pipeline for enterprise text, with the source-based rationale for each stage.

4. Discussion#

Four points stand out. The EDPB's Opinion 28/2024 on AI models has supervisory authorities review the controller's documentation when assessing a claim of anonymity, so evidence of method and measurement matters as much as the method.³⁷ First, no source reviewed supports the idea that automated detection can be treated as a guarantee. Presidio's own documentation says it cannot promise to find everything, HHS declines to set a numeric risk level, and the Utah evaluation cited by NIST found that tools tested were not good enough for public release of the records they were given. The defensible claim for most pipelines is a measured reduction in risk together with controls on the recipient, not a claim of anonymity.

Second, recall is the number that matters, and published figures are mostly from clinical corpora with narrow vocabularies. The LLM results show how an impressive F1 can mask a recall gap, and the code results show how category-level performance can range from above 90% to below 60% in a single study. Organisations should measure on their own data.

Third, re-identification research shows that the risk is often in the combination of non-obvious attributes and in the content of free text itself, so removing named entities addresses only part of the problem. This applies with particular force when the data will train a model: memorisation and inference attacks add risk that classical release-and-forget de-identification did not consider, and Rocher and colleagues conclude that the release-and-forget model is legally and technically challenged.³⁰

Fourth, de-identification interacts with law in ways the technical literature does not resolve. Whether the output is anonymous is a legal conclusion that depends on who receives it and what they can do, as the legal paper in this series discusses. The technical pipeline provides evidence for that conclusion; it does not substitute for it.

5. Limitations#

This is a literature review of a field with fast-moving results. Much of the measured evidence comes from clinical text, which has curated corpora but differs from support tickets, chat and code in vocabulary, structure and risk profile. Results are reported as published and use different data, annotation schemes and metric definitions, so they are not comparable across studies. The DeID-GPT paper is a preprint, and LLM results are changing quickly; the studies reviewed pre-date the latest models. The ISO/IEC 20889 standard was read only through its catalogue description, the EDPB pseudonymisation guidelines are a consultation draft, and the Dwork et al. and Lison et al. papers were read through their abstracts and publisher pages. Tool documentation (Presidio, spaCy, GitHub) describes capabilities and not measured performance on business text. The review does not cover speech and image de-identification, synthetic data generation or federated learning, and it provides no performance benchmark for any commercial product.

6. Conclusion#

De-identifying enterprise text is a layered engineering and risk-management task. Deterministic rules, statistical or neural detectors, secrets scanners and a review pass each catch different content, surrogates can soften the effect of misses, and formal methods such as k-anonymity, l-diversity and differential privacy help with structured fields and model training but do not solve free text on their own. The research record shows both strong detection scores on benchmarks and repeated re-identification of data once thought anonymous. The reasonable reading of the evidence is that organisations should measure recall on their own data, treat the output as reduced-risk rather than risk-free, and pair technical controls with contractual limits and documentation of residual risk.