PII redaction for customer support data: a practical guide

Customer support data is unusually rich in personal information. Customers describe account problems in free text, agents repeat identifiers while troubleshooting, and custom fields store values that generic detectors have never seen. Before this data enters a retrieval index, training run, or evaluation set, teams need a redaction process they can explain and test.

Redaction is not a single model call. It is a pipeline that defines sensitive entities, detects candidates, chooses replacements, verifies the result, and records what happened.

Define the privacy boundary first

Start by documenting where the cleaned dataset will go and who will access it. The right treatment depends on whether data is used for internal analysis, indexed for retrieval, shared with a vendor, or included in model training.

Build an entity inventory across four groups:

The last two groups are where generic PII tools often need company-specific rules. An identifier such as CUS-847291 may be meaningless outside your systems, but it can still connect a conversation to a person inside them.

This guide describes engineering practice, not a legal determination. Your privacy or legal owner should decide which fields and jurisdictions apply to your organization.

Inventory every content surface

Scanning only the main message column creates blind spots. Customer data can appear in:

• ticket subjects and descriptions;
• public replies and private notes;
• user and organization profiles;
• custom fields, tags, and form values;
• attachment names and extracted attachment text;
• event metadata and URLs.

Create a data map that says which surfaces are included, excluded, or transformed. Preserve the raw source in a controlled location, then perform all redaction on derived copies.

Combine detection methods

No detector is reliable enough to handle every entity type alone. A practical stack combines complementary methods:

1. Deterministic patterns for emails, phone numbers, IP addresses, card-like strings, and known identifier formats.
2. Named-entity recognition for names, locations, organizations, and context-dependent text.
3. Dictionaries and allowlists for employee names, product names, city names, and terms that create recurring false positives.
4. Custom recognizers for customer IDs, order numbers, internal domains, and company-specific secrets.
5. Human review for ambiguous high-risk cases and representative quality sampling.

Microsoft Presidio is one open-source framework for combining recognizers and anonymization operators. Whatever tool you choose, evaluate it against your own support data rather than relying on a generic benchmark.

Choose replacements that preserve utility

Deleting every detected value can make a conversation unreadable. Typed placeholders preserve meaning:

Before: Email sarah@acme.io about order ORD-58213.
After:  Email [EMAIL] about order [ORDER_ID].

There are three common replacement policies:

• Typed redaction: replace each value with its entity class. This is simple and easy to audit.
• Consistent pseudonymization: replace the same source value with the same synthetic token inside an approved scope. This preserves multi-turn identity relationships.
• Generalization: reduce precision, such as converting an exact age to an age range when that is sufficient for the task.

Do not use one global pseudonym map indefinitely. Define its scope, protect the mapping, and delete it according to your retention policy. A reversible mapping is sensitive data too.

Handle context, collisions, and false positives

Support text is full of strings that resemble PII. Product versions can resemble dates. Ticket numbers can resemble phone numbers. A customer named May can collide with the month.

Use context to adjust confidence, but do not hide uncertainty. Store the detector, entity type, confidence or rule, and final action for each candidate. Then review:

• high-risk entities even when confidence is high;
• borderline detections near the decision threshold;
• examples from every custom recognizer;
• unredacted samples selected by issue type and channel;
• false-positive-heavy fields such as product codes.

Precision and recall have different costs. Low recall leaves sensitive data behind. Low precision destroys useful context. Your QA sample should measure both.

Keep an audit log without recreating the leak

An audit log should prove that the process ran without becoming a convenient copy of every secret. A useful record contains:

source_record_id, field, entity_type, detector, action, confidence
ticket_48291, comment_3, EMAIL, email_pattern, replace, 1.00

Avoid storing the original entity value in a broadly accessible CSV. If a reversible value is operationally required, keep it encrypted and access-controlled separately from the cleaning report.

Aggregate reporting should show counts by entity type, field, detector, and outcome. Large changes between runs can reveal a broken parser or recognizer before the dataset is released.

Redact before indexing or training

The safest place for required redaction is before data is chunked, embedded, copied into a training file, or sent to downstream tooling. Masking only the final generated answer leaves sensitive source text in intermediate systems and logs.

Retain stable non-sensitive metadata needed for authorization and filtering. PII redaction does not replace source-system access controls; a retrieval system may still need tenant, team, or entitlement metadata to prevent cross-user access.

Test the finished dataset

Use a repeatable release gate:

1. Run exact validators for known patterns and forbidden domains.
2. Re-scan the cleaned output with a detector configuration independent from the primary pass where possible.
3. Review a stratified sample across languages, channels, forms, and time periods.
4. Confirm placeholders remain consistent where identity continuity matters.
5. Inspect false positives to ensure troubleshooting steps remain usable.
6. Record reviewer, date, sample method, findings, and remediation.

Synthetic test cases are useful for regression testing, but they cannot replace samples from the actual data distribution. Real support conversations contain spelling errors, pasted logs, partial identifiers, and unexpected formatting.

Operational checklist

Before releasing a customer support dataset, confirm that you have:

• a written entity policy and surface inventory;
• raw and derived storage separated;
• deterministic, statistical, and custom detection coverage;
• an explicit replacement policy;
• a protected redaction audit;
• automated checks plus human QA;
• retention and deletion rules for sources and mappings;
• a documented owner for exceptions.

If you are starting from a platform export, the companion guide on cleaning a Zendesk export for AI shows where redaction fits in the broader pipeline. ChatClean delivers the cleaned dataset with a redaction audit and quality report; review project pricing for a managed workflow.

Sources

• Microsoft Presidio documentation
• NIST Privacy Framework
• Redacting ticket content — Zendesk