How to clean a Zendesk export for AI

A Zendesk export is a record of how a support system operated. It is not automatically a useful AI dataset. Ticket fields, events, comments, macros, status changes, and user records can all arrive in different shapes. The useful conversation is mixed with routing metadata, internal notes, automated replies, and personal information.

The goal is not to make every row look tidy. The goal is to produce a dataset whose unit, meaning, and privacy properties match the system you plan to build.

1. Start with the intended AI task

Write down the target before changing the export:

• RAG: retrieve prior resolutions, policies, or troubleshooting steps as grounded context.
• Fine-tuning: teach a model response patterns, tone, or task behavior from high-quality examples.
• Evals: build representative cases with expected outcomes to measure another system.

This decision determines what a record should represent. A complete ticket thread can be useful for fine-tuning. A resolved answer with product and issue metadata may work better for retrieval. An eval often needs a prompt, an expected response or rubric, and segmentation fields.

Do not collapse the data into a final format yet. Keep a lossless normalized layer first so you can create more than one downstream dataset without repeating the extraction work.

2. Export the smallest complete source

Zendesk provides account-level export options for tickets, users, and organizations, subject to plan and account settings. Data exports are not enabled by default; the account owner may need to ask Zendesk Support to enable them. Confirm what your account can export in Zendesk's official export guide.

For an initial project, prefer one bounded date range and include the data required to interpret each ticket:

Store the raw export unchanged and record when it was created. Cleaning should produce new files, never overwrite the only source copy.

Before processing, save a small manifest next to the raw files:

export_requested_at: 2026-07-10T09:00:00Z
export_range: 2026-04-01 through 2026-06-30
zendesk_subdomain: example
files_received: 4
raw_bytes: 184293021
sha256_manifest: raw-files.sha256

This creates a reproducible boundary around the source. If another export arrives later, you can prove which files produced which dataset.

Choose the right Zendesk export format

For conversation-level AI work, the file format changes what data is available. Do not choose CSV only because it is easiest to open in a spreadsheet.

Zendesk calls its full JSON format NDJSON: each line is a complete JSON object. Parse it as a stream rather than wrapping a large export in an array and loading the entire file into memory.

Two edge cases need explicit checks. Zendesk does not guarantee record order, so always sort comments using their timestamps. Also, comments can be omitted from a full JSON record when a single ticket exceeds Zendesk's documented 1 MB limit; inspect the companion error file in the download rather than treating that ticket as an empty conversation.

For a first project, a bounded full JSON ticket export plus user, organization, and field-definition lookups is usually the most complete starting point. If you only have CSV, treat it as ticket metadata and obtain comments through another supported export or API before promising a conversational dataset.

3. Build a normalized ticket model

Exports often use IDs where an analyst expects labels. Convert those IDs through explicit lookup tables. Do not guess that a field named custom_field_12345 means “plan tier” because a few values happen to resemble plans.

Write the transformations down as a field map instead of burying them in parser code:

The real mapping should come from your Zendesk field definitions and operating rules. This table is an audit artifact: when a label changes or a new option appears, reviewers can see the effect without reverse-engineering the transformation.

A useful internal model usually includes:

{
  "conversation_id": "ticket_48291",
  "channel": "email",
  "status": "solved",
  "created_at": "2026-06-10T18:42:00Z",
  "metadata": {
    "product": "billing",
    "issue_type": "duplicate_charge"
  },
  "turns": [
    { "role": "customer", "text": "...", "created_at": "..." },
    { "role": "agent", "text": "...", "created_at": "..." }
  ]
}

Preserve stable source IDs in the normalized layer for traceability, but avoid publishing direct customer or user IDs in a downstream training file.

4. Reconstruct the conversation

Sort messages by their actual event timestamps, then assign roles from user type and comment visibility. Treat public agent replies, customer messages, and private internal notes as distinct content classes. Internal notes can contain valuable diagnosis, but silently mixing them into customer-facing replies teaches the wrong behavior.

Handle common thread problems explicitly:

• Remove quoted copies of earlier email messages when they duplicate turns already present.
• Separate system events from human-authored text.
• Mark attachments rather than dropping their existence without explanation.
• Preserve channel changes and handoffs only when they affect interpretation.
• Flag tickets with missing or contradictory timestamps for review.

The output of this stage should read like the conversation a human reviewer would recognize in the Zendesk interface.

Keep the private-note boundary visible in the normalized data even if a downstream output later excludes notes. A response that depends on an internal diagnosis should not become a training example unless the deployed system receives equivalent context.

5. Filter noise with documented rules

Filtering is a quality policy, not a generic “delete short tickets” command. A one-message password-reset resolution might be valuable; a twelve-message autoresponder loop is not.

Create exclusion reasons that can be counted and audited, such as:

• spam or test tickets;
• empty or attachment-only conversations;
• automated acknowledgements with no substantive resolution;
• unresolved threads with no useful agent response;
• exact and near duplicates;
• conversations in an unsupported language or channel.

Keep a rejection log with the source record ID and reason. Sampling excluded records is the quickest way to find an overly broad rule.

6. Redact PII before downstream use

Support conversations can contain names, email addresses, phone numbers, account IDs, order numbers, addresses, payment fragments, secrets, and identifiers specific to your company. Generic pattern matching catches only part of that list.

Run redaction against subjects, message bodies, selected custom fields, and relevant attachment text. Replace entities with typed placeholders such as [EMAIL] or consistent pseudonyms when the conversation needs stable identity across turns. Keep the redaction audit separate from the publishable dataset.

For a deeper workflow, read PII redaction for customer support data.

7. Emit a task-specific dataset

Use the normalized layer to generate the final shape:

• JSONL is convenient for conversation examples and model pipelines.
• Parquet preserves typed columns and works well for analytics and larger retrieval pipelines.
• CSV is inspectable and interoperable, but awkward for nested multi-turn conversations.

Include a schema and a field mapping with the delivery. A dataset is not truly reusable if the next engineer has to infer what each column means.

8. Validate before delivery

Run deterministic checks across every record, then manually review a stratified sample. At minimum, verify:

1. Conversation turns are ordered and roles are correct.
2. Required fields are present and parseable.
3. Duplicate rates are within the expected threshold.
4. Redaction placeholders do not destroy the meaning of the response.
5. Source IDs can be traced through the private audit artifacts.
6. No raw PII appears in the final sample.

Sample across channels, languages, ticket forms, issue types, and time ranges. Fifty random records from the dominant ticket type can miss every important edge case.

Use a measurable quality scorecard

“Looks clean” is not an acceptance criterion. Compare the raw, normalized, excluded, and delivered layers with a scorecard that can be regenerated:

Set acceptance thresholds after profiling the export rather than inventing universal percentages. A billing queue and a community-support queue will have different thread lengths, duplicate rates, and privacy profiles. The invariant is that thresholds are explicit, reviewed, and included in the quality report.

Run the scorecard before and after every rule change. If a new filter removes 30% of one ticket form, inspect that segment before accepting the change.

What a complete delivery contains

A useful delivery is more than one cleaned file. It should include the task-ready dataset, a schema, field mappings, redaction audit, quality report, exclusion counts, and a short summary of decisions. That package lets another person understand what changed without reopening the raw export.

ChatClean applies this workflow to Salesforce, Zendesk, and Intercom exports and returns datasets prepared for RAG, fine-tuning, or evals. Review the project options when you want the cleaning and audit work handled for you.

Common questions about cleaning Zendesk data for AI

Can I use a Zendesk CSV export for RAG or fine-tuning?

Not for conversation content by itself. Zendesk's account-level CSV export omits ticket comments and descriptions. It can still supply useful ticket metadata, but you need a full JSON, full XML, or supported API source for the message thread.

Should internal notes be included?

Preserve them as a separate visibility class in the normalized layer. Exclude them from customer-facing training examples unless the deployed system will receive equivalent internal context and is authorized to use it. Internal notes may still help build private resolution summaries or QA labels.

Do I need to scan data that was already redacted in Zendesk?

Yes. Scan the export independently. Zendesk's in-product redaction has channel and content limitations, and historical exports may include data created before your current redaction process. Treat source-system redaction as one control, not proof that the derived dataset is clear.

How often should the cleaning pipeline be rerun?

For a one-time experiment, use a fixed export and version every artifact. For a production RAG system or recurring model work, use incremental exports, retain a checkpoint, and rerun the same parsing, redaction, quality, and validation gates for each batch.

Sources

• Exporting ticket, user, or organization data — Zendesk
• Redacting ticket content — Zendesk
• Microsoft Presidio documentation