> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://mezmo.ferndocs.com/ai-guides-dedupe-processor/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://mezmo.ferndocs.com/_mcp/server.

# Deduplication Processor

## Deduplication Processor

**Purpose:** Eliminates duplicate events in your pipeline, reducing data sent to the final destination.

**Key Features:**

* **Duplicate Detection:** Compare events based on specified fields (e.g., `ignore`, `match` logic).
* **Customization:** Define how many events to compare and which fields to deduplicate.

**Steps to Add:**

1. Edit the pipeline and click **Add Processor**.
2. Select **Deduplication Processor** and configure:

* Title/Description for clarity.
* Comparison logic (e.g., number of events to compare).
* Fields used for deduplication.