Deduplicate Transform

Deduplicate Transform

Overview

The Deduplicate transform removes exact duplicate records from a dataset.

The Deduplicate transform only removes rows where the following condition is true:

  • For any two or more rows, the values in every column are the same.
Usage

The Deduplicate transform is case-sensitive. You cannot modify the Deduplicate transform with any parameters. Use the following general format for the Deduplicate transform:
Choose a transformation
deduplicate
Example
A user wants to remove duplicate records from the following dataset:
Choose a transformation
deduplicate


Preview of transform:


Result dataset:

The deduplicate transform only removed one row, because the deduplicate transform is case-sensitive.



keywords
deduplicate, duplicates, transform, delete_rows