The flow view of this templateTransformations:
aggregate functions (count), rownumber
Designer Cloud has a deduplicate transformation that allows you to remove rows where the values are identical across all columns. However, what if you want to remove rows where the data is duplicated in only certain columns? This simple template shows you how to find and remove rows when there are duplicate values in some of the columns, but not all columns. To customize the template for your own use, simply update the aggregate group by parameter to include all the columns that you want to check for duplicates