Suppose you are using AWS Datapipeline, it will make use of PK to get rid of duplicates.