There are various software tools available for finding and removing duplicates. They recognise duplicates using identical checksums. Options include: