Really good paper on finding photos with duplicate content (https://cecs.uci.edu/~papers/icme06/pdfs/0000353.pdf). I have been working on a utility based on it that scans my photo library (~21,000 photos, some of which are duplicates due to faulty imports), and it has worked well even on my old dual-core laptop.
Apple Photos can also scan for duplicates, but there is no way to manually start the scan (it is supposed to run in the background but rarely does), so I had to find a better solution #imageeditor #imagemanagement