Even though the cost of storage per GB continues to fall, it’s common for users to need to find and remove duplicates files. Duplicates can stem from any file stored on your computer, be it images, software, music, video or what not. The process of finding and removing duplicates is time-consuming. Considering the volume of duplicate files, this is easier said than done. Fortunately, there are a number of tools that are designed to remove the laborious nature of finding duplicates by scanning and detecting your system for duplicate files in every format.
Duplicate files are a real annoyance. Aside from using disk space, these files do nothing but clutter up your drive, make your PC run slower, and increase the difficulty level significantly when it comes to finding the right version of a specific file: is it the one I just updated, or the other one?
Good duplicate finding software compares more than just a filename – including the file size – so it doesn’t remove files which are different to each other. You need reliable software. But always make sure you are making regular backups. And test the backups actually work.
This roundup focuses on command-line tools only. Here’s our verdict captured in a legendary LinuxLinks-style ratings chart. Only free and open source software is eligible for inclusion.
Let’s scrutinize the 14 tools at hand. For each utility we have compiled its own portal page, a full description with an in-depth analysis of its features, screenshots, together with links to relevant resources.
| Find and Delete Duplicate Files with these CLI Tools | |
|---|---|
| Czkawka | Find duplicate files, big files, empty files, similar images, and much more |
| fdupes | Great CLI tool that's written in C |
| fclones | Efficient duplicate file finder and remover |
| rmlint | Fast tool to remove duplicates and other lint |
| jdupes | Powerful CLI duplicate file finder and 'enhanced' fork of fdupes |
| smash | Find duplicate files super fast |
| rdfind | CLI redundant data find tool written in C++ |
| duff | Command-line utility for finding duplicate files |
| FSlint | Python based CLI and GUI tool |
| samanlainen | Delete duplicate files with SHA512 hashing |
| rmdupes | Option to use a reference directory |
| sdupes | Fast duplicate file detection utility. |
| dupefi | Duplicate file finder designed with Linux philosophy |
| ddh | Directory Differential hTool |
There are also three commands for finding duplicate files in littleutils.
GUI tools are covered in a separate roundup.
 Read our complete collection of recommended free and open source software. Our curated compilation covers all categories of software. Spotted a useful open source Linux program not covered on our site? Please let us know by completing this form. The software collection forms part of our series of informative articles for Linux enthusiasts. There are hundreds of in-depth reviews, open source alternatives to proprietary software from large corporations like Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. There are also fun things to try, hardware, free programming books and tutorials, and much more. |
There are also the repeats, repeats.pl, and repeats.py commands from littleutils.