git clone https://github.com/nil0x42/duplicut.git
cd duplicut/ && make

duplicut [OPTION]... [INFILE] -o [OUTFILE]

Options:
-o, --outfile <FILE>       Write result to <FILE>
-t, --threads <NUM>        Max threads to use (default max)
-m, --memlimit <VALUE>     Limit max used memory (default max)
-l, --line-max-size <NUM>  Max line size (default 64)
-p, --printable            Filter ascii printable lines
-c, --lowercase            Convert wordlist to lowercase
-C, --uppercase            Convert wordlist to uppercase
-h, --help                 Display this help and exit
-v, --version              Output version information and exit

Example: duplicut wordlist.txt -o new-wordlist.txt

$ ./duplicut wordlist.txt -o clean-wordlist.txt -l 255

$ wc -l wordlist.txt clean-wordlist.txt
 33591536 wordlist.txt
 7487676 clean-wordlist.txt