Wordlist Cleanup commands helpful while bruteforcing and password cracking.

wordlist_cleanup.md

covert a entire wordlist to all lowercase with no garbage.

$ cat dirtyfile.txt | awk '{gsub(/[[:punct:]]/,"")}1' | tr A-Z a-z | sed 's/[0-9]*//g' | sed -e 's/ //g' | strings | tr -cs '[:alpha:]' '\ ' | sed -e 's/ /\n/g' | tr A-Z a-z | sort -u > cleanfile.txt

Remove Duplicates

awk '!(count[$0]++)' old.txt > new.txt

Sort Wordlist by Length

awk '{print length, $0}' old.txt | sort -n | cut -d " " -f2- > new.txt

Sort Alphabetical order

sort old.txt | uniq > new.txt

Merge multiple text files into one

cat file1.txt file2.txt > combines.txt

Remove all blank lines

egrep -v "^[[:space:]]*$" old.txt > new.txt