Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Shell: Analyze huge files for repeating text portions
# Sorts the file by duplicate line count
sort /path/to/filename | uniq -c | sort -nr > ./_aggregated.tmp
# Just read the head as it's probably a huge file
head -n 1000 ./_aggregated.tmp | less
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment