Skip to content

Instantly share code, notes, and snippets.

@alrnz

alrnz/remove_duplicate_lines.sh

Last active May 29, 2019 12:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save alrnz/ab5ebef454daa88e38b3790c5d995b6f to your computer and use it in GitHub Desktop.
Save alrnz/ab5ebef454daa88e38b3790c5d995b6f to your computer and use it in GitHub Desktop.
Download ZIP
[remove duplicate lines] remove duplicate lines from files keeping the original order #Shell
Raw
remove_duplicate_lines.sh
# [remove duplicate lines] remove duplicate lines from files keeping the original order #Shell
# From: https://iridakos.com/how-to/2019/05/16/remove-duplicate-lines-preserving-order-linux.html
awk '!visited[$0]++' your_file > deduplicated_file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment