Created
January 17, 2011 03:13
-
-
Save rsvp/782451 to your computer and use it in GitHub Desktop.
uniqnosort : uniq replacement -- shell script to get unique lines WITHOUT prior sort -- preserves original ordering where any subsequent duplicates are removed. Duplicates do NOT have to be adjacent to each other!
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env bash | |
# bash 4.1.5(1) Linux Ubuntu 10.04 Date : 2011-01-16 | |
# | |
# _______________| uniqnosort : get unique lines WITHOUT sorting. | |
# | |
# Usage: uniqnosort [file, or use as pipe] | |
# | |
# NB: More famous "uniq" requires duplicates on adjacent lines. | |
# If you really want to sort and get uniques, | |
# try "sort -u" rather than "sort | uniq". | |
# | |
# To reverse the line ordering, see my "tailflip". | |
# | |
# Dependencies: awk | |
# CHANGE LOG | |
# 2011-01-16 First version: amazing one-liner. | |
# _______________ :: BEGIN Script :::::::::::::::::::::::::::::::::::::::: | |
inf=${1:-'-'} | |
# ^stdin is default so that we can pipe in stuff. | |
awk '!x[$0]++' $inf | |
exit 0 | |
# _______________ EOS :: END of Script :::::::::::::::::::::::::::::::::::::::: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment