Last active
October 20, 2016 14:01
-
-
Save hxmuller/11e986dedea48d3b0149bbe38be97faf to your computer and use it in GitHub Desktop.
delete duplicate lines while keeping order
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# awk - pattern scanning and text processing language | |
# a) When the program is omitted, the default is { print $0 } | |
# b) Each line of filename is read by awk, one at a time. | |
# c) The line, or record, is stored into the field variable $0. | |
# d) If an array index consisting of the line is not (!) present | |
# in the array, then the line is printed. | |
# e) The array is incremented (current index added to the array). | |
awk '!array[$0]++' filename > filename2 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment