Skip to content

Instantly share code, notes, and snippets.

@maliubiao
Created September 14, 2013 02:50
Show Gist options
  • Save maliubiao/6558441 to your computer and use it in GitHub Desktop.
Save maliubiao/6558441 to your computer and use it in GitHub Desktop.
remove dumplicates
#! /usr/bin/env python
if __name__ == "__main__":
import os
import sys
import os.path
path = sys.argv[1]
if not os.path.exists(path):
print("Error: file doesn't exists")
exit(0)
f1 = open(path, "r")
f2 = open("%s.new" % path, "w")
for line in set(f1.readlines()):
f2.write(line)
f2.close()
f1.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment