Skip to content

Instantly share code, notes, and snippets.

@pythonesque
Created September 21, 2013 00:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pythonesque/6645817 to your computer and use it in GitHub Desktop.
Save pythonesque/6645817 to your computer and use it in GitHub Desktop.
#!/usr/bin/env ruby
require 'uri'
require 'set'
urls = Set.new()
STDIN.each_line do|line|
begin
url = URI.parse(line.downcase).host
unless urls.member?(url)
[url, "www.#{url}"].each do |url|
urls.add(url)
STDOUT.puts (url)
end
end
rescue URI::InvalidURIError
STDERR.puts "Invalid URI: #{line}"
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment