Skip to content

Instantly share code, notes, and snippets.

@gabrielecirulli
Created May 20, 2012 12:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gabrielecirulli/49f02ae82740f421406c to your computer and use it in GitHub Desktop.
Save gabrielecirulli/49f02ae82740f421406c to your computer and use it in GitHub Desktop.
fp more pronporn scrapper
#!/usr/bin/env ruby
require 'nokogiri'
require 'open-uri'
require 'net/http'
require 'uri'
require 'fileutils'
require 'colorize'
def file_name path
path.split( File::SEPARATOR ).last
end
# Make results dir
dir = File.join Dir.pwd, "results"
FileUtils.mkdir_p dir
# Request URL
puts "Page URL:"
page = Nokogiri::HTML open gets.chomp
links = page.css( "img" ).map { |img| img['src'] }
hosts = links.map { |x| URI x }.group_by &:hostname
hosts.each do |hostname, uris|
http = Net::HTTP.start hostname
uris.each do |uri|
resp = http.get uri.path
puts "Downloading '#{file_name uri.path}'"
name = file_name uri.path
file_path = File.join dir, name
open file_path, "wb" do |file|
file.write resp.body
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment