Skip to content

Instantly share code, notes, and snippets.

@maliabadi
Created January 9, 2014 02:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save maliabadi/8328468 to your computer and use it in GitHub Desktop.
Save maliabadi/8328468 to your computer and use it in GitHub Desktop.
Want a bunch of harmless looking desktop backgrounds?
require 'rubygems'
require 'mechanize'
class DesktopScraper
BASE = "http://simpledesktops.com"
def destination_folder
File.dirname(__FILE__) + "/desktops"
end
def initialize
@posts = []
end
def agent
@agent ||= Mechanize.new
end
def get url
agent.get url
end
def doc
Nokogiri::HTML.parse(agent.page.body)
end
def run
# get the profile urls...
(1..36).to_a.each do |page|
get("#{BASE}/browse/#{page}")
links = doc.css('.desktops .edge .desktop a').to_a.map do |node|
node.attribute('href').value
end.select do |href|
href =~ /browse/
end.map do |href|
BASE + href
end.uniq
@posts += links
@posts.uniq!
end
# go to each profile url
@posts.each_with_index do |url,index|
get url
get (BASE + doc.at_css('.desktop a').attribute('href').value)
agent.page.save "#{destination_folder}/#{agent.page.filename}"
end
end
end
DesktopScraper.new().run
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment