Skip to content

Instantly share code, notes, and snippets.

@naveensrinivasan
Last active March 17, 2016 10:35
Show Gist options
  • Save naveensrinivasan/5962839 to your computer and use it in GitHub Desktop.
Save naveensrinivasan/5962839 to your computer and use it in GitHub Desktop.
An easy way to screen scrape GoodReads recommendation's because there isn't an API.
# An easy way to get the recommendations from goodreads because there isn't an API to get the information
require 'mechanize'
require 'yaml'
require 'goodreads'
keys = YAML.load_file ENV['HOME']+'/.goodreads'
client = Goodreads::Client.new(:api_key => keys["developer_key"], :api_secret => keys["developer_secret"])
agent = Mechanize.new
page = agent.get 'http://www.goodreads.com/recommendations/'
form = page.form('sign_in')
form['user[email]'] = keys["user_name"]
form['user[password]'] = keys["password"]
agent.submit(form)
temp_jar = agent.cookie_jar
@agent = Mechanize.new
@agent.cookie_jar = temp_jar
page = @agent.get 'http://www.goodreads.com/recommendations/shelf/children-books?recs_current_view=list'
page.links_with(:dom_class => 'readable').each do |item|
book_id = item.href.to_s.split('/').last.split(/[.-]/).first
data = client.book book_id
book_info = data.title + ',' + data.isbn
pp book_info
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment