Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
Small Ruby script that demonstrates how to use Mechanize to scrape some product details from an array of product URLs from
require 'mechanize'
require 'csv'
puts "Product Scraper!!!"
puts ' '
urls = [
file = "product_data.csv"
header = "title,sku,image,alt_images", "w") do |csv|
csv << header
csv << "\n"
(0..urls.length - 1).each do |index|
puts urls[index]
agent =
page = agent.get(urls[index])
title = page.title
title = title[0..title.index(' - ')].rstrip
sku ="#sku").inner_text
sku = sku[4..sku.length-1]
prod_image ="#detailImage img").first
alt_images ="#productImages ul li a img")
brand_text ="#brandText").inner_text
alt_images = { |x| x[:src] }.join("|")
csv << [title, sku, prod_image[:src], "#{alt_images}"]
csv << "\n"
2.times { |x| puts "" }
puts "Done!"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.