Skip to content

Instantly share code, notes, and snippets.

@adnanmc
Last active May 9, 2016 02:35
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save adnanmc/51aa33108a99e2e792809b69e2e81279 to your computer and use it in GitHub Desktop.
Save adnanmc/51aa33108a99e2e792809b69e2e81279 to your computer and use it in GitHub Desktop.
web scraping with mechanize gem
require 'nokogiri'
require 'mechanize'
agent = Mechanize.new
# page = agent.get('http://www.meetup.com/hackerhours/members')
# name = page.search('.memName')[0].text
#puts name[0]
# puts name
# for i in 0..188
# i = i.to_s
# page = agent.get('http://www.meetup.com/hackerhours/members?offset=' + (i)
# # puts page.to_s
# count = page.search('.memName').count
# for j in 0..count-1
# puts page.search('.memName')[j].text
# end
# i = i.to_i
# i *= 20
# end
offset = 0
while offset < 3780
offset = offset.to_s
page = agent.get('http://www.meetup.com/hackerhours/members/?offset=' + offset)
count = page.search('.memName').count
offset = offset.to_i
for j in 0...count
puts page.search('.memName')[j].text
end
offset += 20
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment