Skip to content

Instantly share code, notes, and snippets.

@Lackoftactics
Created August 5, 2013 14:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Lackoftactics/6156160 to your computer and use it in GitHub Desktop.
Save Lackoftactics/6156160 to your computer and use it in GitHub Desktop.
Easy for Kuba
#encoding: utf-8
def fetch_job_links
job_links = []
if next_page_selector
index_doc = Nokogiri::HTML(open(url))
base_url = parse_base_url(url, index_doc)
pagination = index_doc.at_xpath(next_page_selector)
next_page_link = URI.join(base_url, pagination['href']).to_s rescue nil
##przed tą pętlą ustawia mi się next_page_link na nila, wiesz może dlaczego?
until next_page_link == nil
current_url ||= url
job_links << fetch_page_links(current_url)
index_doc = Nokogiri::HTML(open(url))
pagination = index_doc.at_xpath(next_page_selector)
next_page_link = URI.join(base_url, pagination['href']).to_s rescue nil
current_url = next_page_link
end
else
job_links << fetch_page_links(url)
end
job_links.flatten!
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment