Skip to content

Instantly share code, notes, and snippets.

Created January 25, 2021 13:08
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
What would you like to do?
def scrape_job_details
web_page = browser.current_response
job_list = web_page.css('td#resultsCol')
job_list.css('div.jobsearch-SerpJobCard').each do |char_element|
title = char_element.css('h2 a')[0].attributes["title"].value.gsub(/\n/, "")
company = description = char_element.css('').text.gsub(/\n/, "")
salary = char_element.css('div.salarySnippet').text.gsub(/\n/, "")
job_details = [title, company, salary]
@@jobs << job_details if !@@jobs.include?(job_details)
In the end, call the scrape_job_details method inside the parse method and write the @@jobs array to a CSV file.
def parse(response, url:, data: {})
scrape_job_details'jobData.csv', "w") do |csv|
@@jobs.each { |element| csv.puts(element) }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment