Skip to content

Instantly share code, notes, and snippets.

@domingos86

domingos86/apple_POST.py Secret

Last active February 20, 2017 01:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save domingos86/6eb386d2b9112ef1a2aff09d1f649a0b to your computer and use it in GitHub Desktop.
Save domingos86/6eb386d2b9112ef1a2aff09d1f649a0b to your computer and use it in GitHub Desktop.
Download ZIP
Example of scrapy POST request for the Apple spider
Raw
apple_POST.py
def get_list_form_data(page_number, query):
return {'searchRequestJson': '{"searchString":"' + query + '","jobType":"1",' +\
'"sortOrder":"","filters":{"locations":{"location":[{"type":"0","code":"USA"}]}},' +\
'"pageNumber":"' + str(page_number) + '"}',
'csrfToken': 'null', 'clientOffset': '-300'}
query_terms = ["data scientist", "data engineer", "business analyst"]
def start_requests(self):
return [FormRequest('https://jobs.apple.com/us/search/search-result',
formdata = get_list_form_data(0, query_term), callback = self.parse_list,
meta = {'page_number': 0, 'query_term': query_term})
for query_term in self.query_terms]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment