Skip to content

Instantly share code, notes, and snippets.

@davidcomfort
Created December 13, 2015 22:09
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save davidcomfort/0a434f58f65646454ea5 to your computer and use it in GitHub Desktop.
Save davidcomfort/0a434f58f65646454ea5 to your computer and use it in GitHub Desktop.
def get_articles(query):
'''
This function accepts a year in string format (e.g.'1980')
and a query (e.g.'Amnesty International') and it will
return a list of parsed articles (in dictionaries)
for that year.
'''
all_articles = []
for i in range(0,100): #NYT limits pager to first 100 pages. But rarely will you find over 100 pages of results anyway.
articles = api.search(q = query,
fq = {'headline.search':[query], 'source':['The New York Times']},
begin_date = '20141031',
end_date = '20151031',
sort='newest',
page = str(i))
articles = parse_articles(articles)
all_articles = all_articles + articles
return(all_articles)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment