Skip to content

Instantly share code, notes, and snippets.

@jmduke
Created March 7, 2018 16:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jmduke/45a6fc4e70d31d901dca5ac768b85781 to your computer and use it in GitHub Desktop.
Save jmduke/45a6fc4e70d31d901dca5ac768b85781 to your computer and use it in GitHub Desktop.
from typing import Iterable
from airtable import airtable
# Don't know what Airtable is? It's basically Excel for developers.
# It's super cool.
# Check it out: airtable.com
table = airtable.Airtable('API_KEY', 'OTHER_KEY')
# Okay, so we want to fetch all results in a table. Problem is, you know, pagination:
# this table has hundreds of records but we can only get like fifty per call.
# Here's how we abstract that out!
def fetch_all_records(table_name: str) -> Iterable:
# Grab the first page. The page has two fields we care about:
# 1. records — aka the good stuff
# 2. offset — a cursor to the next page.
response = table.get(table_name)
# If there's a cursor to the next page...
while 'offset' in response:
# Yield all the records on this page.
# Records look like this:
# {'fields': {'name': 'Salumi', 'city': 'Seattle', 'rating': 5}, 'id': 'as890ops'}
for record in response['records']:
yield record
# Grab the next page and repeat the process.
response = table.get('Finished', offset=response['offset'])
# Otherwise, yield all the records and then we're done!
for record in response['records']:
yield record
# Now, to access all the records, we don't have to care about:
# 1. How many records are in a page
# 2. How to get to the next page
# 3. How to traverse the page.
# All of that is abstracted away by the generator!
# So we can just iterate through all of them like this:
for record in fetch_all_records('Sandwiches'):
print(record['fields']['Name'])
# And generators are lazy, too, so if we just want the first twenty items
# we can do so without worrying about premature pagination:
for i, record in enumerate(fetch_all_records('Sandwiches')):
if i > 20:
break
print(record['fields']['Name'])
# Generators aren't great for everything.
# For instance, operating on an entire corpus of an iterable is rough:
# this code will force you to refetch the entire list over and over again.
best_rating = max([record['fields']['Rating'] for record in fetch_all_records('Sandwiches')])
worst_rating = min([record['fields']['Rating'] for record in fetch_all_records('Sandwiches')])
# In such a case, you're better off casting the generator to a list.
# (But you might be *best* off with a different approach entirely.)
all_records = list(fetch_all_records('Sandwiches'))
best_rating = max([record['fields']['Rating'] for record in all_records])
worst_rating = min([record['fields']['Rating'] for record in all_records])
# Ultimately, generators are the best kind of Python feature:
# 1. They make it easier to understand code.
# 2. They make it easier to write code.
# 3. They're just *neat*.
for record in fetch_all_records('Sandwiches'):
# Gotta find a great sandwich in Seattle!
if record['fields']['Rating'] > 4 and record['field']['city'] == 'Seattle':
print(record['fields']['Name'])
break
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment