Skip to content

Instantly share code, notes, and snippets.

@corintxt
Created June 3, 2021 15:51
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save corintxt/468070d1102d5354842d022e3d039d37 to your computer and use it in GitHub Desktop.
Save corintxt/468070d1102d5354842d022e3d039d37 to your computer and use it in GitHub Desktop.
q = text(
'''
SELECT *
FROM "facebook-timeline" ft
LIMIT 10000
'''
)
df = pd.DataFrame()
for idx, chunk in enumerate(
pd.read_sql_query(q, con=db.engine,
chunksize=1000
), start=1):
print (f"Retrieving chunk {idx}")
df = df.append(chunk)
print("Complete")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment