Skip to content

Instantly share code, notes, and snippets.

@duncangh

duncangh/s3_to_pandas.py

Forked from jaklinger/s3_to_pandas.py
Created April 7, 2019 03:01
Show Gist options
  • Star 8 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
  • Save duncangh/19f20399899788882ecf94b51804740b to your computer and use it in GitHub Desktop.
Save duncangh/19f20399899788882ecf94b51804740b to your computer and use it in GitHub Desktop.
Download ZIP
Read CSV (or JSON etc) from AWS S3 to a Pandas dataframe
Raw
s3_to_pandas.py
import boto3
import pandas as pd
from io import BytesIO
bucket, filename = "bucket_name", "filename.csv"
s3 = boto3.resource('s3')
obj = s3.Object(bucket, filename)
with BytesIO(obj.get()['Body'].read()) as bio:
df = pd.read_csv(bio)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment