Skip to content

Instantly share code, notes, and snippets.

@brun0xff

brun0xff/read-data-from-storage-to-bq.py

Last active July 8, 2018 00:28
Show Gist options
  • Save brun0xff/3354c2acef9e430addaa2b9c2293ba3e to your computer and use it in GitHub Desktop.
Save brun0xff/3354c2acef9e430addaa2b9c2293ba3e to your computer and use it in GitHub Desktop.
Download ZIP
read data from google storage bucket to google cloud datalab --python3.x and jupyter-notebook
Raw
read-data-from-storage-to-bq.py
# How to read data from Google storage cloud to Google cloud datalab
import google.datalab.storage as storage
import pandas as pd
from io import BytesIO
mybucket = storage.Bucket('BUCKET_NAME')
data_csv = mybucket.object('data.csv')
uri = data_csv.uri
%gcs read --object $uri --variable data
df = pd.read_csv(BytesIO(data))
df.head()
@Gaurav1wani
Copy link

Gaurav1wani commented Jul 8, 2018 •
edited

I am facing following error:

UsageError: Line magic function %gcs not found.

Do you know what could be possible resolution ?

Thanks,
Gaurav

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment