Skip to content

Instantly share code, notes, and snippets.

@agalea91

agalea91/bigquery_csv_upload.py

Created July 5, 2022 14:09
Show Gist options
  • Save agalea91/3887db2e9fc103f47480c8e6d1afe1ba to your computer and use it in GitHub Desktop.
Save agalea91/3887db2e9fc103f47480c8e6d1afe1ba to your computer and use it in GitHub Desktop.
Download ZIP
Raw
bigquery_csv_upload.py
creds = "/path/to/creds.json"
df = pd.read_csv("file.csv")
# check df's dtypes and column names
df.to_parquet("gs://bucket/file.parquet", storage_options=dict(token=creds))
# then upload through console. auto-detects schema
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment