Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
dump a df to s3 via s3fs
# ...
# your data preprocessing...
if __name__ == "__main__":
# ...
# your data preprocessing...
s3 = s3fs.S3FileSystem(anon=False)
df = pd.DataFrame(data={"foo":[0]})
print(df)
with s3.open(f"{BUCKET_NAME}/data_bow_limit40000.csv",'w') as f:
df.to_csv(f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment