Write a Pandas dataframe to CSV format on AWS S3.

_write_dataframe_to_csv_on_s3.py

import boto3
from io import StringIO

def _write_dataframe_to_csv_on_s3(dataframe, filename):
    """ Write a dataframe to a CSV on S3 """
    print("Writing {} records to {}".format(len(dataframe), filename))
     # Create buffer
    csv_buffer = StringIO()
    # Write dataframe to buffer
    dataframe.to_csv(csv_buffer, sep="|", index=False)
    # Create S3 object
    s3_resource = boto3.resource("s3")
    # Write buffer to S3 object
    s3_resource.Object(DESTINATION, filename).put(Body=csv_buffer.getvalue())

Great thanks! But do you know how you can successfully add UTF-8-sig encoding? Otherwise using the above I get words like this: 'R√©union' instead of Rèunion when I download my csv from s3 bucket

Great thanks! But do you know how you can successfully add UTF-8-sig encoding? Otherwise using the above I get words like this: 'R√©union' instead of Rèunion when I download my csv from s3 bucket

I assume you can use the encoding parameter of Pandas to_csv. Are you able to add encoding="utf-8" to the dataframe.to_csv() step?