Skip to content

Instantly share code, notes, and snippets.

@keithrozario
Last active August 28, 2023 10:43
Show Gist options
  • Save keithrozario/7103036520b153cdeb41e269a4a88c50 to your computer and use it in GitHub Desktop.
Save keithrozario/7103036520b153cdeb41e269a4a88c50 to your computer and use it in GitHub Desktop.
Read file from s3.download_fileobj into memory
s3 = boto3.client('s3')
## Reading a JSON file in memory
with tempfile.TemporaryFile() as data:
s3.download_fileobj(bucket_name, 'tests/s3/key.json', data)
data.seek(0)
status = json.loads(data.read().decode('utf-8'))['status']
## Reading a CSV file in memory
with tempfile.TemporaryFile() as package_file:
s3_client.download_fileobj(bucket_name, 'config/packages.csv', package_file)
package_file.seek(0)
file_as_string = package_file.read().decode('utf-8')
csv_reader = csv.DictReader(io.StringIO(file_as_string))
packages = [line['Package_Name'] for line in csv_reader]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment