Skip to content

Instantly share code, notes, and snippets.

@hospadar
Last active April 4, 2019 17:26
Show Gist options
  • Save hospadar/137911b51692ef57508d4df35ea022a3 to your computer and use it in GitHub Desktop.
Save hospadar/137911b51692ef57508d4df35ea022a3 to your computer and use it in GitHub Desktop.
s3cat implementation in python
#!/usr/bin/env python3
"""
Yet-another s3cat implementation.
streams files out of s3.
Requires boto3
> pip3 install boto3
Will use whatever your default credentials are locally (i.e. whatever you set up for awscli)
"""
import boto3, argparse, urllib.parse, sys
if __name__ == '__main__':
client = boto3.client('s3')
parser = argparse.ArgumentParser(usage="Print one or more s3 objects to the terminal")
parser.add_argument("url", help="s3 url(s) to cat. If ", nargs='+')
options = parser.parse_args()
for s3_url in options.url:
parsed_s3_url = urllib.parse.urlparse(s3_url)
if parsed_s3_url.scheme != 's3':
raise Exception("URL scheme must be s3")
if len(parsed_s3_url.path) <= 1:
raise Exception("s3 URL must have a path (the key of the s3 object to cat)")
bucket = parsed_s3_url.netloc
obj_key = parsed_s3_url.path[1:]
obj = client.get_object(Bucket=parsed_s3_url.netloc, Key=obj_key)
for chunk in obj['Body'].iter_chunks():
sys.stdout.buffer.write(chunk)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment