Skip to content

Instantly share code, notes, and snippets.

@sebastianschramm
Created August 20, 2022 10:12
Show Gist options
  • Save sebastianschramm/aa39178cd95eee90a38e18287cc0513a to your computer and use it in GitHub Desktop.
Save sebastianschramm/aa39178cd95eee90a38e18287cc0513a to your computer and use it in GitHub Desktop.
How to handle S3 paths using cloudpathlib
from pathlib import Path
from cloudpathlib import CloudPath
s3_file = "s3://mybucket/foo/bar.txt"
cloud_path = CloudPath(s3_file)
conventional_path = Path(s3_file)
"""
# pathlib.Path does not handle S3 paths
# -> note that the parsed path starts with 's3:/' instead of 's3://'
>>> conventional_path
PosixPath('s3:/mybucket/foo/bar.txt')
# cloudpathlib.CloudPath can handle S3 paths and has the same interface
>>> cloud_path
S3Path('s3://mybucket/foo/bar.txt')
>>> cloud_path.parts
('s3://', 'mybucket', 'foo', 'bar.txt')
"""
@sebastianschramm
Copy link
Author

dependencies: python3.9, cloudpathlib==0.10.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment