Skip to content

Instantly share code, notes, and snippets.

@rajivnarayan
rajivnarayan / compute_etag.sh
Last active August 15, 2024 23:53 — forked from emersonf/s3etag.sh
Calculate checksum corresponding to the entity-tag hash (ETag) of Amazon S3 objects
#!/bin/bash
#
# Calculate checksum corresponding to the entity-tag hash (ETag) of Amazon S3 objects
#
# Usage: compute_etag.sh <filename> <part_size_mb>
#
# filename: file to process
# part_size_mb: chunk size in MiB used for multipart uploads.
# This is 8M by default for the AWS CLI See:
# https://docs.aws.amazon.com/cli/latest/topic/s3-config.html#multipart_chunksize
@rajivnarayan
rajivnarayan / s3path.py
Last active July 20, 2024 19:04
A Pydantic model for validating S3 URLs
import re
from pydantic import BaseModel, Field, computed_field
class S3Path(BaseModel):
"""Pydantic Model for S3 URLs"""
# https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html
_s3_pattern = re.compile(
r'^s3://'
r'(?=[a-z0-9])' # Bucket name must start with a letter or digit