Skip to content

Instantly share code, notes, and snippets.

@jftuga
Created May 9, 2024 14:41
Show Gist options
  • Save jftuga/82e775e35f51234db6746916b6e00f09 to your computer and use it in GitHub Desktop.
Save jftuga/82e775e35f51234db6746916b6e00f09 to your computer and use it in GitHub Desktop.
Batch create S3 objects
#!/bin/bash
# create a number of objects in the given S3 bucket
set -euo pipefail
COUNT=25
START=0
SOURCE=object
DEST="s3://test-bucket-${RANDOM}-${RANDOM}-${RANDOM}"
echo
echo "Creating bucket: ${DEST}"
echo
aws --profile ${AWS_PROFILE} s3 mb "${DEST}"
touch "${SOURCE}"
i="${START}"
let j="${START}+${COUNT}"
while [[ "$i" -lt "$j" ]] ; do
PADDED=$(printf "%04d" "${i}")
NAME="${DEST}/${SOURCE}-${PADDED}"
aws --profile "${AWS_PROFILE}" s3 cp "${SOURCE}" "${NAME}" &
if ! (("${i}" % 10)) ; then
echo
echo "waiting at ${i}"
echo
wait
fi
let i=$i+1
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment