Skip to content

Instantly share code, notes, and snippets.

@stefanbenten
Last active July 23, 2019 21:45
Show Gist options
  • Save stefanbenten/926818f0419e8e88d99398f416ffe965 to your computer and use it in GitHub Desktop.
Save stefanbenten/926818f0419e8e88d99398f416ffe965 to your computer and use it in GitHub Desktop.
Reupload Script to V2 network with two accounts
#!/bin/bash
# USAGE: ./reupload.sh <BUCKET-ID> <NEW-BUCKET-ID>
# Bucket-ID's are optional, if not provided the entire account is reuploaded
# Account Email Addresses
username1=test@storj.io
username2=test2@storj.io
# Account Passwords
password1=passw0rd
password2=passwOrd
#Encryption Keys are the 12-24 word strings
enckey1=SafeFile1
enckey2=SafeFile2
storage="./files-twoaccounts"
bpath=$(pwd)
file="./bucket.list"
reuploadbucket() {
#Go into directory
mkdir $storage/$bucket
pushd $storage/$bucket
#Get all files in bucket
STORJ_BRIDGE_USER=$username1 STORJ_BRIDGE_PASS=$password1 STORJ_ENCRYPTION_KEY=$enckey1 $bpath/storj list-files $bucket > $file
#Make new bucket when reuploading all buckets
if [ "$name" != "" ]
then
bucketcreat="$(STORJ_BRIDGE_USER=$username2 STORJ_BRIDGE_PASS=$password2 STORJ_ENCRYPTION_KEY=$enckey2 $bpath/storj add-bucket $name)"
read a1 newbucketID a3 a4 a5 a6 <<< $bucketcreat
echo "Created new bucket $a6 with ID: $newbucketID"
fi
#Reupload files, when existing
while read line; do
read field1 ID field3 SIZE field4 field5 DECRYPTED field6 TYPE field7 CREATEDAT field8 NAME <<< $line
if [ "$NAME" != "" ]
then
#Download File
echo $ID $NAME
STORJ_BRIDGE_USER=$username1 STORJ_BRIDGE_PASS=$password1 STORJ_ENCRYPTION_KEY=$enckey1 $bpath/storj download-file $bucket $ID $NAME
#Delete File
STORJ_BRIDGE_USER=$username1 STORJ_BRIDGE_PASS=$password1 STORJ_ENCRYPTION_KEY=$enckey1 $bpath/storj remove-file $bucket $ID
#Upload File
echo "Uploading File $NAME"
val="$(STORJ_BRIDGE_USER=$username2 STORJ_BRIDGE_PASS=$password2 STORJ_ENCRYPTION_KEY=$enckey2 $bpath/storj upload-file $newbucketID $NAME)"
val2=$(echo "$val" | grep "File ID: ")
read f1 f2 f3 f4 NEWID <<< $val2
if [ -z "$NEWID" ]
then
echo "File Upload failed.."
else
echo "File Upload successful!"
#Log Information
echo "Bucket: $bucket ID: $ID Name: $NAME Size: $SIZE -> Bucket: $newbucketID ID: $NEWID" >> $bpath/process.log
fi
else
echo "could not find file"
fi
done < $file
popd
}
if [ "$1" == "" ]
then
#Get all buckets and fetch their IDs
buckets=$(STORJ_BRIDGE_USER=$username1 STORJ_BRIDGE_PASS=$password1 STORJ_ENCRYPTION_KEY=$enckey1 ./storj list-buckets)
while read -r line; do
read f1 BID f3 readable f5 f6 f7 name <<< $line
if [ "$readable" == "true" ]
then
echo "Starting reupload of bucket $name with ID: $BID"
bucket=$BID
reuploadbucket
else
echo "Skipping bucket $BID, due to wrong encryption key"
fi
done < <(printf '%s\n' "$buckets")
else
bucket=$1
newbucketID=$2
reuploadbucket
fi
#Cleanup
rm -f $file
@stefanbenten
Copy link
Author

stefanbenten commented Jul 22, 2019

Setup:

  • Set Credentials for both accounts from L5-L13
  • Specify the old bucket on the old account and new bucket on the new account, to only reupload that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment