Skip to content

Instantly share code, notes, and snippets.

@cantino

cantino/curl_upload.sh Secret

Last active Mar 19, 2019
Embed
What would you like to do?
Example file upload flow for Major Tom
#!/bin/sh -e
# This is an example script using curl to show how to upload files to Major Tom.
command -v jq >/dev/null 2>&1 || { echo >&2 "I require jq but it's not installed. Aborting."; exit 1; }
command -v openssl >/dev/null 2>&1 || { echo >&2 "I require openssl but it's not installed. Aborting."; exit 1; }
command -v base64 >/dev/null 2>&1 || { echo >&2 "I require base64 but it's not installed. Aborting."; exit 1; }
command -v curl >/dev/null 2>&1 || { echo >&2 "I require curl but it's not installed. Aborting."; exit 1; }
if [[ "$#" -lt 3 ]]; then
echo "Usage: $0 file system gateway_token host"
exit 1
fi
filename="${1##*/}"
size=$(wc -c < "$1")
checksum=$(openssl md5 -binary "$1" | base64)
system=$2
gateway_token=$3
host=${4:-"http://localhost:3000"}
timestamp=$(date -r "$1" +%s000)
echo "Host: $host"
echo "System: $system"
echo "Filename: $filename"
echo "Byte size: $size"
echo "Checksum: $checksum"
echo "Timestamp: $timestamp"
# Tell Major Tom that you want to upload a file, including the file's name, size, and checksum.
# See https://gist.github.com/cantino/06e3641b629036eceb6533afe7f8e20d for various ways to compute the
# checksum.
json=$(
curl -s \
"${host}/rails/active_storage/direct_uploads" \
-H "X-Gateway-Token: ${gateway_token}" \
-H "Content-Type: application/json" \
--data-binary "{\"filename\":\"${filename}\",\"byte_size\":${size},\"content_type\":\"binary/octet-stream\",\"checksum\":\"${checksum}\"}"
)
# Major Tom responds with a direct upload url that you can use to upload the file. Major Tom also
# provides a signed_id that you will use to refer to this file in the confirmation step after uploading it.
# Display details of the upload request.
echo $json | jq '.'
url=$(echo $json | jq -r '.direct_upload.url')
signed_id=$(echo $json | jq -r '.signed_id')
echo "Upload URL: $url"
echo "Signed ID: $signed_id"
# Upload file to storage using a direct PUT request. For OnPrem installations, this is minio. For cloud
# installations, this is AWS S3.
curl -s \
-H 'Content-Type: binary/octet-stream' \
-H "Content-MD5: $checksum" \
--upload-file "${1}" \
"${url}"
# Tell Major Tom that we've uploaded the file by providing the signed_id from the request to /direct_uploads.
# Also send the system name of the file's source, the file's timestamp, a command_id (optional),
# the filename, and optional metadata.
downlinked_file_json=$(
curl -s \
"${host}/gateway_api/v1.0/downlinked_files" \
-H "X-Gateway-Token: $gateway_token" \
-H 'Content-Type: application/json' \
--data-binary "{\"system\":\"$system\",\"signed_id\":\"$signed_id\",\"timestamp\":\"$timestamp\",\"command_id\":null,\"name\":\"$filename\",\"metadata\":{\"key\":\"value\"}}"
)
# Display the response from Major Tom. The "id" provided can be used to further update the downlinked file record,
# if needed.
echo $downlinked_file_json | jq '.'
# Re-download the file to prove that the upload worked.
# -L makes curl follow redirects
final_file_checksum=$(
curl -s \
"${host}/rails/active_storage/blobs/${signed_id}/foo.png" \
-L \
-H "X-Gateway-Token: $gateway_token" | openssl md5 -binary | base64
)
echo "Checksum: $final_file_checksum"
if [[ "$final_file_checksum" == "$checksum" ]]; then
echo "They match!"
else
echo "They don't match. :("
fi
@cantino

This comment has been minimized.

Copy link
Owner Author

@cantino cantino commented Mar 2, 2019

Example usage:

./curl_upload.sh ~/Desktop/Screen\ Shot\ 2019-03-01\ at\ 3.31.20\ PM.png Satellite0 0077a9141cb5b...501cf84c20d http://localhost:3000
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment