Skip to content

Instantly share code, notes, and snippets.

@RickCogley RickCogley/1. Readme.md
Last active Nov 29, 2018

Embed
What would you like to do?
Transloadit Sample for Tim 20181129

Using Transloadit's "notify_url" to Manipulate Assembly Results JSON

The Transloadit Assembly results JSON has all kinds of information about the processed files, but we need this information to appear in our business database. We need to use some practical scripting to massage the results JSON, and get it loaded into our db how we like.

Use "notify_url" to send to a webhook, which runs a script

Transloadit template can be set to send its json results payload to a notify_url which could be a url you set up at a web service like Hookdoo or others.

"notify_url": "https://x.hookdoo.com/hook/12345abc-ab12-cd34-ef56-1234567890ab?whatever=12345abcde"

Hookdoo's webhook can run a shell script like transloadit-hook.sh below, on one of your servers. The payload can be parsed there, and some action can be taken. In our case, this script:

  • is triggered only if a rule set in the Hookdoo webhook is true, which is determined by the url param ?whatever=12345abcde. This can be anything you like.
  • gets a couple of variables sent to it by Hookdoo, but these could also be set up as environment variables on the target server.
  • sets up a bunch of environment variables for safety's sake, because depending upon how the script is run by Hookdoo on our server, it's possible that some of those binaries are inaccessible.
  • massages the Transloadit payload by using php to decode, splitting into signature and transloadit parts with awk, getting a local hmac signature via openssl, for later comparison to the received signature. Continues on, using jq to extract some parameters like template_id from the payload json, and also to re-arrange the json into something our database will accept.
  • next it uses cmp in an if statement, to check if the locally-calculated signature matches the one that came with the payload. If it does, then it loads the massaged json (the tassy-db-ready.json) up to the db.

In the end, users can drag some images to Uppy, which in turn uses Transloadit to quickly process the files, send the payload to a webhook, which runs a script, checks the signature, and uploads to a database within a few seconds.

Now our users can easily reference and search what they uploaded.

#!/bin/sh
# Run by hookdoo on your server, and INPUT var is created by the script in the hook
# Use ${DBRESTTOKEN} in database load statement
# Use ${TAPIAUTH} in openssl hmac statement for signature check
JQBIN="/home/rcogley/bin/jq"
RMBIN="/usr/bin/rm"
AWKBIN="/usr/bin/awk"
PHPBIN="/usr/local/bin/php"
PERLBIN="/usr/bin/perl"
OPENSSLBIN="/usr/bin/openssl"
CMPBIN="/usr/bin/cmp"
WORKINGDIR="/path/to/my/vhost/hook_transloadit_01"
cd ${WORKINGDIR}
${RMBIN} -rf *
echo ${INPUT} > ${WORKINGDIR}/tassy-payload.out
cat ${WORKINGDIR}/tassy-payload.out | ${PHPBIN} -R 'echo urldecode($argn)."\n";' > ${WORKINGDIR}/tassy-payload-decoded.out
cat ${WORKINGDIR}/tassy-payload-decoded.out | ${AWKBIN} -F'&signature=' '{print $2}' > ${WORKINGDIR}/tassy-signature.out
cat ${WORKINGDIR}/tassy-payload-decoded.out | ${AWKBIN} -F'&signature=' '{print $1}' > ${WORKINGDIR}/tassy-payload-decoded-pre1.out
cat ${WORKINGDIR}/tassy-payload-decoded-pre1.out | ${AWKBIN} -F'transloadit=' '{print $2}' > ${WORKINGDIR}/tassy-result-pre1.out
${PERLBIN} -pe 'chomp if eof' ${WORKINGDIR}/tassy-result-pre1.out | ${OPENSSLBIN} sha1 -hmac "${TAPIAUTH}" | ${AWKBIN} '{print $2}' > ${WORKINGDIR}/tassy-signature-local-confirm.out
cat ${WORKINGDIR}/tassy-result-pre1.out | ${JQBIN} '.' > ${WORKINGDIR}/tassy-result.json
TEMPLATEID=$(cat tassy-result.json | ${JQBIN} --compact-output --raw-output '.template_id')
ASSEMBLYID=$(cat tassy-result.json | ${JQBIN} --compact-output --raw-output '.assembly_id')
ASSEMBLYTS=$(cat tassy-result.json | ${JQBIN} --compact-output --raw-output '.last_job_completed|gsub(" GMT"; "Z")|gsub(" "; "T")|gsub("/"; "-")')
cat ${WORKINGDIR}/tassy-result.json | ${JQBIN} --raw-output --arg tid "${TEMPLATEID}" --arg aid "${ASSEMBLYID}" --arg ats "${ASSEMBLYTS}" '[.uploads,.results.compress_image | .[] | {"Template Id": $tid, "Assembly Id": $aid, "Assembly TS": $ats, "Original Id": .original_id, "File Size": .size, "File Width": .meta.width, "File Last Modified": (.meta.date_file_modified|gsub(" GMT"; "Z")|gsub(" "; "T")|gsub("/"; "-")), "File URL": .ssl_url}]' > ${WORKINGDIR}/tassy-db-ready.json
# Compare sigs and if ok then POST it to our db
if ${CMPBIN} ${WORKINGDIR}/tassy-signature.out ${WORKINGDIR}/tassy-signature-local-confirm.out >/dev/null 2>&1
then
curl -X "POST" "https://businessdb.net/api/v2/${DBRESTTOKEN}/TheLinkTable/create.json" \
-H 'Content-Type: application/json' \
-d @${WORKINGDIR}/tassy-db-ready.json
fi
[
{
"Template Id": "37b12345eb9a1145672158c785abcde",
"Assembly Id": "a2bb1234567811e882408fd8d6a768e5",
"Assembly TS": "2018-11-27T01:49:42Z",
"Original Id": "a4f82ca1f1e611e882408fd8d6a768e5",
"File Size": 119500,
"File Width": 1880,
"File Last Modified": "2018-11-27T01:49:15Z",
"File URL": "https://our-bucket.s3.amazonaws.com/prdn/2018/eSolia-JRC-Screenshot-2018-10-31-at-17.52.40-a4f82ca1f1e611e882408fd8d6a768e5-w1880px.png"
},
{
"Template Id": "37b12345eb9a1145672158c785abcde",
"Assembly Id": "a2bb1234567811e882408fd8d6a768e5",
"Assembly TS": "2018-11-27T01:49:42Z",
"Original Id": "a561eaf1f1e611e882408fd8d6a768e5",
"File Size": 484018,
"File Width": 2026,
"File Last Modified": "2018-11-27T01:49:20Z",
"File URL": "https://our-bucket.s3.amazonaws.com/prdn/2018/eSolia-JRC-Screenshot-2018-09-09-at-8.02.28-a561eaf1f1e611e882408fd8d6a768e5-w2026px.png"
},
{
"Template Id": "37b12345eb9a1145672158c785abcde",
"Assembly Id": "a2bb1234567811e882408fd8d6a768e5",
"Assembly TS": "2018-11-27T01:49:42Z",
"Original Id": "a4f82ca1f1e611e882408fd8d6a768e5",
"File Size": 11110,
"File Width": 150,
"File Last Modified": "2018-11-27T01:49:28Z",
"File URL": "https://our-bucket.s3.amazonaws.com/prdn/2018/eSolia-JRC-Screenshot-2018-10-31-at-17.52.40-a4f82ca1f1e611e882408fd8d6a768e5-w150px.png"
},
{
"Template Id": "37b12345eb9a1145672158c785abcde",
"Assembly Id": "a2bb1234567811e882408fd8d6a768e5",
"Assembly TS": "2018-11-27T01:49:42Z",
"Original Id": "a4f82ca1f1e611e882408fd8d6a768e5",
"File Size": 61577,
"File Width": 800,
"File Last Modified": "2018-11-27T01:49:27Z",
"File URL": "https://our-bucket.s3.amazonaws.com/prdn/2018/eSolia-JRC-Screenshot-2018-10-31-at-17.52.40-a4f82ca1f1e611e882408fd8d6a768e5-w800px.png"
},
{
"Template Id": "37b12345eb9a1145672158c785abcde",
"Assembly Id": "a2bb1234567811e882408fd8d6a768e5",
"Assembly TS": "2018-11-27T01:49:42Z",
"Original Id": "a561eaf1f1e611e882408fd8d6a768e5",
"File Size": 16928,
"File Width": 150,
"File Last Modified": "2018-11-27T01:49:32Z",
"File URL": "https://our-bucket.s3.amazonaws.com/prdn/2018/eSolia-JRC-Screenshot-2018-09-09-at-8.02.28-a561eaf1f1e611e882408fd8d6a768e5-w150px.png"
},
{
"Template Id": "37b12345eb9a1145672158c785abcde",
"Assembly Id": "a2bb1234567811e882408fd8d6a768e5",
"Assembly TS": "2018-11-27T01:49:42Z",
"Original Id": "a561eaf1f1e611e882408fd8d6a768e5",
"File Size": 179676,
"File Width": 800,
"File Last Modified": "2018-11-27T01:49:34Z",
"File URL": "https://our-bucket.s3.amazonaws.com/prdn/2018/eSolia-JRC-Screenshot-2018-09-09-at-8.02.28-a561eaf1f1e611e882408fd8d6a768e5-w800px.png"
}
]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.