Skip to content

Instantly share code, notes, and snippets.

@Skorch
Last active January 18, 2021 09:13
Show Gist options
  • Save Skorch/cc970e88c1472f2c8c08702a53b382bd to your computer and use it in GitHub Desktop.
Save Skorch/cc970e88c1472f2c8c08702a53b382bd to your computer and use it in GitHub Desktop.
AWS Lambda function which receives an S3 upload event, fetches the custom headers, parses the encoded payload, and handles the API call
var async = require('async');
var AWS = require('aws-sdk');
AWS.config.update({region:'us-east-1'});
var request = require('request');
var s3 = new AWS.S3({ apiVersion: '2006-03-01' });
var sns = new AWS.SNS();
var new_upload_arn = "arn:aws:sns:us-east-1:346805855669:vuedating_new_presenece";
//Lambda entry point
exports.handler = function(event, context) {
//S3 sends a batch of events. Need to handle the possibility of mutliple upload events
async.each(event.Records, processSingleEventRecord, context.done);
};
//Generic function to fetch the header, and extract the parameters
var processSingleEventRecord = function(event, callback){
var bucket = event.s3.bucket.name;
var key = decodeURIComponent(event.s3.object.key.replace(/\+/g, ' '));
var params = {
Bucket: bucket,
Key: key
};
//Get the header info for the upload, parse, validate, and process
s3.headObject(params, function(err, data) {
if (err) {
var message = "Error getting object " + key + " from bucket " + bucket +
". Make sure they exist and your bucket is in the same region as this function.";
callback(message);
} else {
//When we created this metadata in iOS, we used the prefix 'x-amz-meta-'
//This has been stripped as that prefix was only for the HTTP headers
var uploadParametersEncoded = data.Metadata['uploadParameters'];
var uploadParameters = JSON.parse(uploadParametersEncoded);
processFileUpload(bucket, key, uploadParameters, function(error){
if(error){
return callback(error)
}
return callback()
});
}
});
};
var processFileUpload = function(bucket, fileName, upload_parameters, callback){
//Do whatever is necessary to verify and/or transform the parameters
//We created these when we encoded the JSON on the iOS project
var param1 = upload_parameters.param_1;
var param2 = upload_parameters.param_2;
//Construct the necessary parameters for the post-upload API request
var apiParameters = {
api_param_1: param1,
api_param_2: param2
};
//From here, one could make a REST Request.
//If you're in the AWS ecosystem, and have a microservice/lambda architecture,
// you could instead publish to SNS topic.
publishNewUpload(presence, callback);
};
//Pro Tip: instead of natively invoking API call, publish to SNS topic
//This provides loose coupling between dependencies
var publishNewUpload = function(presence, callback){
if(!presence){
callback()
}
var message = {
"default": JSON.stringify(presence)
};
var params = {
Message: JSON.stringify(message),
Subject: "New Upload Notification",
TargetArn: new_upload_arn,
MessageStructure: "json"
};
sns.publish(params, callback);
};
@Skorch
Copy link
Author

Skorch commented Aug 8, 2016

This is an AWS Lambda handler which expects a PUT event from S3. It then fetches the S3 asset's header information, and attempts to extract the expected encoded API parameters. With those, it can then make the subsequent call into the API - this could be inserting a new record into a database with the path to the S3 file. It could publish this file elsewhere - slack, trigger a CI server, create a push notification, further image processing/analysis, etc

The purpose of this is to enable a single upload from a mobile device and have an API be called as a side-effect. This is so that the device can fire-and-forget, and doesn't need to be active to make that subsequent API call.

In this example, the "API call" is actually just a publish to SNS topic. In most of my work, I will have one or more Lambda functions (or even publish to a SQS worker queue) subscribing to this Lambda and performing a single task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment