Skip to content

Instantly share code, notes, and snippets.

@ppaquette
Last active November 26, 2021 08:22
Show Gist options
  • Save ppaquette/67149e69e3313eb3b7c5924a1198b3b1 to your computer and use it in GitHub Desktop.
Save ppaquette/67149e69e3313eb3b7c5924a1198b3b1 to your computer and use it in GitHub Desktop.
Lambda to export twilio recordings to S3
'use strict';
const axios = require('axios');
const AWS = require('aws-sdk');
const S3UploadStream = require('s3-upload-stream');
const Twilio = require('twilio');
const ACCOUNT_SID = "AC2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
const AUTH_TOKEN = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
const S3_BUCKET = "BUCKET-NAME";
const S3_PREFIX = "Recordings";
function add_leading_zero(number) {
let n = number.toString();
if (n.length < 2) { n = '0' + n; }
return n;
}
function get_download_url(recording) {
const apiVersion = recording.apiVersion;
const recordingSID = recording.sid;
return `https://api.twilio.com/${apiVersion}/Accounts/${ACCOUNT_SID}/Recordings/${recordingSID}.mp3`
}
function get_upload_path(recording, from_number, to_number) {
const recordingDate = new Date(recording.dateCreated);
const year = recordingDate.getFullYear();
const month = add_leading_zero(recordingDate.getMonth() + 1);
const day = add_leading_zero(recordingDate.getDate());
const hour = add_leading_zero(recordingDate.getHours());
const minute = add_leading_zero(recordingDate.getMinutes());
const second = add_leading_zero(recordingDate.getSeconds());
const duration = add_leading_zero(recording.duration);
return `${S3_PREFIX}/${year}/${month}/${year}.${month}.${day}-${hour}.${minute}.${second}_${from_number}_${to_number}_${duration}s.mp3`
}
async function transfer_recording(download_url, upload_stream) {
const response = await axios({method: 'GET', url: download_url, responseType: 'stream'});
response.data.pipe(upload_stream);
return new Promise((resolve, reject) => {
upload_stream.on('uploaded', resolve)
upload_stream.on('error', reject)
});
}
module.exports.handler = async function(event, context, callback) {
const client = Twilio(ACCOUNT_SID, AUTH_TOKEN);
// Retrieving and deleting all recordings
const recordings = await client.recordings.list();
for (const recording of recordings) {
if (recording.status !== "completed") { continue; }
// Getting the upload and download paths
const call = await client.calls(recording.callSid).fetch();
const download_url = get_download_url(recording);
const upload_path = get_upload_path(recording, call.from, call.to);
let s3Stream = S3UploadStream(new AWS.S3());
// Alternatively, one could download a ".wav" by using ".wav" in the in the download_url and a ContentType of "audio/x-wav"
let upload_stream = s3Stream.upload({Bucket: S3_BUCKET, Key: upload_path, ContentType: 'audio/mpeg'});
// Transferring to S3
console.log(`Transferring ${recording.callSid} to ${upload_path}`);
await transfer_recording(download_url, upload_stream);
// Deleting recording
await client.recordings(recording.sid).remove();
}
callback();
};
@margie-gd
Copy link

image
I deployed this code to my local and throw this error. Do you know why?

@ppaquette
Copy link
Author

Yes. It is a credentials issue. You need to load an access key and secret key in your local environment variables so the script can upload to s3.

@margie-gd
Copy link

sorry, I don't know how to pass it. how to make aws upload request carry accesskey and secret key?

@ppaquette
Copy link
Author

You need to login to the console, create a user, assign him a policy that allows uploading to s3 and then you will get the access key and secret key.

then set it with something like

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_DEFAULT_REGION=us-west-2

@margie-gd
Copy link

image
Now the recording can be upload to S3, but it's throw this warning, do you know why? and sometimes it's will run a long time(more than 1h) in here:"let upload_stream = s3Stream.upload({Bucket: S3_BUCKET, Key: upload_path, ContentType: 'audio/mpeg'});"
FYI, I pass credentialData like this:
let credentialsData={ accessKeyId: 'AKIASXXXXXXUXHFPU', secretAccessKey: 'TAuXXXXXXXXXXXXXLe4mN8jOfrUemvIJfb7fe6swK' };
let s3Stream = S3UploadStream(new AWS.S3(credentialsData));

@margie-gd
Copy link

When It's very slow on code:"let upload_stream = s3Stream.upload({Bucket: S3_BUCKET, Key: upload_path, ContentType: 'audio/mpeg'});"
it's throw warning like this.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment