Skip to content

Instantly share code, notes, and snippets.

@jtsage
Created March 21, 2020 21:14
Show Gist options
  • Save jtsage/f38151b43debb6bd90ebaa4f0ab2687f to your computer and use it in GitHub Desktop.
Save jtsage/f38151b43debb6bd90ebaa4f0ab2687f to your computer and use it in GitHub Desktop.
Asterisk Voicemail --> MP3 w/ IBM bluemix trasnscription
#!/bin/sh
# Asterisk Voicemail -> MP3 And IBM Bluemix Transscription
#
# Based on multiple peoples work:
# Asterisk voicemail attachment conversion script
# Revision history :
# 22/11/2010 - V1.0 - Creation by N. Bernaerts
# 07/02/2012 - V1.1 - Add handling of mails without attachment (thanks to Paul Thompson)
# 01/05/2012 - V1.2 - Use mktemp, pushd & popd
# 08/05/2012 - V1.3 - Change mp3 compression to CBR to solve some smartphone compatibility (thanks to Luca Mancino)
# 01/08/2012 - V1.4 - Add PATH definition to avoid any problem (thanks to Christopher Wolff)
# 16/07/2015 - V1.5 - Handle natively GSM WAV (thanks to Michael Munger)
# Also stole the work of:
# Original source created by N. Bernaerts: https://github.com/NicolasBernaerts/debian-scripts/tree/master/asterisk
# modified per: https://jrklein.com/2015/08/17/asterisk-voicemail-transcription-via-ibm-bluemix-speech-to-text-api/
#
# 2017-09-17 Initial commit by lgaetz, working but unpolished
# 2017-09-18 Original attachment added to email
# 2017-09-18 working with email sans attachments
#
#
# License: There are no explicit license terms on the original script or on the blog post with modifications
# I'm assumig GNU/GPL2+ unless notified otherwise by copyright holder(s)
#
# Usage: copy this file to /usr/sbin/ set ownership to asterisk:asterisk and make it executable.
# In the [general] section of /etc/asterisk/voicemail.conf set mailcmd=/usr/sbin/sendmail.asterisk
#
# Permissions - More: Make sure the asterisk user can execute (and that they exist - fwiw, a new
# install of FreePBX circa March 2020, I didn't need to change or install anything):
# * dos2unix
# * unix2dos
# * sox
# * lame
#
# set PATH
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
# Specify IBM Bluemix API credentials
API_KEY="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
API_URL="https://api.us-south.speech-to-text.watson.cloud.ibm.com/instances/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/v1/recognize"
API_OPTS="?model=en-US_NarrowbandModel"
# save the current directory
pushd .
# create a temporary directory and cd to it
TMPDIR=$(mktemp -d /tmp/astVM.XXXXXXXXXX)
cd $TMPDIR
# dump the stream to a temporary file
cat >> stream.org
# get the boundary
BOUNDARY=$(grep "boundary=" stream.org | cut -d'"' -f 2)
# cut the file into parts
# stream.part - header before the boundary
# stream.part1 - header after the bounday
# stream.part2 - body of the message
# stream.part3 - attachment in base64 (WAV file)
# stream.part4 - footer of the message
awk '/'$BOUNDARY'/{i++}{print > "stream.part"i}' stream.org
# if mail is having no audio attachment (plain text)
PLAINTEXT=$(cat stream.part1 | grep 'plain')
if [ "$PLAINTEXT" != "" ]
then
# prepare to send the original stream
cat stream.org > stream.new
# else, if mail is having audio attachment
else
# cut the attachment into parts
# stream.part3.head - header of attachment
# stream.part3.wav.base64 - wav file of attachment (encoded base64)
sed '7,$d' stream.part3 > stream.part3.wav.head
sed '1,6d' stream.part3 > stream.part3.wav.base64
# convert the base64 file to a wav file
dos2unix -o stream.part3.wav.base64
base64 -di stream.part3.wav.base64 > stream.part3.wav
# Send WAV to Watson Speech to Text API. Must use "Narrowband" (aka 8k) model since WAV is 8k sample
CURL_OPTS=""
curl -s $CURL_OPTS -k -u "apikey:$API_KEY" -X POST \
--limit-rate 40000 \
--header "Content-Type: audio/wav" \
--data-binary @stream.part3.wav \
"$API_URL$API_OPTS" 1>audio.txt
# Extract transcript results from JSON response
TRANSCRIPT=`cat audio.txt | grep transcript | sed 's#^.*"transcript": "##g' | sed 's# "$##g'`
# convert wave file (GSM encoded or not) to PCM wav file
sox stream.part3.wav stream.part3-pcm.wav
# convert PCM wav file to mp3 file
# -b 24 is using CBR, giving better compatibility on smartphones (you can use -b 32 to increase quality)
# -V 2 is using VBR, a good compromise between quality and size for voice audio files
lame -m m -b 24 stream.part3-pcm.wav stream.part3.mp3
# convert back mp3 to base64 file
base64 stream.part3.mp3 > stream.part3.mp3.base64
# generate the new mp3 attachment header
# change Type: audio/x-wav or audio/x-WAV to Type: audio/mpeg
# change name="msg----.wav" or name="msg----.WAV" to name="msg----.mp3"
sed 's/x-[wW][aA][vV]/mpeg/g' stream.part3.wav.head | sed 's/.[wW][aA][vV]/.mp3/g' > stream.part3.mp3.head
# generate first part of mail body, converting it to LF only
mv stream.part stream.new
cat stream.part1 >> stream.new
sed '$d' < stream.part2 >> stream.new
# beginning of transcription section
echo -e "\n--- Automated transcription result ---\n " >> stream.new
# append result of transcription
echo "$TRANSCRIPT" >> stream.new
# end of message body
tail -1 stream.part2 >> stream.new
cat stream.part3.mp3.head >> stream.new
dos2unix -o stream.new
# append base64 mp3 to mail body, keeping CRLF
unix2dos -o stream.part3.mp3.base64
cat stream.part3.mp3.base64 >> stream.new
# append end of mail body, converting it to LF only
echo "" >> stream.tmp
echo "" >> stream.tmp
cat stream.part4 >> stream.tmp
dos2unix -o stream.tmp
cat stream.tmp >> stream.new
fi
# send the mail thru sendmail
cat stream.new | sendmail -t
# go back to original directory
popd
# remove all temporary files and temporary directory
# if things are not working, this is a great place to
# start. If, as in my experience, you can't find the
# temp folder than, check your spelling that FreePBX
# is trying to run the right script.
rm -Rf $TMPDIR
@jtsage
Copy link
Author

jtsage commented Mar 21, 2020

Couple of things for debugging purposes that it took a while for me to notice:

1.) FreeBPX / Asterisk don't appear to tell you if it can't find the script (this was not the correct filename at that point)

DEBUG[24625][C-00000187] app_voicemail.c: Sent mail to <redacted> with command '/usr/sbin/sendmial.asterisk'

2.) If you do turn off removal of the temp directory to pipe the steam file in again:

# cat /tmp/astVM-XXXXXXXX/stream.org | /usr/sbin/sendmail.asterisk

IT WILL NOT APPEAR. There is a message-ID header in there, duplicates are ignored, at least by gmail - change it a bit each time.

3.) Errors from IBM are accessible in the audio.txt file in the created temp directory

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment