Freepbx Voicemail Transcription Script: Google Speech API

sendmail-gcloud

#!/bin/sh

# sendmail-gcloud
#
# Original source created by N. Bernaerts: https://github.com/NicolasBernaerts/debian-scripts/tree/master/asterisk
# modified per: https://jrklein.com/2015/08/17/asterisk-voicemail-transcription-via-ibm-bluemix-speech-to-text-api/
# modified per: https://gist.github.com/lgaetz/2cd9c54fb1714e0d509f5f8215b3f5e6
# current version: https://gist.github.com/tony722/7c6d86be2e74fa10a1f344a4c2b093ea
#
#
# Notes: This is a script modified from the original to work with a FreePBX Distro PBX so that email notifications sent from
#        Asterisk voicemail contain a a speech to text transcription provided by Google Cloud Speech API
#
# License: There are no explicit license terms on the original script or on the blog post with modifications
#          I'm assumig GNU/GPL2+ unless notified otherwise by copyright holder(s)
#
# Usage: copy this file to /usr/sbin/ set ownership to asterisk:asterisk and make it executable.
#        In the [general] section of /etc/asterisk/voicemail.conf set mailcmd=/usr/sbin/sendmail-gcloud
#            or in FreePBX 14 GUI go to Settings->Voicemail->Email Config->Mail Command to set this.
#        This script also uses dos2unix, ensure it is executable by the asterisk user (chmod 777)
#
# Requirements: 
#   Google Cloud services account (paid--free trial available) with Google Speech to Text API enabled
#   gcloud cli utility https://cloud.google.com/sdk/docs/downloads-yum
#      gcloud utility must be authenticated. Before doing this, 'su asterisk' (or 'su -s /bin/bash asterisk') so authentication happens in the correct user account
#      authenticate with: gcloud auth login
#      select project: gcloud config set project <Project ID>
#   jq (yum install jq)
#   sox with flac
#
# Version History:
# 2019-02-27  Initial commit by tony722

# set PATH
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

# save the current directory
pushd .
 
# create a temporary directory and cd to it
TMPDIR=$(mktemp -d)
cd $TMPDIR
 
# dump the stream to a temporary file
cat >> stream.org
 
# get the boundary
BOUNDARY=$(grep "boundary=" stream.org | cut -d'"' -f 2)
 
# if mail has no boundaries, assume no attachment
if [ "$BOUNDARY" = "" ]
	
then
	# send the original stream
	mv stream.org stream.new
else
	# cut the original stream into parts
	# stream.part  - header before the boundary
	# stream.part1 - header after the bounday
	# stream.part2 - body of the message
	# stream.part3 - attachment in base64 (WAV file)
	# stream.part4 - footer of the message
	awk '/'$BOUNDARY'/{i++}{print > "stream.part"i}' stream.org
 
	# cut the attachment into parts
	# stream.part3.head - header of attachment
	# stream.part3.wav.base64 - wav file of attachment (encoded base64)
	sed '7,$d' stream.part3 > stream.part3.wav.head
	sed '1,6d' stream.part3 > stream.part3.wav.base64

	# convert the base64 file to a wav file
	dos2unix -o stream.part3.wav.base64
	base64 -di stream.part3.wav.base64 > stream.part3.wav

	# convert the wav file to FLAC
	sox -G stream.part3.wav --channels=1 --bits=16 --rate=8000 stream.part3.flac

	# convert to MP3
	sox stream.part3.wav stream.part3-pcm.wav
	lame -m m -b 24 stream.part3-pcm.wav stream.part3.mp3 
	base64 stream.part3.mp3 > stream.part3.mp3.base64

	#create mp3 mail part
	sed 's/x-[wW][aA][vV]/mpeg/g' stream.part3.wav.head | sed 's/.[wW][aA][vV]/.mp3/g' > stream.part3.new
	dos2unix -o stream.part3.new 
	unix2dos -o stream.part3.mp3.base64
	cat stream.part3.mp3.base64 >> stream.part3.new

	RESULT=`gcloud ml speech recognize stream.part3.flac --language-code='en-US'`

	FILTERED=`echo "$RESULT" | jq -r '.results[].alternatives[].transcript'`
	 	   
	# generate first part of mail body, converting it to LF only
	mv stream.part stream.new
	cat stream.part1 >> stream.new
	sed '$d' < stream.part2 >> stream.new

	# beginning of transcription section
	echo "" >> stream.new
	echo "--- Google transcription result ---" >> stream.new

	# append result of transcription
	if [ -z "$FILTERED" 	]
	then
	  echo "(Google was unable to recognize any speech in audio data.)" >> stream.new
	else
	  echo "$FILTERED" >> stream.new
	fi

	# end of message body
	tail -1 stream.part2 >> stream.new

	# add converted attachment  
	cat stream.part3.new >> stream.new

	# append end of mail body, converting it to LF only
	echo "" >> stream.tmp
	echo "" >> stream.tmp
	cat stream.part4 >> stream.tmp
	dos2unix -o stream.tmp
	cat stream.tmp >> stream.new
 
fi
 
# send the mail thru sendmail
cat stream.new | sendmail -t
 
# go back to original directory
popd
 
# remove all temporary files and temporary directory
rm -Rf $TMPDIR

I have been attempting to make this work for the last 4 hours and getting nowhere. I know I have google cloud set up correctly because I can change the TMPDIR to an actual location and have the system write the files there then copy the "gcloud ml speech recognize stream.part3.flac --language-code='en-US'" command and run it in that dir as the asterisk user and get the JSON however it is not running as part of the script.

This is code that I'm using on a production FreePBX server (just diffed it to be sure) . Keep trying, checking logs, messages, permissions, and whatever else. Hopefully you can get it. 😄 Good luck!

I forgot to post back I decided to try a stupid this morning and "reboot -h now" I left a vm after the system restarted and it transcribed properly.

Thanks a lot for your great work.
Could you please make this work with VitalPBX?
This is working good with IBM Watson (example):
http://incrediblepbx.com/sendmailibm-vitalpbx.tar.gz
Doc here: https://nerdvittles.com/?p=25697

But i would love this do work with Google Speech to Text.

Tried copy/paste your script but i only get:
--- Google transcription result ---
(Google was unable to recognize any speech in audio data.)

That's a great idea @pete1019. However I'm not running VitalPBX so I don't have a way to do this. I maintain a single FreePBX system, and just put this out there in case it helps anyone else. :-) I will say that I get this message from Google occasionally. However if you get it every time there's likely a problem.

Feel free to fork this and make any changes needed. Good luck!

I think it is just a owner / rights problem.
Everything works but nothing happens here when your script is used here:
RESULT=gcloud ml speech recognize stream.part3.flac --language-code='en-US'

If i run this command as root everyhing is fine and i get results.
How should i set rights and ownership in CentOS since you said:
"gcloud utility must be authenticated. Before doing this, 'su asterisk' so authentication happens in the correct user account"
I did this as root cause i did not know any better.
If i do su asterisk it says: This account is currently not available. Maybe since user asterisk has nologin? Solution?
THANK YOU!

I've never looked at VItalPBX at all, so I don't know what user accounts it runs under. I'd suggest taking this script to the guys on the VitalPBX forums and see if they can help--I'm really unable to help at all! 😄 Good luck!

"gcloud utility must be authenticated. Before doing this, 'su asterisk' so authentication happens in the correct user account"
This does the trick:
su -s /bin/bash asterisk

I've been using this script for some time now (with FreePBX 14), and was previously using the IBM Bluemix version with no issues. However lately I've noticed that every several weeks I'll start getting the "Google was unable to recognize any speech in audio data." result. But if I pipe a raw message through the script as the asterisk user it works perfectly fine. It seems the issue only occurs when the script is called by FreePBX. Only a reboot seems to solve it. Dug through the mail/asterisk/gcloud logs and I'm not seeing anything useful. Any ideas?

Might be good to ask on the FreePBX forums. I've seen that kind of issue randomly, but it's always the odd message here or there and never required a reboot. I'm not even pretending to be an expert on all this. But this script worked for me so I posted it here. 😄 Please post back a fix here if you find one! :-)