FreePBX Voicemail Transcription - UPDATED to use Google Cloud

All,

I wanted to post a big update I made to my original voicemail transcription system. There have been 2 major updates:

  • Conversion from Python 2.7 to Python 3
  • Conversion from Microsoft cognitive services to Google speech services

The primary reason for conversion away from Microsoft is that, quite plainly, Sangoma OS v7 doesn’t have a new enough glibc++ to support Microsoft’s python3 library. Additionally, it was a lot easier to implement transcription for long voicemails.

Here is the Github link: https://github.com/westparkcom/FreePBX-VM-Transcription

Please let me know if you encounter any issues or if you have an improvement suggestion. Thanks!

4 Likes

Thank you for a great idea and project. Any idea on how to debug problems?
I’ve followed the instructions carefully, but after installing and configuring everything I now don’t get any emails from the PBX at all.

Can I somehow manually invoke the scripts with an audio file as parameter so I can watch where things go wrong? On the Google Page I don’t see any requests in the log, so I guess it doesn’t even go this far.

One thing I noticed, while installing with pip3.6: first it aborted, and told me I should upgrade pip (which I did), then it installed fine. But it complained “Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv” Was it ok to run that as root? If not, as what user should I have installed the pip packages?


Update: I found how to debug it. I added “echo “$VALUE” >> /tmp/FreePBX-VM-Transcription.log” to the emailproc script, and then was able to feed this file using cat /tmp/FreePBX-VM-Transcription.log | sttparse

This allowed me to find the error. I was totally stupid and had used the wrong json authentication file by mistake. Your script then correctly complained that this was not a valid syntax.

Its working fine now and I think this might prove to be very useful. I have two improvement suggestions,

  1. Google offers additional settings, which are however not available for all languages
    I have tried the following and like the automatic punctuation. model=“phone_call” seems to make sense, but for my language (de-DE) it is not available and causes the recognition to fail. Enhanced is supposedly better and more costly, in my short tests it did not make a difference.

     enable_automatic_punctuation=True,
     model="phone_call",
     use_enhanced=True,
    
  2. my main problem with this script is that if any error occurs (for example: parameters incorrect, google api offline, insufficient funds) this leads the loss of the entire voicemail and the user is not notified (I only get the voicemails by email, they are deleted on the PBX). IMHO a failed google speech recognition should be caught programmatically, and then instead of the recognized voice the error message should be appended to the email (e.g. “google.api_core.exceptions.InvalidArgument: 400 Invalid recognition ‘config’: The phone_call model is currently not supported for language : de-DE.”) Any chance you could improve error handling? Otherwise I need to learn python and do it myself, which is bad because I have not written a single line of python code in my life :frowning:

Good point on the error handling. This was my first stab at this script so I missed a few things, I’ll see about adding that to the script, and also adding the ability to set the values of the parameters you listed in the global config area.

OK I’ve commited a patch that catches errors from google, I also added the config items suggested, please let me know if there are any other gremlins I need to chase down :slight_smile:

We’ve been using Tony722’s sendmail-gcloud script from https://gist.github.com/tony722/7c6d86be2e74fa10a1f344a4c2b093ea

My problem with it is that even though I’ve tried to get recognize-long-running working it is still giving me a transcription unavailable on anything over a minute if that flag can be added in your python script or if the python is already allowing for it that would be wonderful

Mine already deals with that as it is using the streaming capability of Google cloud, so it can do long running translations. I haven’t tested it past a minute and a half, but I have no doubt it will go longer.

Thank you for the quick fix, unfortunately the update seems to have screwed up the entire file’s formatting regarding tab and spaces and now it doesn’t run anymore. Apparently you cannot mix space indentation and tab indentation in python?

[root@freepbx bin]# cat /tmp/FreePBX-VM-Transcription-21-06-13\ 19\:57.log | sttparse.1
  File "/usr/local/bin/sttparse.1", line 44
    responses = client.streaming_recognize(
                                          ^
TabError: inconsistent use of tabs and spaces in indentation

I tried to get it to work by changing tabs and space in a couple of lines but failed :frowning:

I’m an idiot, I’ve fixed this in the latest commit, please pull it down again.

1 Like

Thank you for fixing the tab/space indentation problems, it’s working now.

I’ve tested the new try/except mechanism:

  • invalid configuration options for google: works, the error “An error was encountered when transcribing: 400 Invalid recognition ‘config’: The phone_call model is currently not supported for language : de-DE.” was correctly embedded in the voicemail as text.

  • google server not responding (to simulate this, I added 8.8.8.8 accounts.google.com oauth2.googleapis.com www.googleapis.com to the /etc/hosts file) => took a long time, but error was properly embedded in the voicemail message and the message was correctly delivered: “An error was encountered when transcribing: Deadline of 5000.0s exceeded while calling functools.partial(<function _wrap_stream_errors..error_remapped_callable”

So far, so good, however:

  • invalid json file specified in configuration: doesn’t work, voicemail is lost. When manually processing the voicemail, sttparse crashes this way:

[root@freepbx bin]# cat “/admin/VoiceMail-processing/FreePBX-VM-Transcription-21-06-14 16:15.log” | sttparse
Traceback (most recent call last):
File “/usr/local/bin/sttparse”, line 159, in
sys.exit(main())
File “/usr/local/bin/sttparse”, line 124, in main
speech = speechrec(waveobj)
File “/usr/local/bin/sttparse”, line 31, in speechrec
client = speech.SpeechClient.from_service_account_json(globalconfig(‘creds_file’))
File “/usr/lib/python3.6/site-packages/google/cloud/speech_v1/services/speech/client.py”, line 152, in from_service_account_file
credentials = service_account.Credentials.from_service_account_file(filename)
File “/usr/lib/python3.6/site-packages/google/oauth2/service_account.py”, line 234, in from_service_account_file
filename, require=[“client_email”, “token_uri”]
File “/usr/lib/python3.6/site-packages/google/auth/_service_account_info.py”, line 72, in from_filename
with io.open(filename, “r”, encoding=“utf-8”) as json_file:
FileNotFoundError: [Errno 2] No such file or directory: ‘/usr/local/etc/vmtranscibe-316511-8af2e892f278.json2’

  • invalid contents of the json file (i changed a single character in the key from 0 to 1) => Voicemail is lost and never arrives

[root@freepbx bin]# cat “/admin/VoiceMail-processing/FreePBX-VM-Transcription-21-06-14 16:15.log” | sttparse
Traceback (most recent call last):
File “/usr/local/bin/sttparse”, line 159, in
sys.exit(main())
File “/usr/local/bin/sttparse”, line 124, in main
speech = speechrec(waveobj)
File “/usr/local/bin/sttparse”, line 31, in speechrec
client = speech.SpeechClient.from_service_account_json(globalconfig(‘creds_file’))
File “/usr/lib/python3.6/site-packages/google/cloud/speech_v1/services/speech/client.py”, line 152, in from_service_account_file
credentials = service_account.Credentials.from_service_account_file(filename)
File “/usr/lib/python3.6/site-packages/google/oauth2/service_account.py”, line 234, in from_service_account_file
filename, require=[“client_email”, “token_uri”]
File “/usr/lib/python3.6/site-packages/google/auth/_service_account_info.py”, line 74, in from_filename
return data, from_dict(data, require=require)
File “/usr/lib/python3.6/site-packages/google/auth/_service_account_info.py”, line 55, in from_dict
signer = crypt.RSASigner.from_service_account_info(data)
File “/usr/lib/python3.6/site-packages/google/auth/crypt/base.py”, line 114, in from_service_account_info
info[_JSON_FILE_PRIVATE_KEY], info.get(_JSON_FILE_PRIVATE_KEY_ID)
File “/usr/lib/python3.6/site-packages/google/auth/crypt/_cryptography_rsa.py”, line 134, in from_string
key, password=None, backend=_BACKEND
File “/usr/lib64/python3.6/site-packages/cryptography/hazmat/primitives/serialization.py”, line 20, in load_pem_private_key
return backend.load_pem_private_key(data, password)
File “/usr/lib64/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py”, line 1006, in load_pem_private_key
password,
File “/usr/lib64/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py”, line 1225, in _load_key
self._handle_key_loading_error()
File “/usr/lib64/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py”, line 1275, in _handle_key_loading_error
raise ValueError(“Unsupported public key algorithm.”)
ValueError: Unsupported public key algorithm.

So it currently correctly catches the specific reason of a google server responding with an error and the google server not responding at all, but not an incorrect json configuration file, incorrect json authdata in the file… If you have time for further reliability improvements, this would be much appreciated. Perhaps there could some very generic error handling, so whatever the error was, at least the voicemail continues to be delivered? I don’t know if in python you can catch those generic Traceback messages and just embed them as transcribed text - that’s how I would approach it in C#.

I’ve moved the try, except block to encompass the configuration file issues as well, it is committed to Git if you wish to pull and try again.

Thank you very much, I’ve done my tests again and it now passed with flying colors. Also the error message that came with the email was very nice and concise.

Thank you again for your great work with this amazing tool.

That’s great I’ll be trying it out as soon as I get time to mess with it to get it going

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.