Send calls on Outgoing Number to an AI application for human voice detection and send the call back to agents when it detects call is answered by the real human

Hi Guys,

we have developed an application which will detect if the Call is picked by answering machine or a Human.
I want to integrate this with my Freepbx on outgoing Calls on mostly Insurance Calls and route or shadow whatever is best to my AI Application which can hold the call to itself and when it checks the Agent is started to talk it will route to my Agent.

now the tricky part is how to send the call to Application and get it back from the application to agent.
My application is in python.

This functionality already exists in Asterisk. Answering Machine Detection (AMD). Have you looked at that?

Conceptually, I agree, but what the OP is looking for is the ā€˜how toā€™ part, and looking at any one of several system components (like CID Superfecta or AMD) would work to get you there.

Effectively, it works the same basic way any other ā€˜reach out to another systemā€™ component would - it passes the information around through return codes and pass-by-reference data structures. You have to get the call into your application, then pass the call back to Asterisk using return codes. AMD is an excellent way to approach this, since it performs a similar set of activities.

I doubt that you can make this work with usable reliability. When the called agent takes the call from his queue, he is trained to give a greeting like ā€œThank your for calling State Farm, this is Jesse.ā€ Often, itā€™s even longer. So your system decides ~2 seconds later that a human is on the line, about the time that Jesse says ā€œHello?ā€ About 3 seconds after that, Jesse hangs up, being well aware that his performance is measured by number of issues handled per hour. So, the system needs to get your agent on the line within about 4 seconds, otherwise the call is lost and has to be restarted. Unless you usually have an agent sitting idle, thatā€™s unlikely to happen.

1 Like

Now our OP is going AI which is a good thing, as such why would one not immediately have an AI agent answer the call which would say exactly the same greeting but further process the ongoing interaction with AI. (AMD is essentially a binary decider, he will need STT here to keep up with the current sophistication of todayss robocallers, if they do use it, and they do, you will need to fight fire with fire.)

The ultimate aim is to discover ā€˜real peopleā€™ and handle them in a timely fashion but send bots wherever you decide they belong.

1 Like

As mentioned, AMD, or other Asterisk applications like BackgroundDetect, Talk_Detect might be very useful.

Anyway, to answer your question, Iā€™d probably use ARI.

Actually what we are doing is from python application we are transcribing the media stream and set up few checks on words that will ensure us that now the Agent is speaking.

so this is where we want to route the call back to Asterisk.

But the problem is If we handover the call to python application, how can we do it in the dialplan for outgoing calls and when call hit the url what will happen and how can we take the instructions back from python application.

if your media processing is synchrounous, the good old way is

if you are post-processing a short interaction which is a little simpler, I just ask them their name and then who they want to be connected to, proper names can be hard to get right but offering a limited vocabulary works well.
I ā€˜holdā€™ the caller then SHELL(script.py your/recording.wav) ,anything you `print(". . .",end="ā€™) in a Set() dialplan.

ok what about If we manually dial the call and send the audio stream to a url in parallel to the call.

how is that possible ?

Ho can I send the live call stream to another url ?

I pmā€™d you

that would be the STDIN file in your eagi is one leg , STDOUT is the other, whatever you play on STDERR will be heard by both legs

any example of how to do it cause I am not a developer.

really missing this Development part :frowning:

I PMā€™d you

I have to send the live call stream to the below url:

192.168.1.10:8888/audio

how can I do that.

That requirement is underspecified. What scheme? What method? (Probably has to be POST, but might be PUT, or CONNECT.) What Content-Type? What constraints within that content type?

Basically, though, that part of the solution is independent of getting it out of Asterisk. EAGI will given you a linear PCM stream, although Iā€™m not sure if it is locked at 8kHz sampling. You need to find tools that will convert that to meet the interface specification of the recipient.

Is there any other way you think which will work like we give audio to that application.

If you google res_speech asterisk you will find several ā€˜appsā€™ that are based on our own @jcolp 's work

https://asterisk-doxygen.osso.pub/master/api/db/d2a/res__speech_8c.html

perhaps he can put you in touch with the right Sangoma resource for your own solution.

1 Like

Thereā€™s nothing inherently built in for such a thing. The module referenced is just an application interface. Otherwise there is external media in ARI[1].

[1] https://www.asterisk.org/external-media-a-new-way-to-get-media-in-and-out-of-asterisk/

1 Like

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.