@tm1000 did something with AWS Lex and Polly as a demo.
There was some latency due to the back and forth. I think at the right scale this could work.
Lumenvox is something that can be deployed on-prem, which speeds things up, and they have decent examples on how to connect.
https://www.lumenvox.com/resources/speech-under-hour/asterisk.aspx