I built and open-sourced a self-hosted, real-time voice AI stack (STT → LLM → TTS) that answers calls and holds a natural spoken conversation, running entirely on your own box — no cloud, no per-minute billing, audio never leaves your infra. It’s free, no signup, and ships with the bridge. Full disclosure: it’s my project and I’d love your feedback.
Works with FreePBX (it runs Asterisk under the hood): the bundled bridge connects via ARI + external-media (RTP, g711 μ-law) into a Stasis app — you route the calls you want into it. Validated with a live PJSIP call. Barge-in works.
What it’s for (all on-prem): contact-center automation, appointment booking & reminders (clinics/salons), IVR replacement (real conversation, no press-1 menus), 24/7 support hotlines — no per-minute cloud, and the call audio stays on your box.
Runs desktop → datacenter (sub-second TTFA):
DGX Spark (~$4K desktop) → ~20 concurrent calls
H100 / H200 → ~75 concurrent · scales ~linearly
Even a consumer RTX 4090 (just validated) — the whole stack fits in ~21.5/24 GB
Drop-in bonus: the server speaks the OpenAI Realtime API protocol, so if you’ve already wired cloud OpenAI Realtime to Asterisk/FreePBX, just point it at your own box — change the URL, done. Stack: NVIDIA NeMo (STT/TTS) + Qwen3-8B FP8 on vLLM. Bilingual ES/EN.
Why the NVIDIA GPU is a feature, not a tax: local inference = ~0.1 s first-audio and consistent (no cloud jitter/rate-limits), one-time card vs per-minute forever, and no kill-switch if a provider deprecates a model or cuts access.
This is misleading. The GPU is only a feature if it’s actually needed but it’s 100% a tax. It’s a cost center plain and simple.
A saturated GPU can run horribly and much worse than API calls. This is a conditional thing. Generally it’s true but the actual workload will determine how true it stays.
This is not true at all. AI inference hardware is moving at an exceptional pace. While a GPU from 2026 will work in 2030 by then the models, CUDAs, etc will have left the GPU behind. Anyone doing AI inference hardware deployments should expect a 4-5 year refresh window to stay current. Sure you could never update but as previously stated, you will be left behind. Your GPU will not be able to use current things and you’re frozen in time with what your GPU can do.
“One-time Card” actually means on top of GPU/hardware you have:
electricity + cooling + maintenance cycles + admin costs + 100% responsibility. Then you have to account for refresh cycles.
For example, the DGX Spark will be about $6K (mid-range) for the GPU + hardware. But since that’s only CapEx, there is still recurring expenses (above). Over a 36 month period that is roughly $9,000 - $10,800 in total cost of ownership. So you’re really looking at a $250-$300/month cost for this solution even though you paid $6K upfront, this is the real monthly cost you need to use to determine “is this worth it”. For example, paying $100/month for OpenAI API usage…this increases your monthly costs by almost 3x. However, paying $400/month this saves you costs. Now this is based on 20 or less concurrent calls, you need more than 20 then the monthly TCO jumps almost 4x as well.
Then will you refresh after five years? You’ve already spent $11K - $14K for the 5 year lifespan, continue to pay $996 - $1,596 a year for outdate performance or repeat the cycle?!
This isn’t “I bought a Digium Wildcard TE110P 15 years ago and it still performs 100%” this is “I bought a GPU 5 years ago and now it can’t perform 100% because it’s outdated and nothing cares about it any more. What took it 1 second to perform now takes 15 seconds”.
Unlike telecom hardware you just can’t let it sit forever and expect full functionality. With AI it will never be “one time and done”, it will be “one time now and another time in 5 years and another time in 5 years, rinse…repeat”
On “feature, not a tax”: fair, that phrasing was too cute. It is a capital cost. Whether it’s a tax or an investment depends entirely on what it replaces — which is a volume question, and there we mostly agree.
On the TCO math: ~$250–300/mo is a reasonable standalone allocation. But the buyer here isn’t greenfield — it’s a provider that already runs racks, power, cooling and an ops team (i.e. most people reading this forum). Adding one box to an existing room is much closer to amortized CapEx + its power draw than a fresh full allocation.
On refresh cycles: genuine question — what hardware in your rack doesn’t get replaced? Servers, switches, drives are all on 3–5 year cycles; that’s just infrastructure, and it’s already in the budget. Where the pressure does differ: you refresh a server to stay supported, you’d refresh a GPU to run newer models. But a fixed workload doesn’t force that — a box running a voice agent today answers calls in 2030 exactly as fast as it does now. It doesn’t get slower. What does eventually bite is driver/CUDA support dropping, and that one is real — my default FP8 model already won’t run on Ampere.
Where the comparison breaks: your example is $100/mo of API usage — a few hundred call-minutes a month. At that volume you’re right: don’t buy a GPU, use the API. But this box is sized for ~20 concurrent calls (200 connected 10:1 overbooking). Actually use that capacity even a few hours a day and you’re past 100,000 minutes/month. Price that at whatever per-minute rate you’re paying and it isn’t $300 vs $100. So — agreed, this isn’t for small deployments. It’s for volume.
On saturation: completely true, everything has a limit. A saturated GPU is worse than an API call. That’s exactly why I publish concurrency numbers rather than a single latency figure — you size for your load, with headroom.
And what TCO doesn’t price: for some operators the audio simply can’t leave the building (compliance), and a provider deprecating a model or cutting access isn’t a cost line — it’s an outage.
Well, thanks, if I do it, I’ll keep that in mind. I didn’t copy it from the LLM; I simply didn’t want to get into an argument about that, but rather focus on the substance of the matter.
Seriously? You didn’t copy your response from AI while responding to the pushback? Really? What key combination do you use to type the em-dash? And in your normal writing you pepper your write-ups with bold and italics? And use fancy words like “cute” in a technical response? It’s okay to say you use AI, but to deny the use of AI in your previous response is to look us dead in the eyes and assume we are sub-intelligent (for want of a better description)
What I notice about this is that it is using a freemium model, but, unlike FreePBX itself, which also uses the freemium model, the free part, on its own, doesn’t appear to include anything that is actually useful to a business, in isolation. To make it useful you would need RAG, and/or tools, which are a premium features, or Loras, for which there seems to be no support for training, even as a premium features.
RAG (retrieval augmented generation), is running external code to fetch information from external sources, which is added to the user’s input, to allow the LLM to provide information specific to the business and user.
RAGs are special case of tools, but here I mean particularly external code that manipulates things in the real world, e.g. initiates phone calls.
Loras are small, additional, neural networks, which are trained on things important to the business, like the typical problems that customers encounter.
The free part only seems to support idle chatting.
Yes, you’re right! This community version shares the fundamental workflow concept—VAD, STT, LLM, and TTS—running on your own local GPU.
The application layer is single-tenant and does not include RAG or tool calling. This version is geared toward use cases that handle voice call or chat workloads via a System Prompt; you know, you can draft all the instructions (with a context of up to 32,000 tokens) using natural language. This project can serve as an integration example; if you need additional functionality—such as multi-tenant architecture, RAG, tool calling, or new features—you can extend the application layer to suit your needs. That’s the idea!
Or, if you want to save time, you can also contact me directly.
Thank you!
Edit: Additionally, this version also simulates the server side of OpenAI’s real-time API; perhaps that will also be of interest. Regards.