Best method to troubleshoot call quality

Some of my users are reporting poor audio quality (jumbled or broken audio) or calls dropping altogether. It’s pretty rare, but it is happening. When this happens, is there a best-practices procedure to find the culprit? Is there a log that may show me the issue or a test that I can run?

In one word, Wireshark

1 Like

It should not be hard to see (from Asterisk’s point of view) what is going wrong, though it may not be easy to fix. As root, start a tcpdump capture of everything on the server, using a command like
tcpdump -s 0 -C 100 -W 100 -w rbuf -Z root &
This writes a circular buffer of 100 files (rbuf00, rbuf01, … rbuf99), each 100 MB long. Be sure that you have 10 GB of extra disk space available.

If there are ~100 active call legs @ ~ 80 kbps * 2 each, you’ll have ~1.4 hours of historical data. When trouble is reported, identify the time the call ended, find the file (based on modification time) containing the problem, copy it to your PC and analyze it with Wireshark.

1 Like

Thanks, Stewart1. I’ve started the tcpdump. I’ll wait now until there is an issue.

So while that is happening you can describe the network setup. How the phones connect to the PBX (locally/remotely, what type of Internet there is where the PBX and/or phones are (speeds, etc). When does this happen? Internal or external calls?

If this is a very rare thing to happen then running a wide capture that is going to grab everything is not only going to take up space on the server to do the capture but at the end of the day you have to go through it to see what the issue is.

The issues you are describing are due to poor networking/bandwidth. Audio deteriorates like that due to jitter, packet loss, latency, etc. Knowing more about the actual setup of where the PBX and phones are in relation to each other can help narrow down the issue.

We have Mediacom Cable internet at each location with about 100mb down and 20mb up. I ran voip tests at each location. The results are below. I ran 7 tests at each location, on test 1 at the server location the jitter hit 86ms, on the last 6 tests it was between 1-8ms. All 7 tests at the remote location had a litter under 6ms

3

At the remote location, the cable modem is connected to an ASUS RT-N66R router. The Sangoma S500 phones are connected to an umanaged POE switch. It’s the same at the server location but we use an ASUS RT-AC1750 router. The phones use the built-in VPN connection to the PBX server. I have QoS enabled on each router with highest priority assigned to ports 10001-20000 (TCP/UDP).

My jitter buffer in FreePBX is disabled.

Sangoma phones are running firmware v. 2.0.4.67

I have only heard complaints about external calls, but we don’t do a ton of internal calls.

Not sure if this is related, but my Sangoma S500 desk phone will run slow after a few days. It seems like once a week I have to power-cycle the phone.

Jitter is rather sensitive. At 30ms of jitter the call is useless and you can end up with the call dropping or at least being very distorted. That means when you get to about 8-10ms of jitter in a call, you’re going to have issues. Keeps getting higher, there will be more. Even 5ms of jitter can cause audio gaps or out of order packets making it garbled.

Also, if both locations are supposed to be 100x20 for the speeds, I’d be concerned as to why your remote location is getting less than 50% of its upload speed and why your upload at the server location isn’t getting up to its 20Mbps mark either.

The speeds from my ISP are “advertised” speeds. Also, I had my QoS on when testing.

Would you recommend using the Jitter buffer in FreePBX to smooth the calls out?

I did not realize that VPN was involved.

Do the quality issues affect only phones at the remote location? If HQ users are also affected, that should be much simpler to troubleshoot.

Is the jumbled / broken audio heard by the extension user, the external party, or both? About how often does it occur and how long does it last?

In a situation where the trouble occurs and two or more calls are in progress, are both (or all) calls affected, or just one?

What kind of trunks do you have? If SIP, are they encrypted? If you have any POTS or PRI trunks, are calls using those trunks also affected?

At the remote site, that is not correct – audio from the phones is tunneled through the VPN, so UDP traffic to/from port 1194 on the PBX should be given priority

At HQ, if you have non-VPN SIP trunks, you want to prioritize traffic to/from the trunking provider(s). I know nothing about how QoS works on your router, but note that the RTP ports are 10000-20000 on the PBX side, but could be anything on the provider’s side. Also, you want to prioritize auto sent to the remote site over the VPN. If the router can do it, it may be easiest to prioritize all UDP to/from the PBX.

I have only heard of issues at the remote location.

When there is jumbled audio it is heard by both parties on the call. It usually happens for the entirety of the call, but sporadically. It may be good for 10 seconds, bad for 2 seconds, good for 20 seconds, bad for 10 seconds, etc.

I’m not aware of a time when we have had the issue when more than one call is happening.

I’m using the trunks from SIPStation.

I’ve changed my QoS settings to include port 1194. I’ll see if that helps.

Interesting. That makes many possibilities much less likely, as it’s rare to have both upstream and downstream issues at the same time. My first bet is the upstream at HQ is being saturated (by QoS not working correctly, or a problem on the connection). This would affect both packets sent to the extension and packets sent to the trunk, so both parties would hear the trouble. If the PBX is running virtual, another possibility is that the VM has insufficient resources.

You may be able to cause the trouble at will, which would make it easy to track down and fix. From your mobile, call someone at the branch. Have him play some music (from a smartphone, radio, etc.) into his desk phone. Listen carefully at your end and try to saturate the lines by running a speed test. If neither the download nor the upload test at HQ affects the sound, try running a test at the branch.

How many phones are at this remote location? Why are you running individual VPN sessions on the phones instead of a VPN session between the two locations? VPN’s add overhead so you have take that into consideration.

In my experience it’s usually bandwidth related (lack thereof, or latency… and QOS isn’t guaranteed when traversing public internet) OR a hardware issue (usually timing, specifically if you’re running in a VM… Azure/Hyper-V sucks for FPBX FYI; KVM based hypervisors are much better i’ve found). Of course there are other causes too, but most of the time it’s one of those.

Audio quality isn’t ever going to be perfect unless your phones, pbx and SIP trunks (termination) are all on the same network (or connected via direct links like MPLS) so you can control QOS/priority. The moment the public internet gets involved you lose control. Almost all our deployments are using public internet, and we rarely have complaints (there are call issues every once in a while, but it’s “reasonable”… certainly not “chronic”).

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.