Intermittent one-way audio

General Help
1

Hello! I have a number of FreePBX installations (various versions) hosted in a datacenter, and they are experiencing a very annoying and persistent bug.

Intermittently - multiple times a day, but not on any particular schedule - calls will briefly have one way audio. The ‘direction’ of the one way audio seems to be random - sometimes I can hear the caller but they can’t hear me, and other times it is reversed. Typically after 15 seconds or so, two way audio resumes.

I have been working under the assumption this is some kind of interruption in the RTP stream, so I had rtp debug enabled for my IP, and managed to catch one of these events in action- here is my RTP debug log from just before, during, and after:

[2016-06-23 14:18:02] VERBOSE[23567][C-0000005b] res_rtp_asterisk.c: Sent RTP packet to 11.22.33.44:2232 (type 18, seq 040829, ts 3502718872, len 000020)
[2016-06-23 14:18:02] VERBOSE[23567][C-0000005b] res_rtp_asterisk.c: Got RTP packet from 11.22.33.44:2232 (type 18, seq 035046, ts 2193506874, len 000020)
[2016-06-23 14:18:02] VERBOSE[23567][C-0000005b] res_rtp_asterisk.c: Sent RTP packet to 11.22.33.44:2232 (type 18, seq 040830, ts 3502719032, len 000020)
[2016-06-23 14:18:02] VERBOSE[23567][C-0000005b] res_rtp_asterisk.c: Sent RTP packet to 11.22.33.44:2232 (type 18, seq 040831, ts 3502719192, len 000020)
[2016-06-23 14:18:06] VERBOSE[22457] res_rtp_asterisk.c: Sent Comfort Noise RTP packet to 11.22.33.44:2232 (type 02, seq 040832, ts 3502719192, len 000001)
[2016-06-23 14:18:10] VERBOSE[22457] res_rtp_asterisk.c: Sent Comfort Noise RTP packet to 11.22.33.44:2232 (type 02, seq 040833, ts 3502719192, len 000001)
[2016-06-23 14:18:14] VERBOSE[22457] res_rtp_asterisk.c: Sent Comfort Noise RTP packet to 11.22.33.44:2232 (type 02, seq 040834, ts 3502719192, len 000001)
[2016-06-23 14:18:18] VERBOSE[22457] res_rtp_asterisk.c: Sent Comfort Noise RTP packet to 11.22.33.44:2232 (type 02, seq 040835, ts 3502719192, len 000001)
[2016-06-23 14:18:22] VERBOSE[22457] res_rtp_asterisk.c: Sent Comfort Noise RTP packet to 11.22.33.44:2232 (type 02, seq 040836, ts 3502719192, len 000001)
[2016-06-23 14:18:26] VERBOSE[22457] res_rtp_asterisk.c: Sent Comfort Noise RTP packet to 11.22.33.44:2232 (type 02, seq 040837, ts 3502719192, len 000001)
[2016-06-23 14:18:27] VERBOSE[23567][C-0000005b] res_rtp_asterisk.c: Got RTP packet from 11.22.33.44:2232 (type 18, seq 036298, ts 2193707194, len 000020)
[2016-06-23 14:18:27] VERBOSE[23567][C-0000005b] res_rtp_asterisk.c: Sent RTP packet to 11.22.33.44:2232 (type 18, seq 040838, ts 3502919352, len 000020)
[2016-06-23 14:18:27] VERBOSE[23567][C-0000005b] res_rtp_asterisk.c: Got RTP packet from 11.22.33.44:2232 (type 18, seq 036299, ts 2193707354, len 000020)
[2016-06-23 14:18:27] VERBOSE[23567][C-0000005b] res_rtp_asterisk.c: Sent RTP packet to 11.22.33.44:2232 (type 18, seq 040839, ts 3502919512, len 000020)

During the bits in the middle about the comfort noise packets, I was receiving no audio on my Polycom handset. The person on the other end of the call (an internal extension, not someone across the PSTN) could still hear me he says.

Some background on my exact setup:
FreePBX 2.11.0.43 / Asterisk 11.5.1 (This issue also happens on 12.0.76.2 / 11.20.0, and other versions)
The system is hosted on VMware; we have moved the VMs from one host to another, no effect.
The PBX has two NICs, one of which has a direct WAN IP with no firewall other than iptables on the box itself
The internet circuit in use for SIP traffic is dedicated for that purpose and has no other traffic on it. It is a 20Mbps circuit running at approximately 10% utilization.
The Polycom endpoints are WAN attached and not local to the PBX

I don’t know what else to provide information wise. I’m looking for someplace to start. Thanks!

2

Ah, an update. It’s possible the other caller was mistaken on the one-way audio, it’s possible that audio simply cut out completely for those seconds. It happened again, and this time the remote extension reported he was unable to hear me.

3

Something suspicious here:-

See that you posted 11.22.33.44:2232 (IP address:rdp port) for both directions with different sequence numbers, is that exactly what you saw?

4

I took out my own IP and replace it with 11.22.33.44, but mine was the only IP listed. It’s the only IP I had rtp debug enabled on though, which explains why it was the only one in the log.

5

I know how to read rtp debug . Yours shows yourself tslking to yourself on the same port with two different sequences , except for about 20 secs when you obviously stopped talking to yourself, This is generally not what one expect to see. Perhaps you could double check your editing?

6

If there is some other kind of capture I should perform or logs I should be looking through, I’m all ears. I’m positive that the log I posted is accurate; my IP was replaced with the obfuscated IP via find and replace, not by hand. I assume the two-way rtp stream is for sending and receiving audio from my phone - one stream per direction - but I could be wrong. I just don’t have any indication at all of why it stopped for 25 seconds.

7

Pretty well always a network problem , I suggest tcpdump or wireshark if you understand how to set it up. Possibly a udp connection timeout misconfiguration.