Intermittent 'Static' in Calls

I posted about this a couple of months ago - a link to the original thread is below. Since, I’ve upgraded the phone firmware, dug into the network side of things, shortened the route to our SIP provider, and asked them to look at their systems for any clues. None have resulted in any improvement or isolated the problem.

In short, there is VERY intermittent ‘static’ in calls, but only the caller seems to be able to hear it when it happens. My hunch is that it’s digitization of some kind, but am going to use ‘static’ to describe it in this post as that’s how it’s been described by our customers. It’s only been reported by our operators for customers who call in; all have Sangoma s500 phones. None of our other users with different brand/model phones have reported it, though this could be coincidence as the operators handle MANY more incoming calls than anyone with a regular DID. The call recordings are clean, which eliminates such things as the handset cord, ‘dirty’ PoE, & other problems between our user and the PBX as then the recording would have the static. It has been reported with customers on cell phones as well as those with land lines.

The primary problem I have in tracking it down is that it’s SO intermittent - we may have one call a week where the customers can’t hear us or we might have two or three inside a 10 minute window. Once a call has the ‘static’, it seems to keep it until the call is terminated. If a customer hangs up and immediately calls back, the ‘static’ is gone. There is no pattern I’ve been able to discern; I’ve checked day of the week, time of day, and wrote a script to track the number of calls active. Doing a packet capture really wouldn’t be feasible given how randomly it occurs.

The only other idea I had was to write something to record how much bandwidth is being used in set intervals and then compare the arrival of the ‘static’ calls with this log, but this is going to be considerably more involved to get it right and frankly, I don’t know if it makes sense. Given that a call keeps the ‘static’ for the duration of the call and other calls can come in clean while the call with the ‘static’ is active, it doesn’t seem to be a limited bandwidth / line saturation problem. Same thing with tracking CPU usage on the PBX - it wouldn’t stay on a call until terminated and would affect all calls at the same time if this were the source.

Is there anything else anyone can think of that I could do to track down & fix the source of this ‘static’? I’m really scratching my head and it’s unbelievably frustrating to not be able to reproduce or even hear it, instead having to rely on our customers to describe the sound. It’s not a huge deal given how infrequently it happens, but is a considerable annoyance to the users & worrisome to me as an indication something isn’t right. Or is this type of thing considered normal for a PBXact-based phone system?

If you have the disk space, install pcapsipdump , it records every sip ‘conversation’ in seperately dated and tagged files with src and dest numbers and also organised in directories by date and hour.

It conveniently can delete calls after n days, so you can leave it running un monitored .

When you get a complaint, get the time and the participants, there will likely be one or two files that contain the legs involved in each call and the tx/rx flows are stereophonically seperatedbin each file
Grab the files and throw them at wire shark for further analysis.

You probably don’t want most ‘noise’ of registrations and options so add the optional ’ -m INVITE’ . You have now got seperate flows to help you discover the exactly source of the ‘artifacts’.

It’s pretty cheap on resources needed but on a raspberry it might make your ‘static’ worse :wink: .

I’ve never heard of such a thing - thank you @dicko ! Sounds like the perfect tool for this type of issue. It’ll be interesting comparing a normal call with one that has ‘static’ in it… been completely unable to detect anything so far.

This is probably going to depend on the number of calls, but how much disk space are we talking about it using? Would using a big flash drive be acceptable or should it really go to the hard drive?

My vacation is starting after tomorrow, but will play with that program after I return in the unlikely event it causes problems. :slight_smile:

160 kb/s per g711 call, you do the math :wink:

Unless there is a setting to avoid it, there would be twice that (trunk to PBX, PBX to extension, extension to PBX, PBX to trunk, ~80 kb/s each).

My math, one conversation = 2 (bridged) calls , one call = 80kbits each way, you do the “new math”

You will see the files in /var/spool/pcapsipdump//

They are raw 64kb streams with pcap headers.

You are correct, but I suspect that the OP considers a conversation between a customer and an agent to be one call, even though in Asterisk it is two, both of which would be captured – 320 kbps.

Hehe, but off topic he is more concerned with WTF than pedantry, pragmatically if he has a couple of gig spare and less than 100 users, he is fine with a 7 day rollover.

I’m with @dicko - check pcaps first. You can tell at the very least if the static is localized to a phone, FreePBX or if it’s coming from something upstream.

Matthew Fredrickson

2 Likes

Just for the record here, VoIP/SIP is Full-Duplex like a telephone/PRI line. Meaning that you can send and receive audio and talk over each other. So the 87.6Kbps (per Cisco’s chart) is for the audio at full-duplex. So it’s not Phone A is sending audio 87.6Kpbs and Phone B is doing the same. The audio is streaming two-ways and that consumes the bandwidth.

That would mean if the PBX is in the cloud and you have an extension to extension call and both phones are at the same location, roughly 162Kbps will be needed for that call at that location and the PBX connection (each). Now if the call was PSTN to extension then the PBX bandwidth will still need 162Kbps (both sides) but the phone location would only consume the 87.6Kbps for that side of the call.

Just think about it, a PRI uses g711 and has 23 x 64Kbps channels doing the audio in full-duplex which means 1 call with 2-way audio is 64Kbps. Same with SIP the there’s just about 20Kbps of IP overhead added.

If disk space is a concern, use plain tcpdump as described in

You’ll know when you start it how much space will be used. If you have more traffic than expected, you’ll have less time to get the data before it’s overwritten, but that shouldn’t be an issue if the problem is promptly reported. You can of course adjust the command parameters to control how much space will be used.

Having all the network traffic in one place may make it easier to determine the cause, e.g. it may be related to an attempted attack, software update, high jitter on several concurrent calls, etc.

Please guys , stop with the "off topic " stuff . . . .

If anyone has anything useful to the OP, please post it . . . . .

IT is so effing easy, record the calls with pcapsipdump, analyse them with wireshark, Identify the source of the artifacts. . . .

Hopefully case closed, if you want to play clever with bits per second and whatever then do so , I just don’t think all this “clever” BS is helping, If you have ever actually used pcapsipdump, then you might get a brownie point for pertinent ! :wink:

Well honestly, it would be helpful if the OP gave an update on what happened from the original thread they opened (and was closed for no activity after a while) because in that thread we went over all this and the OP discovered that the only people having this issue where using the S505 series Sangoma phones. Something that wasn’t brought up in this thread.

The OP had updated those phones with the latest firmware as of Oct 7th and was supposed to monitor and update the thread with the results. Instead the thread was closed for lack of activity and two months later this thread was started with a link to the old thread.

@RealRuler2112 I’m going to take a wild guess and say the firmware update on the S505’s didn’t fix the problem? Is this still only happening with the S505’s? Was the phone that you said had a bad microphone and then the speakerphone broke on a S505 (from previous thread)?

The OP is on “Holiday” ( presumably a Limey) as of tomorrow

Let’s all actually STFU until then . . .

My vacation starts tomorrow and I’ll start playing with the phone system again after that. (I’d REALLY rather not be called in… :wink: )

Not a Brit @dicko , though I do enjoy drinking tea. :wink: :smiley: My extended time off is just an artifact of not having taken any time off all year so far - I ‘use it or lose it’ when it comes to vacation time and would rather use it, even though it’s now cold/snowy out. I GOTTA start taking time off in the summer when I can go drive my boat around and catch fish…

Actually, I enjoy the discussion of exactly how the system works, the bandwidth used, etc. The more I understand how the system works, the more likely it is that I can troubleshoot problems on my own. :slight_smile:

The first paragraph of my original post in this thread was basically a link & update to the other one @BlazeStudios … I listed all the stuff I’ve tried (including updating the firmware of the phones in question) and then reported that nothing changed. I originally went back to the original thread I posted a link to, but as you said, it had been closed due to inactivity. Getting our network & fiber providers to analyze/change the route traffic from my network takes to get to the SIP provider is not something that happens overnight; took mine over a month to get the network changed so that traffic took a more direct route to the data center… our latency dropped significantly, but monitoring proved this not to help this issue.

Plus, having only 7 days for a thread isn’t a whole lot of time when it comes to troubleshooting an issue that might only get reported once or twice a week. (Guess I could post a nonsense message every week saying they’re still working on it, though this would be annoying to everyone.) Maybe it’s my unfamiliarity with this forum software, but I couldn’t figure out how to re-open the original thread or I would have done so… best I could find was to link it in this one. Please let me know if there is a way to re-activate old threads that I’m missing. (This one is probably going to be closed due to inactivity as well due to my not wanting to waste my vacation time; knowing how to re-open them would be helpful. :slight_smile: )

Also, I apologize - I mis-reported the model phone in the original notes of the old thread, though corrected myself in subsequent posts. The users having this issue are not using S505 phones, but rather S500 phones. I did mention this in the second paragraph of my original notes of this thread. There was never anything mentioned about packet captures being an option for diagnostics in the old thread. The ‘static’ is still only being reported by people in our call center, who all have S500 phones. They obviously have a much higher call volume than the rest of the users & are therefore more likely to run into issues of this sort, though I’ve talked to a dozen others with both Sangoma & Cisco phones - everyone says the phones are solid. (The manager of our call center seems to enjoy coming up with stuff for me to work on, whether real problems or imagined, so problems in that department are much more likely to be reported.)

Not certain which problem you’re referring to @BlazeStudios . An S305 had the handset not work out of the box, but the hardware warranty was already up. Turned out to be a bad wiring harness inside the phone. Thankfully I was able to swap the harness between the handset & speakerphone (they never use speaker on this particular phone anyway) inside the phone and it fixed the problem. The send volume on all of our S500 phones is extremely low; I had to boost this by changing P249, P20083, and P20084 parameters within the basefile editor and this worked around that problem. I don’t see how either issue relates to this one, which is why I did not mention them.

Then there is your culprit. If you have a spare non S500 replace a S500 with it before you leave or after you get back. See if that user no longer gets the ‘static’ on their calls. If the issue goes away then that will confirm the S500’s are the core of the issue.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.