Voice Quailty Issue / Jittering voice

General Help
#1

We are using Distro Free PBX version 2.9.09. We have hosted this PBX system on a reasonably fast server i.e Dell power edge 1950 host on a collocation site with dedicated 10Mbps connection burstable to 100mpbs. Objective was to sell VoIP Phone package to home and business users like VoIP companies such a Vonage and commwave.

For over a year we slowly increase our client base from 50 extensions to 170 extensions. From last 2 months we started experiencing voice quality issues. We are experiencing Jittering and crackly voice at random times.

From the research we did on the forums, the most possible cause given for jittering voice could be the bandwidth. We monitored the bandwidth and our peak usage at a given time was lower than 3mpbs, our dedicated bandwidth was much higher than that.

We tried to troubleshoot with the collocation support team they did some fine tuning bit were unable to help us further.

We then sourced another Collation provider and moved our PBX server to the new site but the Jittering is still there.

Other observation is that during business hours (peak hours) the jittering is much more and off-peak hours its much less.

This makes me feel, is this load related? Our system statistics range is as follows:
Processor:
Load Average: 0.41
Typicaly: CPU:1% - 26%

Memory
App Memory6%
Swap0%
Disks
/12%
/boot10%
/dev/shm0%

Networks
eth0 receive 65.31 KB/s - 150 KB/s
eth0 transmit 66.55 KB/s - 150 KB/s
Our Hardware overall I see the system is used at 10-20% on average.

And FreePBX Statistics range is typically as follows:
Total active calls: 2 - 25
Internal calls: 2 - 25
External calls: 2 - 25
Total active channels: 6 - 50
FreePBX Connections
IP Phones Online: 168
IP Trunks Online: 11

Two Month ago when we probable has 100 minus extension we never experience jitteriness.
Q1: What I want to know how scalable is free pbx. What is the loads level it can typically handle? Can this platform be used to become a phone service provide over WAN link to approximately 1000+ extensions with 25% concurrent live calls? Or it is designed for small medium size companies to handle 25-50 extension over a LAN link?

Q2: From the scenario explained above what could be the most probable case of jittering

Experience advise is desperately needed.

Thanks

#2

Bandwidth is just part of the equation, jitter is much more important.

Certainly Asterisk can scale to what you want. FreePBX really has little to do with that side of it.

The big deal is that FreePBX is not a service provider softswitch, it was never designed for this application and does not have the facilities to handle multi-tenancy.

Also, if in the US how are you addressing CALEA and other legal requirements of a carrier that FreePBX provides no facilities for?

As a legitimate carrier, who pays all my fees and invests in a carrier class infrastructure I have a real problem with tossing a system you know little about in a colo and calling yourself a phone company.

#3

Thank you for your response.

Can you please elaborate on the following?

  1. If Asterisk can scale to any size, FreePbx is using asterisk also in the background, why FreePBX is not saleable like Asitrix. Do you mean that I should install Asterisk itself rather than free PBX?

  2. We were using FreeBBX from over a year as Multi-tenancy and until we reach 100 extensions, we did not have any voice quality issue.

  3. Can FreePBX be tweaked to be optimized for voice crackling or jitteriness? We can use paid support for this if possible.

  4. If FreePBX is not suitable for multi-tenancy, can you recommend another system that is.

You response will be greatly appreciated.

#4

FreePBX is not and has never been designed for Multi Tenanat. FreePBX uses Asterisk and we provide support to customers running 1000’s of extensions on a single box. The issue you are describing seems to be a network congestion issue.

#5

Since you neatly evaded my question about CALEA and other regulatory issues I can only assume you are not playing by the rules and I can’t participate in assisting you.

As far as the multi-tennancy, for other users reading this thread. Here are my main gripes:

1 - Call forwarding is insecure
2 - All users can dial each others extensions
3 - Queue users can log into each others queues
4 - Channel Spy works across all channels

A2billing runs on Asterisk and is designed to provide line replacement, calling card, and trunking services. The folks at star2billing get about a grand to install it. The software itself is free. I would recommend professional install.

#6

I did not mean to avoid your question on CALEA. We are a Canadian based company and will certainly look in to al legalities and would abide by the regulations.

As mentioned earlier we are try to become a service provider and would like to do all it take to be a good one. If we don’t follow the regulation we will not be able to continue or grow.

Anyway my current problem is some unhappy clients who are going through jittering/cracking sound problem.

I need to get that fix ASAP and then I will research on CALEA or equivalent Canadian regulation.

Can you please recommend us a multi-tennant Soft Swith that full fills CALEA regulation and is suitable for a small and medium size phone service providers?

We thank you for your advise in the right direction.

#7

I’d double check all of my routers/switches to make sure there is no problem such as a faulty port or misconfiguration. I know you’ve already checked this, but could there be a QOS misconfiguration that is giving some other traffic priority over VOIP?

Finally, make sure you are really getting the thruput the provider claims. Remember, he is guaranteeing that at HIS demark, he usually can’t control what is happening between his demark and yours.

Just a few random thoughts.

Bill

#8

Thank you for your response Tony.

I also feel it is a network congestion issue. As I monitor the system during peak hours and at time when voice is crackling, we are never over 25-30 concurrent calls and the voice crackles.

How many concurrent call can FreePBX handle in a given time?

The hardware statistics shows that the overall system used is 20% of Capacity, The network bandwidth usage shows that we are using less than 3 Mpps during over peak hour. Our Colo provider provides us speed burstable to 100Mbps

After trying to troubleshooting this with the Colo provider we moved to a different provider hoping that it will resolve the issue but we had NO luck.

Please advise what we shall do in other to troubleshoot this further.
Thanks

#9

Thank you for your response

I have tried the following:
I have cloned the image of the server hosting the PBX and restored it on identical hardware. To ensure it not due to a hardware issue.

I have removed the router/firewall and connected the PBX server to the main feed at the collocation rack.

I have connected a windows based machine to the router and ran utilities for speed test of the provider. And all result was good no packet loss etc, speed through put reports 50+ MB + downlink and 30+ MB upload.

All the above has NOT help.

The voice does not crackle all the time. It normally crackles during peak hour’s i.e normally during business ours when more concurrent calls are live. When the call load are below 10 the quality seems to be ok, when it above 10+call the quality is too poor.

Any further advise will be appreciated.

#10

Probably all of the tests you are running utilize TCP while SIP uses UDP. Wonder if there could be some inability of the network or your provider in handling UDP?

Bill

#11

Is this a virtual server with FreePBX. When you say you cloned it that tells me you are virtualized.

#12

I will run test using UDP and report the results soon.

#13

I have cloned an image using Ghost to another physical server. (Identical DELL server).

I have tried replacing all wires.

QoS is disabled on router.
I am going to remove the router and connect the server directly to the main feed for testing in next day or two.

Will report the result soon.

Please keep providing suggestion I will try all of them and I despite to resolve this.

#14

Have you tried changing the ethernet switch? Make sure all ports in signal path are at 100M Full Duplex.

#15

Hello Again,

2 hours ago, I have replaced the physical hardware (DELL Power Edge 1950). Therefore if this crackling had to do anything with memory leaks or NIC card or any other hardware component, it should make a difference. The software is the same as I cloned the hard drive from an identical hardware server. (Please note we are NOT using any Virtual machines).

For testing purpose, I have also removed the ethernet switch and router. I have connected the main collocation feed to the FreePBX eth0. If the layer 2/3 device was having an issue it should also resolve.

As crackling occurs at random times mostly during business hours, I cannot confirm the results so far. I will observe this for next 48 hours and will report the results to this threat.

Thanks

#16

After removing the router and the switch from the loop and connecting the freePBX to the direct feed the quality was improved tremendously. It must be a bad router or switch. I shall replace them 1 at a time to isolate the device that caused this problem.

So the conclusion is that either we have a fault router or fault switch.
However I have not replaced the new router yet. I can’t use the direct feed for long as there are other devices that need internet connection. I will have to replace with another router soon.
I hope the problem does not return once a new router and switch will be placed back.

I shall report the results once done for other readers.

Thank you all for their contribution to this thread.

#17

Attique,

As W5waf stated earlier you might have a QoS issue. Quality of Service is a huge part of Voip and is definitly worth looking at.

If you don’t have your switches or router looking at and giving precedence to your Voip traffic then swapping them out might not change anything. What kind of switches and router do you have? When you removed the switch and the router you also removed all the other traffic that they are dealing with.

Start by looking into having Freepbx tag the traffic to allow your switch/router to prioritize it. Then turn it on on your router.

Pathos

#18