Trouble with high call volume

I am having trouble when my call volume gets much over 400 calls a minute, the users get indication but can’t answer, or if they can answer there is no audio, I have done everything I can think of to limit the inbound on the numbers I’m concerned with, but haven’t gotten anywhere.

It appears as if all the inbound requests get assigned an RTP port until I’m out. Is there any way to prevent that until the call is answered or a different way of returning busy that doesn’t require as much overhead.

My provider used to limit inbound per DID, but discontinued that service, also in spite of the fact that I only have 80 concurrent they will send thousands of requests, and I’ve exhausted my knowledge trying to find a way around it.

Is your hardware under powered to support your peak call volume?

It’s a Dell Poweredge R530 with 128G of RAM. If it is I can’t afford to do better.

You can use the dashboard widgets or the htop command (in ssh) to look at your resource utilization.

You could also increase your RTP port range beyond the default (10000-20000) if you need more.

Look at the full log to see what kind of errors you are getting during the issue. That might help narrow down what needs to change.

Run htop and see what your Load average is, as well as cpu, ram, and swap useage. I’ve seen some strange things on freepbx with high call volumes.

these events are so transient that htop is going to be problematic…

Which log will I find load statistics in?

Just run htop from your ssh client and it will show load average in the top right corner of the terminal.

I get that but I would have to wait for tomorrows morning shows to see anything. Is there any historical, or a debug I could turn on to collect it?

@tteagarden
Well if you are truly getting 400 calls per minute that means you’re getting 6.6 calls per second. Are you 100% sure all these calls are from your provider? That’s a lot of calls in a second. You don’t have something like Guest/Anonymous Calling turned on do you?

Oh I get way more than that, there’s 6 radio stations in this building, 400 a minute is just where it starts to have problems.
CenturyLink used to limit it for me, but Embarq removed a lot of those features when they took over

Look at /var/log/asterisk for the full logs and review the logs during the challenge period(s). Look for orange/red lines specifically and see if anything stands out. HTOP during an even will fully rule out resource constraints (which it doesn’t seem like is the issue)

Also, with an 888-day uptime, you might want to look into a reboot at some point in the near future if at all possible. Although it is a shame to break the streak, it can be good to periodically do.

install and enable sysstat, it’s sar reporting gives you granular reporting of system resource usage

2 Likes

Got sysstat, up other than a minor snafu of where it went after I yummed, it it looks like just what I need. Giving away Beyonce tickets here in 2 hours… That will give me a good idea where it’s melting down.

My experience has been to ask you if you are using toll-free numbers for origination?
But by any means , absolutely make sure the provider is not rate-limiting your calls in any fashion, many have special treatment for such sporadic but heavy call volume numbers.

I have toll fees, all inbound, and not the issue. My provider recently changed because of asset swaps between telcos. I used to have all the provisioning I needed to make this work right, but the new company took away several features. Trunk limiting was provided, and now it’s not, geo choking was too, but not any longer, so now the same 5 people will get through all the time…Telcos ain’t what they used to be.

The toll free resp-org needs to be paid for concurrent channels and the PSTN endpoint needs to accept that traffic unimpeded , check with both ‘as to’.

You’re saying that you were getting 400+ calls a minute but CenturyLink was only limiting you to 80 channels? So you had over 80%+ of your call volume rejected until recently?

You are confusing call attempt rates with occupancy. Although calls would have to be extremely short to get 100% occupancy with 80 erlangs capacity and >400 busy hour call attempts.

No, I’m not. The claim was that CenturyLink had them limited to 80 channels. That would include attempts and connected calls. I in no way was referring to CPS in that question. My original point was in order to have 400 calls in a 60 second period it would equate a CPS of 6.6 calls per second.

However, 400 calls per minute with an 80 call limit on the trunk means only 80 calls can be in use at any given time (regardless of CPS, eh). So if we start at 00:00 with a 7 CPS (rounded up) and every call is answered by the system that means by 00:11 seconds 80 calls will have been hit. That is assuming that across 6 radio stations the sales, marketing, promotion teams aren’t using any voice channels themselves for calls and this is all listeners calling in. Does this mean for the full minute every call beyond that 80 is rejected? No, it doesn’t. It means that for a time period there will be rejected calls until some of those 80 existing calls drop off.

I guess the real question here is how is the service being paid for? Generally, a channel limit is imposted when there are X channels being purchased. However, if you are paying per minute per call they generally don’t limit your channels because they want that per minute per call rating.

With very peaky traffic they may well want to protect their supply side capacity, so as not to penalise other users.

Historically, I believe, call in programmes have been handled specially by phone networks in order to protect the network.

Also, on call durations, there is a guard period, of, I think, about 30 seconds after a SIP call ends, to allow for retransmissions, it is possible that occupancy limits are calculated including this period.