I wasn’t referring to the GUI, the GUI is quite a bit faster in 14. Just watching htop while not touching the GUI at all with the UCP off. Doing a reload from fwconsole reload causes the load average to spike far more than it used to on 13.
Performing reloads is starting to cause a lot of worry due to the spikes.
The cross referenced post still applies.
So we found something that was causing CPU spikes on a schedule. It also seems that it is having a consistent error over and over and over and over:
Bad File Mode
Checking the /etc/cron.d/ folder we had this:
It had this in the file:
# This runs the update check every night at midnight. It sleeps for up to an hour
# as a random delay.
#
# This file was installed by the 'sangoma-pbx' package.
#
0 0 * * * root [ -e /etc/profile.d/z001-updates.sh ] && /etc/profile.d/z001-updates.sh update
Isn’t only the root user suppose to be in the etc/cron.d/ directory?
Change the ownership of the file if you think that is what’s causing your systemwide issues (I doubt it is)
Whoops, yes, that’s a bug. That REALLY shouldn’t be owned by the asterisk user! Thanks very much for picking that up!
This is now fixed in sangoma-pbx-1802-2, just run ‘yum update’ and it’ll pick it up. (You may need to ‘yum clean metadata’ first, if it doesn’t pick it up as available). Thanks!
^ 
found another security bug, thanks for fixing that.
We found a way to make asterisk run better during normal operations and through reloads, although reload still cause issues at times. We used HTOP to decrease the nice setting to -10 on the main asterisk thread and that made it run MUCH better and stopped some of the call quality issues plus bought us time to look into this more.
It seems that the more calls asterisk has processed without restarting the worse it’s performance gets. If we let it get to around 60,000 calls processed we will hear small sound issues and around 100,000 we can notice sound issues on customers that should have no sound issues. Upon an fwconsole restart everything sounds great again.
Reloads also follow a similar pattern and progressively use more resources the more calls that have been processed but it has an odd little pattern that I am not sure what to make of:
We will apply changes or fwconsole reload.
I will watch asterisk be fed the new dial plan.
Load jumps to around 7-10 in HTOP
We then get informed that the reload is done.
Load then spikes to 10-24 on a 12 core server. It progressively rises and hits a very high load average then drops.
I would expect the largest spike to be the asterisk dialplan switch over but a large spike continues on after the dialplan switch over happens. What happens after fwconsole reload says it has completed?
Rest Apps is restarted.
Does anything else happen? I stopped restapps from PM2 and reloaded.
While Fwconsole reload was running load average maxed at 9.46 then once Fwconsole was done it shot up quickly shooting up and went to 17 then held there for about a minute. 36 calls active 89425 calls processed.
Call quality went to shit even when it was at 9.46. Have 12 cores and none of them were maxed according to htop but the main asterisk thread was at 200% cpu despite the CPU cores not reflecting that individually.
Asterisk will only basically use 1 thread. So having more than 1 core doesn’t help the asterisk side of things.
So the main asterisk thread will only use 1 thread. Am I correct in thinking that the child threads of the main thread can then use whatever core the kernal assigns it?
Hi,
I have just tried to transition from FreePBX 13 to FreePBX 14.
This would not do an upgrade in place so I did a clean install and followed the migration steps, which worked for the most part, a few tweeks and a missing trunk, but it worked.
Problem is that I have excessive load in general, let alone when something is happening… Initially I thought it was just re-configuring as a background process so I muddled with it until it is still the same after 24 hours.
About this time I loose web GUI connections as it is all running so slow.
Rebooting the server is not making things any better, a good 6 mins to get asterisk running, and more before I can get a GUI connection.
My setup is nothing like what you would expect to be causing load. This is just running in my home for personal and home office lines.
I have a total of five extensions and four trunks.
Most of the time the server is sitting doing nothing and I have a load ranging from 1.2 up to over 30.0 on a 5 min average.
When it takes off everything grinds, I am able to login to the console after about 45 seconds and then it is slow to respond when in the shell.
When the load picks up, top shows several php scripts running.
I moved back to the FreePBX13 that was running and the load at idle remains between 0.02 and 0.06, most of that is from the top application itself.
The environment is a ProxMox VM. Both have the same config:
two cores on one CPU socket
2Gb RAM
20GB storage.
With FreePBX13 running, my ProxMox host is running at a 5min load average of 0.5 - 0.8
The FreePBX14 server is the latest ISO image and when it installed I performed an OS update.
I have removed most modules I will not use or have no license for and the load did not change.
Is there a fix I am missing? Not much came up under searching for excessive load and the like.
Can anyone offer a solution?
Thanks
Paul Budden
Nope. There is no solution.
I’ve done micro-improvements here and there that have lessened the load (Removing old stuff, condensing other stuff, making sure recordings and music on hold have proper file extensions, switching extensions over to PJSIP) but no real solution.
After cleaning up a lot of stuff I have noticed that most of the normal non-reload spikes come from new inbound calls or calls that are being hung up. When you reload the spikes come from BLF/Hint updates.
Have you found an answer to that question?
Nope.
Yes but the code section has to be coded for multi-threading. EG PJSIP is multi-threaded. chan_sip is not. Other parts of asterisk are other parts are not.
That would explain why we are showing performance gains with PJSIP. We have tons of mostly idle cores.
Tangent but related: Another fun one. Parking lots have a max integer value defined. So you can’t have a parking lot number that is longer than the hard coded length. Regardless of your CPU type (32 v 64)
Point is it depends on who is writing the code for what in Asterisk. As with open source projects everyone codes a bit differently. Parking lots have that hard coded length because someone thought that was a good idea at the time. FreePBX has plenty of this too… however new features are often coded as ‘multi-threaded’ as opposed to the old code in Asterisk
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.