We are having load spikes on our FreePBX 15 distro we switched to about a week and a half ago that I have yet to be able to get fully resolved.
I’m at the point now that I’m strongly considering disabling CEL(I never view those logs anyways). After this system being in production for 1 and 1/2 weeks we already have 1,143,409 records in the asteriskcdrdb.cel table. Looks like I need to change a .conf file manually, then I’m guessing restart asterisk to get it to stop logging?
Furthermore, we are seeing taskprocessor.c: The “‘stasis/p:channel:all-0000c3c8’ task processor queue reached 500 scheduled tasks” Warning as much as 15 times per minute in the logs. Also with the stasis/m:cache_pattern:1/channel:all-000003f7 thrown into the mix as well.
We’ve already tried going through ALL of our extensions and changing the “MWI Subscription Type” to “Solicited”. Not sure if that helped or not, but we are still getting the “task processor queue reached 500” what feels like a lot.
There is a pending update we have for the “core” module that appears to change the “taskprocessor_overload_trigger=pjsip_only” settings, which I’m terrified of what that might do for our situation.
We have about 350 active phones, MANY of which are in groups of 5-10 “queues” that are “ring all”.
Please let me know your thoughts on what we should do to try to get those warnings fixed / resolved.
Unless someone has a better idea / suggestion, tonight I’m going to try to disable Call Event Logging when I’m able to restart asterisk.
I’m clueless as to what is hammering that stasis/p:channel:all so hard. Looks like to get the commands to see the details on stasis you have to manually compile asterisk with developer mode or something along those lines.(I don’t see us being able to do that on our production server)
We do still have the majority of our digium phones running on the much older firmware 2.2.2.2. We’ve been slowly upgrading the phones incase the newer firmware might cause any issues. (perhaps the old firmware is causing issues?)
Any help or suggestions on this issue would be greatly appreciated.
A taskprocessor is a serialized work queue. You put a work item in, and then it gets handled. Internally they’re used all over the place. Overloaded means that the rate at which work items are being queued exceeds the rate at which they are being processed and handled. The channel:all receives all the messages relating to every channel in the system, and is used by things that want to know what is going on. Ring all will certainly produce a ton of work items. I always think of ring all as essentially a DoS attack. Old firmware wouldn’t cause an issue, purely depends on the underlying system characteristics and usage patterns. We’ve tweaked things here and there in Asterisk to try to help speed up processing of work items in certain scenarios, but there are still some that just hammer things.
Thank you for your response. Do you think turning off CEL is a good first step? Is there an alternative configuration we should consider using instead of a “ring all” for our 36 locations that would give similar functionality? All employees at our locations pitch in answering the phones regardless of where they are, be it the sales counter, warehouse, etc. That is why we use the ring all, so they can know the phone is ringing from any phone in the building.
CEL will alter the conditions of the system, I can’t really say anything further on it. That’s the thing about this kind of stuff - without digging in, reproducing precise environments and usage patterns, it’s all surface level guessing and answers. I don’t really have a suggestion on ring all alternatives and such.