FreePBX | Register | Issues | Wiki | Portal | Support

Subm:ast_channel_topic_all warning - need some reassurance


(Jonnie) #1

Firstly, I know the log messages are warnings and if they’re not causing call issues can safely be ignored… however…

I have one 6 SIP trunks, 5 have negligible traffic, 1 is used for most inbound and all outbound calls.

FreePBX 14.0.3.13 running on Asterisk 15.5.0. The system is hosted on a VMware ESXi installation - the VM guest is configured with high latency sensitivity, a dedicated 4GB RAM and 2 CPU cores dedicated (3.06Ghz each). All resources reserved. The VM is hosted on a PCIe SSD storage device (although using VMDKs) with high share percentage configured.

There are 9 queues configured - although they don’t all receive callers at the same time. Six of the queues are used as a day/night endpoint for three parts of the company. Each of these 6 queues has one static agent configured for it - this static agent is another queue set to not answer which “fixes” the MoH from the first queue. These 3 queues then have, at most, 8 extensions as static agents.

When I make an external call into the system, I am receiving log messages like these:-

[2018-08-13 11:14:11] WARNING[13147][C-00000046] taskprocessor.c: The ‘subm:ast_channel_topic_all-cached-0000007b’ task processor queue reached 500 scheduled tasks again.
[2018-08-13 11:14:11] WARNING[13147][C-00000046] taskprocessor.c: The ‘subm:ast_channel_topic_all-00000084’ task processor queue reached 500 scheduled tasks again.
[2018-08-13 11:14:11] WARNING[13147][C-00000046] taskprocessor.c: The ‘subm:ast_channel_topic_all-0000007e’ task processor queue reached 500 scheduled tasks again.
[2018-08-13 11:18:33] WARNING[14229][C-00000047] taskprocessor.c: The ‘subm:ast_channel_topic_all-cached-0000007b’ task processor queue reached 500 scheduled tasks again.

The taskprocessor reports:-

subm:ast_channel_topic_all-0000007e 120935 0 586 450 500
subm:ast_channel_topic_all-00000084 120934 0 610 450 500
subm:ast_channel_topic_all-cached-0000007b 159626 0 764 450 500
subm:ast_channel_topic_all-cached-0000007c 159625 0 1058 450 500
subm:ast_channel_topic_all-cached-0000007d 159624 0 789 450 500
subm:manager_topic-00000007 233451 0 5162 2700 3000

I think I’d be less concerned if I could figure-out what the ast_channel_topic taskprocessor is being utilised for. The system only has 19 devices attached to it (using PJSIP) and 6 trunks (using CHANSIP). We hit a maximum of four simultaneous calls via the queues - and yet these stats seem to suggest the system is overloaded.

I tried building a new FreePBX install on an i7 NUC with 8GB RAM and an SSD at the weekend, restoring the config here and pointing DNS to it. After a couple of test calls, it too was showing these warnings.

I am concerned with some of the things I have read regarding taskprocessors which seem to indicate that PJSIP can “pause” responding if a queue size reaches the high water level - although the documentation doesn’t seem clear on whether this is true of ALL taskprocessor queues or only those specific to the PJSIP components.

Sorry for the brain dump - but to be honest, this is causing me sleepless nights. Just a FYI… last week we were having severe issues with the system. The VM was shutdown one evening to allow a copy to be made of it’s VMDK/VMX files etc as a kind of “super backup”. However, when the VM was powered-on, the system was not working properly. At random points in the day, when a call came in, the target handsets would start to ring but would never stop. When the system got into this state, I was unable to register new extensions and incoming calls on the existing trunks gave the busy tone to the external caller - the FreePBX logs showed no activity when these incoming calls were attempted. I assumed that something which had been configured in the week since the last system reboot was responsible and started “backing-out” changes (call recording enabling etc) but was unable to get the system operational for more than a week. In desperation this weekend, I created a new VM, re-installed FreePBX and restored the configuration. Since this time, no more weird “ghost” calls or busy incoming trunks… but these warning are freaking me out a little as I have never seen them on previous systems.

In short HELP - technical or psychiatric…


How can I flush call state from the astdb? Is there a CLI command for that?
#2

I have the exact same issues and questions.
My freepbx machine (4 cores 16GB on a xen virtual server) started failing every 20 minutes (all endpoints unregistered, couldnt register, no incoming or outgoing calls…all endpoints are pjsip)
The only thing I could see was these taskprocesor warnings.
I uninstalled, then removed the commercial “queues pro” module (the only thing I had changed in the past week was activating that module)…now I get those warnings at 500 and 3000 but if I immediately try to view the taskprocessors, the in queue column is all at or very near zero.

some things like cel_aggreation topic show max of over 200,000 but are at zero now…what can cause this?
thank.


(ADTopkek) #3

Ours throws errors often and shows:

[2018-08-24 15:48:54] WARNING[20848][C-00001624]: taskprocessor.c:893 taskprocessor_push: The 'subm:manager_topic-00000007' task processor queue reached 3000 scheduled tasks again. [2018-08-24 15:48:54] WARNING[20849][C-00001624]: taskprocessor.c:893 taskprocessor_push: The 'subm:ast_channel_topic_all-cached-000013c6' task processor queue reached 500 scheduled tasks again.

Get this afterwards sometimes too:
[2018-08-24 15:49:04] WARNING[21184][C-00001620]: channel.c:1136 __ast_queue_frame: Exceptionally long voice queue length queuing to Local/LC-8201@from-internal-00000db8;2

subm:manager_topic-00000007 29040613 0 7339 2700 3000
subm:ast_device_state_topic-00000004 1202667 0 6408 450 500

It’s becoming a real problem.


(Milauria) #4

Anybody got any clue on possible root causes and or fixes ?
I have the same on subm:ast_channel_topic_al often reaching the 500 limit