Disk or Database Bottleneck

Thanks. on 18.17.1 now. We will see what happens.

Not entirely sure what I need, everything looked okay (memory, cpu, network, channels, database, io, space). My only clue is I started seeing Exceptionally long voice queue length queuing to Local/1XXXXXXXXXX@from-internal

These are very tough to debug and fix.

Yeah, well while the calls seem to be flowing fine right now. I did a quick check this morning, and I still see a few of these errors. Maybe I will try Asterisk 20 before going the backtrace route.

Well that did not last long. System slowness is back. I’ll work on the backtrace now.

I think you’re generating a ton of CEL because your queues are busy and are dialing out to remote agents across a trunk. If you don’t use CEL, disable it. CEL is a database hog with busy queues

edit: I just noticed that you participated in that thread so you probably already considered this. Leaving it here for the record :slight_smile:

1 Like

I read your earlier post on CEL and proactively disabled that a couple weeks back :slight_smile: (Thank you)

I am on CEL 16.0.16 & Advanced settings > CEL Report Module > Enable CEL Reporting = No

The box is not running a lot of traffic currently 30ish channels and 17 calls up. Resource utilization looks next to nothing.

Another error I am seeing every few seconds is

[2023-06-07 12:36:24] NOTICE[28816]: manager.c:3653 authenticate: 127.0.0.1 failed to authenticate as 'admin'

Is there a way to see where this is coming from? I do use admin for Asternic Call Center Stats, but that looks like it is working (I can see realtime and historical data).

That likely isn’t related to the failure but some request is failing from some system.

1 Like

Did you upgrade to 20.3.0 or 20.2.1?

20.2.1

Yeah, I think Asternic was caught in the slowdown. Apply config is taking 8+ minutes, every call has the message now, and calls are failing.

Reading through the backtrace to see how to do it now.

Question about backtrace Getting a Backtrace - Asterisk Project - Asterisk Project Wiki

I try running sudo /var/lib/asterisk/scripts/ast_coredumper core but all it says is No coredumps found I am missing something.

Got it but not sure what to do with it.
Creating /tmp/core-asterisk-running-2023-06-07T21-56-16Z-thread1.txt
Creating /tmp/core-asterisk-running-2023-06-07T21-56-16Z-brief.txt
Creating /tmp/core-asterisk-running-2023-06-07T21-56-16Z-full.txt
Creating /tmp/core-asterisk-running-2023-06-07T21-56-16Z-locks.txt
Creating /tmp/core-asterisk-running-2023-06-07T21-56-16Z-info.txt

look in /var/log/syslog as to why/if asterisk is crashing.

to watch those files , install ‘multitail’ (yum/apt install multitail) and then

multitail /tmp/core-asterisk*.txt
1 Like

I have no files in /var/log/syslog that are core-asterisk*.txt

This aligns with my experience, as the system has never crashed (yet), only grinded to the speed of a snail.

I had a spare FreePBX15/Asterisk 16, I am migrating users to right now while I get things figured out.

/var/log/syslog (perhaps /var/log/messages) is a system log file not a directory, if there is a reference to asterisk in any way, you can ‘grep it out’ (there will be)

1 Like

You’ll need to post them some where we can get to them. I’m trying to see if I can get someone that can read these better than me to take a look but we’ll need access to those files.

1 Like

Brief: https://privatebin.net/?1e99499390c18451#A6mhwNCDbw5FLtq8JGJiVgePjrY7rqDD4J4pN5iwLt9W

Full: PrivateBin

Locks: PrivateBin

Thread1: PrivateBin

That was it. grep asterisk messages