PBX slow to power off, flooding logs, high CPU usage

[thehammer86 (Kristopher Kolpin)]

I’m experiencing the following issues:

1. PBX takes 2 min to shut down
   PBX takes 2 minutes to shut down on reboot

2. And I’m also experiencing a flooding of logs with high CPU usage (100-176%) on a quad core system.

This was a fresh install of FreePBX (https://downloads.freepbxdistro.org/ISO/SNG7-FPBX-64bit-1904-2.iso) and I noticed that after I did the latest round of module updates this morning that these issues started.

Please see screenshots.

[thehammer86 (Kristopher Kolpin)]

You can see that in screen fail2ban takes several minutes to drop down from 100% cpu usage.

I think I’ve narrowed down the issue to a fail2ban/firewall issue. As soon as an endpoint tries to many times with the wrong password asterisk then jumps to 100% or more indefinitely.

[dicko]

IMHO, using pyinotify as fail2ban’s ‘backend’ , significantly improves its responsiveness

[thehammer86 (Kristopher Kolpin)]

I’m using FreePBX distro. Is pyinotify not being used by default for some reason?

[dicko]

I can’t say, I went the

https://github.com/fail2ban/fail2ban/archive/0.10.tar.gz

route

[PitzKey (Itzik)]

Not sure what’s happening, but this is I think the 3rd or 4th complain within the last week about FreePBX restarting/Power Off delays.

Were there any recent updates that might have caused this?

[thehammer86 (Kristopher Kolpin)]

I thought I solved it as there were some emails sitting in the postfix queue but that wasn’t it. And then fail2ban spat out a bunch of banned IPs in an email but that didn’t solve it either.

The strange part right now is that asterisk is sitting at 178% usage but when a call is placed (incoming or outgoing) the asterisk process usage drops to a meagerly 5% and call quality is fine. As soon as the call is done asterisk usage jumps back up to 178%.

I’m using the latest distro FreePBX 14 (build 1902 I think) and asterisk 16.4.1.

Any thougts @tonyclewis or other Sangoma team members?

[tonyclewis (Tony Lewis)]

Sorry I have nothing to do with Sangoma or the FreePBX project. Heck I was told I was not welcomed at Astricon anymore. First time in over 10 years I will miss it.

[thehammer86 (Kristopher Kolpin)]

@tonyclewis Sorry to hear things have soured.

Who might I ping for a second look at this issue. I just want to help figure out what’s causing the high cpu usage as I’m not the only one seeing it.

[tonyclewis (Tony Lewis)]

I have no clue. Maybe @mfredrickson can assist.

[thehammer86 (Kristopher Kolpin)]

Doing some more digging. I had there error pop up today.

Fclose 110 is not a valid stream resource

PBxact version 14 is not making or recieving calls, internally or externally. Phones show registered. When I restart the system, it worked for a couple hours. I ran fwconsole restart and it says In php-asmanager.php line 489: fclose () : 110 is not a valid stream resource. This is happening on another system as well, except it says 111 is not a valid stream resource. What is going on?

Apparently it was fixed in Asterisk 13.25 and 15.x but it showed up for me today in 16.4.1

[mattf (Matthew Fredrickson)]

Sorry to hear about the trouble!

It’d be surprising that it would be fixed in 13.25.0 as it was released in February-ish of this last year, whereas 16.4.1 was released around July unless it was some sort of regression that was introduced. It sounds like it might be worth opening a bug report on issues.freepbx.org with your system info and relevant version of Asterisk that exhibits the issue.

Best wishes!
Matt