FreePBX 17 - Slow Shutdown, second try

schau · July 16, 2025, 10:39am

Hi,

first time poster. English is not my first language. So please be kind, if I do wrong or stupid things.

I found out, that my current problem seems to be the same as described here:

My freepbx needs nearly 4 Minutes to shut down (including the vm). Thats way to long for the currently configured USV which shuts down Power 180 seconds after sending the shutdown command … anyway. I googled a bit and found some interesting things not mentioned in the abouve Thread.

The freepbx is currently running on a proxmox ve with

4 GB Ram
2 cores
20 GB Harddisk
debian 12
freepbx 17.0.19.24

After startup, and waiting 10 minutes, everything is working fine, Dashboard is showing hooks, especialy for the asterisk part.

A

systemctl | grep -E “asterisk|maria|apache2|fail2ban”

shows up with:

apache2.service loaded active running The Apache HTTP Server
fail2ban.service loaded active running Fail2Ban Service
mariadb.service loaded active running MariaDB 10.11.6 database server

So no Asterisk.

systemctl status asterisk.service

shows

asterisk.service - LSB: Asterisk PBX
Loaded: loaded (;;file://asteriskng/etc/init.d/asterisk/etc/init.d/asterisk;;; generated)
Active: inactive (dead)
Docs: ;;man:systemd-sysv-generator(8)man:systemd-sysv-generator(8);;

This seems to be ok, because the /run/systemd/generator.late/asterisk.service should just start the service and stop then (heard it from chatgpt).

But with a “ps -aux” I could see, that asterisk is still running (and working).

With a shutdown in this state, it will last about 3:50 Minutes to shutdown.

With a “fwconsole stop” it is down to about 2:30 Minutes.

With a “systemctl restart (or start) asterisk.service” the

systemctl | grep -E “asterisk|maria|apache2|fail2ban”

shows

apache2.service loaded active running The Apache HTTP Server
asterisk.service loaded active exited LSB: Asterisk PBX
fail2ban.service loaded active running Fail2Ban Service
mariadb.service loaded active running MariaDB 10.11.6 database server

and

systemctl status asterisk.service

asterisk.service - LSB: Asterisk PBX
Loaded: loaded (;;file://asteriskng/etc/init.d/asterisk/etc/init.d/asterisk;;; generated)
Active: active (exited) since Tue 2025-07-15 15:38:03 CEST; 20h ago
Docs: ;;man:systemd-sysv-generator(8)man:systemd-sysv-generator(8);;
CPU: 13ms

With this setup, the time for a Shutdown is about 1:10 Minutes.

So it seems, that the start/restart of asterisk.service does “only” change the state for systemd, so that systemd can stop the service during shutdown without the, in the linked post mentioned 90 seconds timout.

The quick and dirty solution with my limited linux knowledge is, to create a cronjob, that does a “systemctl start asterisk.service” 15 Minutes after boot (should be possible with the cron @reboot option). But it seems to be not “elegant (german)” or “sophisticated”.

Chatgpt mentioned another way to change the service setup that it is directly controled by systemd…but I don’t know, if the knowledge of chatgpt is real knowledge or only a “idea” how it could work.

Any hints, what is a better way?

Regards
Peter

shahin · July 16, 2025, 10:42am

Hi @schau
Not Clear what is your issue ? Could you pls make basic explanation one more time if you can.

schau · July 16, 2025, 10:51am

Of course.
The problem is the time for a shutdown. With CentOS it was really fast in about 1 Minute (quite shure, that it depends on the system speed) but with debian 12 it is really slow with nearly 4 minutes. Same system, same setup. Every other debian 12 on the proxmox will shutdown faster.

Does this make it clear?
Peter

shahin · July 16, 2025, 11:05am

Okay where is installed your FreePBX-17 Debian12 system? in a VM side or in a HW (HardWare) side?

schau · July 16, 2025, 11:13am

On a Proxmox VM

shahin · July 16, 2025, 11:14am

Try to install clear Debian-12 on your Proxmox VM to compare it

shahin · July 16, 2025, 11:21am

I do have 2 installation in a ESXi side + 1 in a FreeBSD bhyve side..
I don’t have any reboot/restart issues.

Esxi 7 side

root@freepbx-17:~# uname -r
6.1.0-37-amd64
root@freepbx-17:~# 
root@freepbx-17:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 12 (bookworm)
Release:        12
Codename:       bookworm
root@freepbx-17:~#

FreeBSD - Bhyve
~ uname -a
FreeBSD freebsd 14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC amd64

root@freepbx-17:~# uname -r
6.1.0-37-amd64
root@freepbx-17:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 12 (bookworm)
Release:        12
Codename:       bookworm
root@freepbx-17:~#

schau · July 16, 2025, 11:59am

Shutdowntime on a “clean” debian 12: 12 seconds

shahin · July 16, 2025, 12:31pm

I don’t get it why Shutdown / Power off time is more important for you.. Servers has some steps when poweroff it. Process by Process and with waiting time all process getting close.
If you want to make fast Power Down Pull out Electrical cable..

schau · July 16, 2025, 12:37pm

I wrote it abouve. With a powerdrop the UPS will jump in and with the time long enough it will shutdown all servers it supplies. And the controller for the UPS will send a “kill power” signal to the UPS which will poweroff the UPS power after 180 second. This time should be enough to shutdown alle servers gracefully to keep the filesystems clean. If this time is higher, the power will be killend just in the shutdown process with unpredictable results. As you can read, the question is: is there a better way to get the systemd be aware of the state of the asterisk service as to start it again to change the state to a state the systemd can handle? And it is not only a problem in my setup, there seems to be several others who found this problem, but no fix for it.

Anyway, thanks for you help
Regards
Peter

shahin · July 16, 2025, 12:40pm

Also you can see at your FreePBX which service are up and running.. All this services needs some time to close or kill..

# systemctl list-units --type=service --state=running

you can see from log last poweroff time for service stops.

# journalctl -b -1 -r

schau · July 16, 2025, 12:42pm

Yes, and with a

shutdown -h now

the system needs 3:45 Minutes

With a

systemctl start asterisk.service
shutdown -h now

only 1:10 Minute. Please explain.
Peter

shahin · July 16, 2025, 12:42pm

180 sec.. might be not engough time for safe power off/shutdown.

schau · July 16, 2025, 12:43pm

Will not fix the problem, only the symptom.

shahin · July 16, 2025, 12:50pm

You are here starting Asterisk… Then sending sigkill ( killing all running process) and takes less time?

you can check yours Freepbx system you can see 23 running process has.. this process closeing or killing time is different. And might be you havent check your logs which process when is received SIGKILL signal.

# systemctl list-units --type=service --state=running | grep -i service | wc -l
23

# journalctl -b -1 -r

shahin · July 16, 2025, 12:55pm

Faster (Force stop timeouts) with data loss guaranteed way.

systemctl poweroff --force

Skips waiting for services to stop cleanly
Forces shutdown (may cause data loss)
Useful for stuck systems or servers where speed > safety

schau · July 16, 2025, 1:42pm

Ok, checked it:

systemctl list-units --type=service --state=running

shows 21 services (not 23)

With

journalctl -b -1

I find a this for the system without a “systemctl start asterisk.service” before shutdown: shutdown time here: 172 seconds…just in time:-)

…
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1792 (asterisk) with signal SIGKILL.
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1791 (asterisk) with signal SIGKILL.
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1790 (asterisk) with signal SIGKILL.
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1789 (asterisk) with signal SIGKILL.
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1785 (asterisk) with signal SIGKILL.
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1782 (asterisk) with signal SIGKILL.
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1781 (asterisk) with signal SIGKILL.
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1775 (asterisk) with signal SIGKILL.
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1774 (asterisk) with signal SIGKILL.
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1768 (asterisk) with signal SIGKILL.
Jul 16 15:15:18 asteriskng mariadbd[661]: 2025-07-16 15:15:17 0 [Note] /usr/sbin/mariadbd (initiated by: unknown): Normal shutdown
Jul 16 15:15:18 asteriskng mariadbd[661]: 2025-07-16 15:15:17 46 [Warning] Aborted connection 46 to db: ‘asteriskcdrdb’ user: ‘freepbxuser’ host: ‘localhost’ (Got an error reading communication packets)
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: Killing process 1729 (asterisk) with signal SIGKILL.
Jul 16 15:15:17 asteriskng systemd[1]: freepbx.service: State ‘stop-sigterm’ timed out. Killing.
Jul 16 15:13:47 asteriskng systemd[1]: freepbx.service: Control process exited, code=exited, status=1/FAILURE
Jul 16 15:13:47 asteriskng fwconsole[7551]: Asterisk is still running and we can’t stop it!
Jul 16 15:13:46 asteriskng fwconsole[7551]: Killing asterisk forcefully.
Jul 16 15:13:46 asteriskng fwconsole[7998]: stty: ‘standard input’: Inappropriate ioctl for device
Jul 16 15:13:46 asteriskng fwconsole[7551]: [>---------------------------] < 1 sec
Jul 16 15:12:46 asteriskng fwconsole[7967]: stty: ‘standard input’: Inappropriate ioctl for device
Jul 16 15:12:46 asteriskng fwconsole[7965]: stty: ‘standard input’: Inappropriate ioctl for device
Jul 16 15:12:46 asteriskng fwconsole[7551]: Press N to shut down NOW
Jul 16 15:12:46 asteriskng fwconsole[7551]: Press C to Cancel
Jul 16 15:12:46 asteriskng fwconsole[7551]: Shutting down Asterisk Gracefully. Will forcefully kill after 30 seconds.
Jul 16 15:12:46 asteriskng fwconsole[7551]: Stopped UCP Node Server
…

and with “systemctl start asterisk.service” followed by a shutdown I found this:
time to shutdown: 45 seconds

Jul 16 14:58:35 asteriskng avahi-daemon[500]: Leaving mDNS multicast group on interface lo.IPv4 with address 127.0.0.1.
Jul 16 14:58:35 asteriskng avahi-daemon[500]: Leaving mDNS multicast group on interface lo.IPv6 with address ::1.
Jul 16 14:58:35 asteriskng avahi-daemon[500]: Leaving mDNS multicast group on interface ens18.IPv4 with address 192.168.156.150.
Jul 16 14:58:35 asteriskng systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Jul 16 14:58:35 asteriskng systemd[1]: ssh.service: Deactivated successfully.
Jul 16 14:58:35 asteriskng systemd[1]: Stopped chrony.service - chrony, an NTP client/server.
Jul 16 14:58:35 asteriskng systemd[1]: chrony.service: Deactivated successfully.
Jul 16 14:58:35 asteriskng systemd[1]: Stopped asterisk.service - LSB: Asterisk PBX.
Jul 16 14:58:35 asteriskng systemd[1]: asterisk.service: Deactivated successfully.
Jul 16 14:58:35 asteriskng asterisk[231229]: Stopping Asterisk PBX: asterisk.
Jul 16 14:58:34 asteriskng systemd[1]: rsyslog.service: Consumed 2.117s CPU time.
Jul 16 14:58:34 asteriskng systemd[1]: Stopped rsyslog.service - System Logging Service.
Jul 16 14:58:34 asteriskng systemd[1]: rsyslog.service: Deactivated successfully.
Jul 16 14:58:34 asteriskng systemd[1]: incron.service: Consumed 17min 18.039s CPU time.
Jul 16 14:58:34 asteriskng systemd[1]: Stopped incron.service - file system events scheduler.

As you can see from the original post, the only visible difference seems to be the state for the systemd.

Anyway. I will give the cron fix a try. Will report the outcome here for the archive.

Thanks a lot for the discussion.
Peter