Have a Freepbx 15 up and running.
Whenever I try to apply configuration it is painfully slow(up to 5 minutes) It usually completes but there are times where it says couldnt do it try fwconsole verbose etc… I am not sure where to go with this except to the best that are in this forum. I am a relative newbe at this so take it easy just running one small system
thanx
Yosef G
I’ve noticed a slower apply config in general in v15. I’ve had some success removing some modules in Module Admin that don’t get used, but it’s still slower than previous versions.
Try changing:
Settings -> Advanced Settings -> Enable Module Signature Checking
to ‘No’.
It will be changed back to ‘Yes’ each time new modules are installed.
If this solves your problem, the following script can be used instead of Module Admin to upgrade modules without incurring long reload times and having to repeatedly disable signature checking:
#!/bin/sh
fwconsole ma download framework
fwconsole ma installlocal
fwconsole ma upgradeall
fwconsole setting SIGNATURECHECK 0
fwconsole reload
Distro or did you install the softwre yourself?
Participating in your own troubleshooting is key
4 Likes
I will try that now but just tried an apply config before I read this and got this error
There was an error during reload: Unknown Error. Please Run: fwconsole reload --verbose
Any Ideas
Yosef Guttman
Also didn’t help seems to be taking upwards of 5 minutes and still not always working
Yes but you will have to actually do some troubleshooting work, as already suggested.
Earlier I asked whether this is a Distro or you installed FreePBX yourself. Please answer that.
What is the output when you run the reload at the command line with the verbose option?
Is your system up to date with OS updates (yum update) and FreePBX modules?
What are your DNS resolvers?
Do you have a firewall or proxy that might block outbound HTTP(S) traffic?
When you are doing a reload and it hangs, in SSH or console, run ps ax and see what processes are running.
[quote=“billsimon, post:8, topic:76345”]
Yes but you will have to actually do some troubleshooting work, as already suggested.
I AM TRYING BUT JUST NOT THAT GOOD AT LINUX PROGRAMMING AND NOT SURE WHAT I AM LOOKING FOR
Earlier I asked whether this is a Distro or you installed FreePBX yourself. Please answer that.
I DOWNLOADED FREEPBX 12 OR 13 AND THEN UPDATED IT ACCORDING TO DIRECTIONS FROM THE WEBSITE TO 15
What is the output when you run the reload at the command line with the verbose option?
ON THE GUI I CANNOT SEE ANY OUTPUT (THERE IS A SMALL BLACK LINE BUT NEVER BIG ENOUGH TO SEE ANY OUTPUT. I JUST RAN IT THROUGH PUTTY ON THE SERVER. THERE WAS ONE RED ERROR MESSAGE
In Process.php line 1335:
[Symfony\Component\Process\Exception\ProcessTimedOutException]
The process “/usr/bin/ffmpeg -version” exceeded the timeout of 60 seconds.
Is your system up to date with OS updates (yum update) and FreePBX modules?
FREEPBX IS UPDATED ACC TO MODULE ADMIN I WILL CHECK THE LINUX WITH YUM UPDATE
What are your DNS resolvers?
20.67.222.222
Do you have a firewall or proxy that might block outbound HTTP(S) traffic?
NO
When you are doing a reload and it hangs, in SSH or console, run ps ax and see what processes are running.
HOW DO I RUN IT I TRIED JUST TYPING PS AX ON THE LINUX SCREEN GOT AN ERROR
Thank you for all your help
Yosef G
You have to type it in lower case…
5 Likes
You’re right
here is the response from the command after hitting apply config
PID TTY STAT TIME COMMAND
1 ? Ss 64:08 /usr/lib/systemd/systemd --switched-root --system --deserialize 22
2 ? S 0:01 [kthreadd]
4 ? S< 0:00 [kworker/0:0H]
6 ? S 5:51 [ksoftirqd/0]
7 ? S 0:00 [migration/0]
8 ? S 0:00 [rcu_bh]
9 ? R 100:41 [rcu_sched]
10 ? S< 0:00 [lru-add-drain]
11 ? S 3:09 [watchdog/0]
13 ? S 0:00 [kdevtmpfs]
14 ? S< 0:00 [netns]
15 ? S 0:09 [khungtaskd]
16 ? S< 0:00 [writeback]
17 ? S< 0:00 [kintegrityd]
18 ? S< 0:00 [bioset]
19 ? S< 0:00 [bioset]
20 ? S< 0:00 [bioset]
21 ? S< 0:00 [kblockd]
22 ? S< 0:00 [md]
23 ? S< 0:00 [edac-poller]
24 ? S< 0:00 [watchdogd]
27 ? S 76:21 [kswapd0]
28 ? SN 0:00 [ksmd]
29 ? SN 0:58 [khugepaged]
30 ? S< 0:00 [crypto]
38 ? S< 0:00 [kthrotld]
40 ? S< 0:00 [kmpath_rdacd]
41 ? S< 0:00 [kaluad]
44 ? S< 0:00 [kpsmoused]
46 ? S< 0:00 [ipv6_addrconf]
59 ? S< 0:00 [deferwq]
94 ? S 1:26 [kauditd]
269 ? S< 0:00 [ata_sff]
276 ? S 0:00 [scsi_eh_0]
278 ? S< 0:00 [scsi_tmf_0]
279 ? S 0:00 [scsi_eh_1]
280 ? S< 0:00 [scsi_tmf_1]
283 ? S 0:00 [scsi_eh_2]
284 ? S< 0:00 [scsi_tmf_2]
285 ? S 0:00 [scsi_eh_3]
286 ? S< 0:00 [scsi_tmf_3]
287 ? S 0:00 [scsi_eh_4]
288 ? S< 0:00 [scsi_tmf_4]
289 ? S 0:00 [scsi_eh_5]
290 ? S< 0:00 [scsi_tmf_5]
296 ? S< 0:00 [ttm_swap]
365 ? S< 0:00 [kdmflush]
366 ? S< 0:00 [bioset]
375 ? S< 0:00 [kdmflush]
376 ? S< 0:00 [bioset]
389 ? S< 0:00 [bioset]
390 ? S< 0:00 [xfsalloc]
391 ? S< 0:00 [xfs_mru_cache]
392 ? S< 0:00 [xfs-buf/dm-0]
393 ? S< 0:00 [xfs-data/dm-0]
394 ? S< 0:00 [xfs-conv/dm-0]
395 ? S< 0:00 [xfs-cil/dm-0]
396 ? S< 0:00 [xfs-reclaim/dm-]
397 ? S< 0:00 [xfs-log/dm-0]
398 ? S< 0:00 [xfs-eofblocks/d]
399 ? S 56:58 [xfsaild/dm-0]
400 ? S< 6:49 [kworker/0:1H]
485 ? Ss 41:43 /usr/lib/systemd/systemd-journald
508 ? Ss 0:00 /usr/sbin/lvmetad -f
512 ? Ss 0:00 /usr/lib/systemd/systemd-udevd
567 ? S 0:00 [jbd2/sda1-8]
568 ? S< 0:00 [ext4-rsv-conver]
575 ? S<sl 5:02 /sbin/auditd
598 ? Ss 0:20 /sbin/rpcbind -w
602 ? Ss 25:07 /usr/lib/systemd/systemd-logind
630 ? Ss 55:06 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
645 ? Ss 0:10 /usr/sbin/incrond
647 ? Ss 5:17 avahi-daemon: running [freepbx.local]
651 ? Ssl 16:44 /usr/lib/polkit-1/polkitd --no-debug
664 ? S 0:00 avahi-daemon: chroot helper
667 ? S 0:15 /usr/sbin/chronyd -f /etc/sangoma_chrony.conf
778 ? S< 0:00 [cfg80211]
818 ? Ssl 24:32 /usr/bin/python2 -Es /usr/sbin/tuned -l -P
822 ? Ss 2:43 /usr/sbin/dnsmasq -k
825 ? Ss 0:00 /usr/sbin/sshd -D
832 ? Rsl 196:32 /usr/bin/redis-server 127.0.0.1:6379
834 ? Ssl 29:29 /usr/sbin/rsyslogd -n
837 ? Ss 0:00 /usr/sbin/xinetd -stayalive -pidfile /var/run/xinetd.pid
843 ? Ss 10:29 /usr/sbin/httpd -DFOREGROUND
859 ? Ss 0:00 /usr/sbin/atd -f
864 ? Ss 6:31 /usr/sbin/crond -n
937 ? Ss 0:00 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
1302 ? Ss 0:48 /usr/libexec/postfix/master -w
1304 ? S 0:21 qmgr -l -t unix -u
1320 ? Sl 466:33 /usr/bin/mongod --quiet -f /etc/mongod.conf run
1347 ? Ss 159:29 /usr/local/fop2/fop2_server
1939 ? S 0:00 /bin/sh /usr/sbin/safe_asterisk -U asterisk -G asterisk
2020 ? S 0:00 [kworker/u16:0]
3222 ? S 0:00 /usr/sbin/crond -n
3223 ? S 0:00 /usr/sbin/crond -n
4922 ? S 0:01 [kworker/0:2]
5072 ? S 0:00 [kworker/u16:2]
5451 ? S 0:03 /usr/sbin/httpd -DFOREGROUND
5452 ? S 0:00 /usr/sbin/httpd -DFOREGROUND
5453 ? S 0:03 /usr/sbin/httpd -DFOREGROUND
5454 ? S 0:00 /usr/sbin/httpd -DFOREGROUND
5455 ? S 0:00 /usr/sbin/httpd -DFOREGROUND
5457 ? S 0:00 /usr/sbin/httpd -DFOREGROUND
5791 ? S 0:00 /usr/sbin/httpd -DFOREGROUND
6215 ? R 0:00 [kworker/0:0]
7229 ? S 0:00 [kworker/0:1]
7352 ? S 0:00 [kworker/u16:1]
7460 ? S 0:01 /usr/sbin/httpd -DFOREGROUND
7473 ? S 0:00 /usr/sbin/httpd -DFOREGROUND
7474 ? S 0:00 /usr/sbin/httpd -DFOREGROUND
7541 ? S 0:01 php /usr/sbin/fwconsole reload --json
7549 ? S 0:00 /usr/sbin/CROND -n
7550 ? S 0:00 /usr/sbin/CROND -n
7551 ? Ss 0:00 /bin/sh -c /usr/sbin/fwconsole queuestats --syncall >> /tmp/reader.log 2>&1
7552 ? Ss 0:00 /bin/sh -c [ -e /usr/sbin/fwconsole ] && sleep $((RANDOM%30)) && /usr/sbin/fwconsole job --run --quiet 2
7553 ? S 0:01 php /usr/sbin/fwconsole queuestats --syncall
7578 ? S 0:00 sh -c cd /var/www/html/admin/modules/pm2/node && mkdir -p /home/asterisk/.pm2 && mkdir -p /var/www/html/
7581 ? Dl 0:00 node /var/www/html/admin/modules/pm2/node/node_modules/pm2/bin/pm2 jlist
7592 ? D 0:00 php /usr/sbin/fwconsole job --run --quiet
7593 pts/0 R+ 0:00 ps ax
7594 ? D 0:00 /sbin/runuser asterisk -c /var/www/html/admin/modules/firewall/bin/getservices
7595 ? S 0:00 sh -c stty -a | grep columns
7596 ? D 0:00 [stty]
7597 ? D 0:00 [grep]
9472 ? Ss 0:00 login – root
13772 ? S 67:13 php /var/www/html/admin/modules/firewall/hooks/voipfirewalld
13990 ? Sl 102:18 /usr/bin/python /usr/bin/fail2ban-server -b -s /var/run/fail2ban/fail2ban.sock -p /var/run/fail2ban/fail
13992 ? S 19:50 /usr/libexec/gam_server
14074 ? S 115:22 voipfirewalld (Monitor thread)
14086 ? S 0:00 /usr/sbin/crond -n
14416 ? Ss 0:00 /usr/bin/python /usr/local/bin/pnp_server
14901 ? S 0:00 /usr/sbin/crond -n
15112 ? S 0:00 /usr/sbin/CROND -n
15219 ? S 0:00 /usr/sbin/crond -n
15220 ? S 0:00 /usr/sbin/crond -n
15425 ? S 0:00 /usr/sbin/crond -n
15426 ? S 0:00 /usr/sbin/crond -n
15730 ? S 0:00 /usr/sbin/crond -n
15731 ? S 0:00 /usr/sbin/crond -n
15962 ? S 0:00 /usr/sbin/crond -n
15963 ? S 0:00 /usr/sbin/crond -n
16348 ? S 0:00 /usr/sbin/crond -n
20856 tty1 Ss+ 0:00 -bash
21010 ? S 0:00 /usr/sbin/crond -n
21011 ? S 0:00 /usr/sbin/crond -n
21012 ? S 0:00 /usr/sbin/crond -n
21013 ? S 0:00 /usr/sbin/crond -n
21555 ? S 0:00 /usr/sbin/crond -n
21556 ? S 0:00 /usr/sbin/crond -n
22105 ? S 0:00 /usr/sbin/crond -n
22106 ? S 0:00 /usr/sbin/crond -n
23687 ? Sl 214:54 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-e
23724 ? S 0:00 /usr/sbin/crond -n
23725 ? S 0:00 /usr/sbin/crond -n
23729 ? S 0:00 /usr/sbin/crond -n
23730 ? S 0:00 /usr/sbin/crond -n
24912 ? Sl 924:00 /usr/sbin/asterisk -f -U asterisk -G asterisk -vvvg -c
26918 ? Ssl 47:16 PM2 v4.5.0: God Daemon (/home/asterisk/.pm2)
26928 ? Ssl 150:32 node /var/www/html/admin/modules/core/node/fastagi-server.js
26940 ? Ssl 141:47 letschat
27876 ? Ssl 0:29 node /var/www/html/admin/modules/ucp/node/index.js
31062 ? S 0:00 pickup -l -t unix -u
31168 ? Ss 0:01 sshd: root@pts/0
31182 pts/0 Ss 0:00 -bash
I’ll suggest installing atop (yum install atop) and running that as this tool lets one simultaneously see onging disk, cpu, network activity as well. FPBX reload hits the disk quite a bit so slow disks will impact things - is this a physical system or a virtual one?
I want to notice that we have dozens of ‘Fpbx Distro 15’ instances on all kind of vps’s or applied systems, with all kind of hardware etc etc etc.
An we also noticed there is aquite a difference between reload time with Distro version 15, compared to Distro v14, v13, v12 etc.
I also heard other users with this same info, about slow reloads.
Disable module signaturecheck is not a good idea.
It does not take 10 minutes, but more to 30 seconds, compared to 2 seconds with a v13 Distro.
Nothing to do with configs of hardware or software. V15 just takes longer on reloads, compared to pre v15 Distro’s.
It would be nice if this can be improved in the future.
Now I just hold back on pressing the ‘Apply changes’ button.
it is a physical system. How do I post the results from atop
Yosef G
It would be nice to identify why you have dozens of cron jobs running in the foreground crond -n
ps auxf
would identify the user doing that .
There should only be one of those, which tells me one of your cron entries is not completing correctly. I’d say that “crontab -e” would be a good place to start - see which of your cron programs is still running.
An alternative to that would be kill ps -ax | cut -f1 -d' ' (all of the quotes are required) and see if getting rid of all of these dead cron programs might help…
1 Like
Back quotes are markup, for pre-formatted text. There is an alternative syntax for bash, which should survive the forum:
kill $( ps -ax | cut -f1 -d’ ’ )
1 Like
ps -ax -o pid= | xargs kill
no quotes or backtics needed 
2 Likes
That in my opinion is exactly why it’s taking so long, killing the processes might help but the root cause would be (my guess that if crond -n is holding dozens of fwconsole processes from completion) fwconsole will have then just wait in line until it can do useful stuff, no ?
bug?
1 Like
I ran crontab -e
result at bottom of screen
“/tmp/crontab.JCypxV” ol, oc
but then I cant continue I don’t know how to get out of whatever it is running and get back to command line