Asterisk not starting after update

In my previous post (My [awful] upgrade from 13 to 14) I documented that after upgrading my system from Freepbx 13 to SNG7 that the asterisk sysv service was set to DISABLED and my server was unusable because the asterisk process never started. A FreePBX developer said that it’s supposed to be that way because another process is supposed to start asterisk. Well, tonight I just updated my system (FreePBX modules and yum updates) and the exact same thing happened. The sysv service got set back to DISABLED and my server was unusable. The web interface said that it couldn’t connect to asterisk and, at the console, I verified that the asterisk service was not running. I had to set the sysv service back to ENABLED and manually start the service to get everything back to normal. Here is a screenshot to prove at after the reboot asterisk isn’t running. I would love some actual help with this since, apparently, the way my server is acting isn’t how FreePBX is supposed to act.

no-asterisk

There is more to this story than you are telling us. Out of the (literally) thousands of installations world wide, you appear to be the only person reporting this.

The “other program” that is supposed to be starting your system is a call to fwconsole, which is used to establish the run-time environment and trouble check the system before Asterisk starts. The fact that Asterisk is not starting is a symptom of a larger problem. You shouldn’t be using the SysV start (“service asterisk start”) system to be running the system.

I don’t know what else to tell you. There is something about your startup environment that is keeping the system from running, but telling us over and over again that it isn’t running, isn’t going to help us help you troubleshoot this. We need more.

As a suggestion, try adding “fwconsole start” (or restart) to your /etc/rc.local. This might help you moving. Another place to check would be to wait for the system to get up and running, then log in as “root” and try running “fwconsole start” and see what the debug logs (in /var/log/) tell you.

Where can I start looking? What, upon system start, is supposed to issue “fwconsole start” so that I can investigate why that’s not happening? I don’t have details because I don’t know how freepbx was reworked to not rely on sysv services.

The startup scripts (including whatever the rc.local equivalent is) should be starting the system with fwconsole. With that said, it’s possible that the system is actually issuing the correct fwconsole command, and it’s failing.

The real conundrum is going to be if the system tried to start, fails (automatically) then starts OK with a manual start.

if you find that is the way the problem is happening, you need to file an Issues ticket that documents your system config as well as the startup steps that both fail and succeed.

So I found that freepbx.service is the systemd script that calls fwconsole. Checking its status I found that the process exited.

[[email protected] ~]# systemctl status freepbx.service -l
â freepbx.service - FreePBX VoIP Server
Loaded: loaded (/usr/lib/systemd/system/freepbx.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2018-06-26 23:11:37 EDT; 13h ago
Process: 1264 ExecStart=/usr/sbin/fwconsole start (code=exited, status=0/SUCCESS)
Main PID: 1264 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/freepbx.service

Jun 26 23:11:36 voip2 runuser[5101]: pam_unix(runuser:session): session opened for user asterisk by (uid=0)
Jun 26 23:11:36 voip2 runuser[5101]: pam_unix(runuser:session): session closed for user asterisk
Jun 26 23:11:36 voip2 runuser[5111]: pam_unix(runuser:session): session opened for user asterisk by (uid=0)
Jun 26 23:11:36 voip2 runuser[5111]: pam_unix(runuser:session): session closed for user asterisk
Jun 26 23:11:36 voip2 runuser[5121]: pam_unix(runuser:session): session opened for user asterisk by (uid=0)
Jun 26 23:11:37 voip2 fwconsole[1264]: [>---------------------------] < 1 sec
Jun 26 23:11:37 voip2 fwconsole[1264]: Started Chat Server. PID is 5079
Jun 26 23:11:37 voip2 fwconsole[1264]: Running Asterisk post from Zulu module
Jun 26 23:11:37 voip2 fwconsole[1264]: PJSIP Transports for WS and WSS must be enabled in Asterisk SIP Settings under the Chan PJSIP Settings tab. Then restart Asterisk
Jun 26 23:11:37 voip2 systemd[1]: Started FreePBX VoIP Server.

Looking for the PID it says it started with, 1264, it’s not running.

[[email protected] ~]# ps ax | grep 1264
17413 pts/1 S+ 0:00 grep --color=auto 1264

My system is working at the moment, however, because I started the asterisk systemd service (which didn’t start everything, as you mentioned it wouldn’t) then reloaded the config from the web interface, which kicked off everything else that was needed.

So now I just have to figure out why freepbx.service dies… Any hints on which log I should look in to find these answers? Nothing in the default “messages” log shows errors; it looks like it worked fine although it died.

Jun 26 23:11:37 voip2 systemd: Started FreePBX VoIP Server.
Jun 26 23:11:37 voip2 systemd: Reached target Multi-User System.
Jun 26 23:11:37 voip2 systemd: Starting Multi-User System.
Jun 26 23:11:37 voip2 systemd: Starting Update UTMP about System Runlevel Changes…
Jun 26 23:11:37 voip2 systemd: Started Update UTMP about System Runlevel Changes.
Jun 26 23:11:37 voip2 systemd: Startup finished in 693ms (kernel) + 1.297s (initrd) + 1min 38.309s (userspace) = 1min 40.300s.

fwconsole is not a long running process. So it wouldnt be running later in time. Its sole purpose is to start other services

Stop using systemd to start asterisk. Use fwconsole. You’d be able to see if there was an issue or not by just using fwconsole.

You mean sysv… But I get your point. I’ll disable the sysv service again and reboot tonight, then try manually running fwconsole and see what happens. More to come…

What are those other services, specifically? Can I see them in a “ps” after executing fwconsole to verify that fwconsole did its job?

Asterisk obviously, DAHDI, as well as other services, most of which can be seen by running

 fwconsole pm2 --list

Perfect. I’ll reboot tonight, check for asterisk and run the pm2 list to verify that nothing is running, then manually run fwconsole to start everything and see what happens.

You can also run fwconsole stop to stop everything. Not sure how that will play with the system service running however.

If you are ‘off the reservation’ as you appear to be, then I suggest you research a bit how the various flavors of Linux ‘startup’ applications individually work

So after Grub(2) there are various flavors. mostly SysV, upstart. runit and systemd

You mention SysV and that was the mainstay for years, however more modern ‘linui’ have moved on, there was for a while upstart and even runit, but basically most all now use systemd.

Now, Asterisk source comes with templates for many of these and following a standard Asterisk installation including

make config

is a step that you probably initally used to generate your SysV startup script in /etc/init.d, similarly, that would now setup a systemd asterisk.service file .

FreePBX is not just astererisk however and relies on two key services , mysql (technically flavor independent) and a webserver (technically flavor independent) and the raw install script, you also probably used originally , has parameters you can set as to the constraints needed.

Basically Astererisk needs to be run as user:asterisk group:asterisk, this is easy to set with any particular OS or startup method, BUT . . . .

It also needs to start after your Webserver and mysql are running, also both easy to modifiy the startup script that asterisk would have done to force that condition.
BUT . . . . .

It is much easier to have a wrapper script, originally called ‘amportal’ now ‘fwconsole’ that ‘curates’ the basic order of startup of services with thecorrect permissions, and enables further ‘add-ons’ to be similarly managed.

Now although not necessarily a ‘clean’ solution, that script is found at the end of /etc/rc.local

You can certainly start FreePBX cleanly not using that script but unless you are fully aware of the how to similarly manage permission and order of prerequisites etc. I suggest you just trust /etc/rc.local/ to call fwconsole for you and further use the same script when you need to “change shi*” outside the Gooey. So also delete any other attempt to startup asterisk otherwise.

1 Like

So I disabled the sysv asterisk service as recommended and restarted. Asterisk didn’t start.

[[email protected] ~]# chkconfig

Note: This output shows SysV services only and does not include native
systemd services. SysV configuration data might be overridden by native
systemd configuration.

If you want to list systemd services use ‘systemctl list-unit-files’.
To see services enabled on particular target use
‘systemctl list-dependencies [target]’.

asterisk 0:off 1:off 2:off 3:off 4:off 5:off 6:off

[[email protected] ~]# fwconsole pm2 --list
±-------------±----±-------±---------±-------±----±----+
| Process Name | PID | Status | Restarts | Uptime | CPU | Mem |
±-------------±----±-------±---------±-------±----±----+
[[email protected] ~]#

I started it manually with fwconsole and it came up fine.

[[email protected] ~]# fwconsole start
Running FreePBX startup…
Taking too long? Customize the chown command, See http://wiki.freepbx.org/display/FOP/FreePBX+Chown+Conf
Setting Permissions…
Setting base permissions…Done
Setting specific permissions…
81277 [============================]
Finished setting permissions
Running Asterisk pre from Dahdiconfig module
Wanrouter: No valid Sangoma Hardware found, if you have no Sangoma cards this is OK
DAHDi: Already started
Running Asterisk pre from Firewall module
Running Asterisk pre from Pms module
Running Asterisk pre from Sysadmin module
Running Sysadmin Hooks
Restarting fail2ban

fail2ban Restarted
Updating License Information for 21793354
Checking Vpn server
Starting Asterisk…
[============================] < 1 sec
Asterisk Started
Running Asterisk post from Dahdiconfig module
Running Asterisk post from Endpoint module
Running Asterisk post from Pagingpro module
Running Asterisk post from Restapps module
RestApps Server has already been running on PID 3489 for 19 seconds
Running Asterisk post from Ucp module
UCP Node Server has already been running on PID 4617 for 14 seconds
Running Asterisk post from Vqplus module
RestApps is not licensed.
Running Asterisk post from Xmpp module
Chat Server has already been running on PID 5138 for 10 seconds
Running Asterisk post from Zulu module
Enabling the Asterisk ARI Server
Increasing HTTP Session Limit
Changing Websocket Mode to PJSIP
Reloading PBX to update new settings…Done
Enabling PJSIP Websocket server
Websocket changed, updating webrtc clients…Done
This product is not licensed

I went to /var/log/asterisk and saw which logs were recently written to and saw freepbx.log, so I poked around looking for the time I rebooted the server. There were a million lines like this one:

[2018-Jun-27 22:05:02] [WARNING] (libraries/modulefunctions.legacy.php:7) - Depreciated Function module_getinfo detected in /var/www/html/admin/modules/cxpanel/functions.inc.php on line 76

If you’ve never used it, cxpanel is the internal name for iSymphony, which I have installed. No one uses it, so I disabled it via the sysv service (the only way it’s configured to start):

[[email protected] asterisk]# chkconfig iSymphonyServerV3 off

I rebooted again and once I was able to login I watched the startup of the systemd service freepbx.service with

systemctl status freepbx.service -l

Eventually I saw that it started correctly, my phone came online, and the web interface worked again. No idea what iSymphony was doing, but disabling it made my server work normally again.

If you don’t use iSymphony , don’t just disable it, delete it. :slight_smile:

Eventually :slight_smile: I have to make sure that no one actually uses it first. I just kind of assumed it wasn’t being used. I should probably be certain before uninstalling…

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.