Asterisk periodically unresponsive, only fix is reboot PBX

izzyfanto · May 20, 2019, 4:14pm

As the title says, Asterisk will periodically become unresponsive, and the only fix is to constantly reboot the PBX. Trying to restart Asterisk won’t work, see below.

The PBX will function for about a day, then randomly stop processing calls.
sip show peers and sip show channels will be both be normal with all phones and SIP trunks peer successfully, and channels correctly reporting for active calls.
Then in about 24 hours give or take:
- Both commands peer and channels commands will return nothing
- The phone’s BLFs will freeze
- No outside incoming/outgoing calls and no local ext to ext calls
- PBX can ping the phones
- On the GUI dashboard, it’ll report a high number of channel use one right after another, but no calls are actually being processed at that point
SIP guests is turned off
Password changed to a new one
Nothing in fail2ban
No abnormal logins in logs
SIP ports restricted to SIP provider IPs
No web GUI or SSH ports open outside

fwconsole restart
Running PBXact shutdown…
Stopping RestApps Server
Stopped RestApps Server
Stopping UCP Node Server
[>---------------------------] 1 sec
Stopped UCP Node Server
Stopping Chat Server
Stopped Chat Server
Zulu Server is not running
Shutting down Asterisk Gracefully. Will forcefully kill after 30 seconds.
Press C to Cancel
Press N to shut down NOW
[============================] 30 secs
Killing asterisk forcefully.
Asterisk is still running and we can’t stop it!

The /var/log/asterisk/full log shows what appears to me as normal functions, but it will go silent when Asterisk freezes. Here’s the last bit of the log before it freezes, tell me if I need to post more:

[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx_builtins.c: Goto (macro-dial-one,dstring,15)
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [dstring@macro-dial-one:15] GotoIf(“SIP/fpbx-”, “0?skipset”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [dstring@macro-dial-one:16] Set(“SIP/fpbx-”, “DSTRING=SIP/99200&SIP/200&”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [dstring@macro-dial-one:17] Set(“SIP/fpbx-”, “ITER=3”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [dstring@macro-dial-one:18] GotoIf(“SIP/fpbx-”, “0?begin”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [dstring@macro-dial-one:19] ExecIf(“SIP/fpbx-”, “0?Return()”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [dstring@macro-dial-one:20] Set(“SIP/fpbx-”, “DSTRING=SIP/99200&SIP/200”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [dstring@macro-dial-one:21] Return(“SIP/fpbx-”, “”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:30] GotoIf(“SIP/fpbx-”, “0?nodial”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:31] GotoIf(“SIP/fpbx-”, “0?skiptrace”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:32] GosubIf(“SIP/fpbx-”, “1?ctset,1():ctclear,1()”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [ctset@macro-dial-one:1] Set(“SIP/fpbx-”, “DB(CALLTRACE/200)=”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [ctset@macro-dial-one:2] Return(“SIP/fpbx-”, “”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:33] Set(“SIP/fpbx-”, “D_OPTIONS=HhTtr”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:34] GosubIf(“SIP/fpbx-”, “0?func-set-sipheader,s,1(Alert-Info,)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:35] NoOp(“SIP/fpbx-”, "Blind Transfer: , Attended Transfer: , User: , Alert Info: ") in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:36] ExecIf(“SIP/fpbx-”, “0?Set(ALERT_INFO=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:37] ExecIf(“SIP/fpbx-”, “0?Set(ALERT_INFO=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:38] ExecIf(“SIP/fpbx-”, “0?Set(ALERT_INFO=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:39] ExecIf(“SIP/fpbx-”, “0?Set(ALERT_INFO=Normal;volume=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:40] ExecIf(“SIP/fpbx-”, “0?Set(ALERT_INFO=Normal;volume=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:41] GosubIf(“SIP/fpbx-”, “0?func-set-sipheader,s,1(Alert-Info,)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:42] ExecIf(“SIP/fpbx-”, “0?Set(CHANNEL(musicclass)=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:43] GosubIf(“SIP/fpbx-”, “0?qwait,1()”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:44] Set(“SIP/fpbx-”, “__CWIGNORE=”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:45] Set(“SIP/fpbx-”, “__KEEPCID=TRUE”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:46] GotoIf(“SIP/fpbx-”, “0?usegoto,1”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:47] GotoIf(“SIP/fpbx-”, “1?godial”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx_builtins.c: Goto (macro-dial-one,s,52)
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:52] Macro(“SIP/fpbx-”, “dialout-one-predial-hook,”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dialout-one-predial-hook:1] MacroExit(“SIP/fpbx-”, “”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:53] ExecIf(“SIP/fpbx-”, “1?Set(D_OPTIONS=HhtrI)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:54] Gosub(“SIP/fpbx-”, “zulu-user-settings-map,s,1(200)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@zulu-user-settings-map:1] GotoIf(“SIP/fpbx-”, “0?200,1”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@zulu-user-settings-map:2] Return(“SIP/fpbx-”, “”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [s@macro-dial-one:55] GotoIf(“SIP/fpbx-”, “1?crm-save”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx_builtins.c: Goto (macro-dial-one,s,58)

lgaetz · May 20, 2019, 4:27pm

Asterisk version? Removal of Asterisk 13.26.0-1 and Asterisk 15.7.2-1

izzyfanto · May 20, 2019, 9:17pm

Ah I see. Yes, it was on 13.26.0. Downgraded since I don’t have time to run the updates. Will report back if this fixes the issues. Really appreciate the help Lorne, thank you!

system · May 27, 2019, 9:17pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.