Asterisk periodically unresponsive, only fix is reboot PBX

As the title says, Asterisk will periodically become unresponsive, and the only fix is to constantly reboot the PBX. Trying to restart Asterisk won’t work, see below.

  • The PBX will function for about a day, then randomly stop processing calls.
  • sip show peers and sip show channels will be both be normal with all phones and SIP trunks peer successfully, and channels correctly reporting for active calls.
  • Then in about 24 hours give or take:
    • Both commands peer and channels commands will return nothing
    • The phone’s BLFs will freeze
    • No outside incoming/outgoing calls and no local ext to ext calls
    • PBX can ping the phones
    • On the GUI dashboard, it’ll report a high number of channel use one right after another, but no calls are actually being processed at that point
  • SIP guests is turned off
  • Password changed to a new one
  • Nothing in fail2ban
  • No abnormal logins in logs
  • SIP ports restricted to SIP provider IPs
  • No web GUI or SSH ports open outside

fwconsole restart
Running PBXact shutdown…
Stopping RestApps Server
Stopped RestApps Server
Stopping UCP Node Server
[>---------------------------] 1 sec
Stopped UCP Node Server
Stopping Chat Server
Stopped Chat Server
Zulu Server is not running
Shutting down Asterisk Gracefully. Will forcefully kill after 30 seconds.
Press C to Cancel
Press N to shut down NOW
[============================] 30 secs
Killing asterisk forcefully.
Asterisk is still running and we can’t stop it!

The /var/log/asterisk/full log shows what appears to me as normal functions, but it will go silent when Asterisk freezes. Here’s the last bit of the log before it freezes, tell me if I need to post more:

[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx_builtins.c: Goto (macro-dial-one,dstring,15)
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:15] GotoIf(“SIP/fpbx-”, “0?skipset”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:16] Set(“SIP/fpbx-”, “DSTRING=SIP/99200&SIP/200&”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:17] Set(“SIP/fpbx-”, “ITER=3”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:18] GotoIf(“SIP/fpbx-”, “0?begin”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:19] ExecIf(“SIP/fpbx-”, “0?Return()”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:20] Set(“SIP/fpbx-”, “DSTRING=SIP/99200&SIP/200”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:21] Return(“SIP/fpbx-”, “”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:30] GotoIf(“SIP/fpbx-”, “0?nodial”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:31] GotoIf(“SIP/fpbx-”, “0?skiptrace”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:32] GosubIf(“SIP/fpbx-”, “1?ctset,1():ctclear,1()”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:1] Set(“SIP/fpbx-”, “DB(CALLTRACE/200)=”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:2] Return(“SIP/fpbx-”, “”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:33] Set(“SIP/fpbx-”, “D_OPTIONS=HhTtr”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:34] GosubIf(“SIP/fpbx-”, “0?func-set-sipheader,s,1(Alert-Info,)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:35] NoOp(“SIP/fpbx-”, "Blind Transfer: , Attended Transfer: , User: , Alert Info: ") in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:36] ExecIf(“SIP/fpbx-”, “0?Set(ALERT_INFO=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:37] ExecIf(“SIP/fpbx-”, “0?Set(ALERT_INFO=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:38] ExecIf(“SIP/fpbx-”, “0?Set(ALERT_INFO=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:39] ExecIf(“SIP/fpbx-”, “0?Set(ALERT_INFO=Normal;volume=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:40] ExecIf(“SIP/fpbx-”, “0?Set(ALERT_INFO=Normal;volume=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:41] GosubIf(“SIP/fpbx-”, “0?func-set-sipheader,s,1(Alert-Info,)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:42] ExecIf(“SIP/fpbx-”, “0?Set(CHANNEL(musicclass)=)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:43] GosubIf(“SIP/fpbx-”, “0?qwait,1()”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:44] Set(“SIP/fpbx-”, “__CWIGNORE=”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:45] Set(“SIP/fpbx-”, “__KEEPCID=TRUE”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:46] GotoIf(“SIP/fpbx-”, “0?usegoto,1”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:47] GotoIf(“SIP/fpbx-”, “1?godial”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx_builtins.c: Goto (macro-dial-one,s,52)
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:52] Macro(“SIP/fpbx-”, “dialout-one-predial-hook,”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:1] MacroExit(“SIP/fpbx-”, “”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:53] ExecIf(“SIP/fpbx-”, “1?Set(D_OPTIONS=HhtrI)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:54] Gosub(“SIP/fpbx-”, “zulu-user-settings-map,s,1(200)”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:1] GotoIf(“SIP/fpbx-”, “0?200,1”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:2] Return(“SIP/fpbx-”, “”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx.c: Executing [[email protected]:55] GotoIf(“SIP/fpbx-”, “1?crm-save”) in new stack
[2019-05-17 14:56:27] VERBOSE[10303][C-0000018e] pbx_builtins.c: Goto (macro-dial-one,s,58)

Asterisk version? Removal of Asterisk 13.26.0-1 and Asterisk 15.7.2-1

2 Likes

Ah I see. Yes, it was on 13.26.0. Downgraded since I don’t have time to run the updates. Will report back if this fixes the issues. Really appreciate the help Lorne, thank you!

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.