Issue with Asterisk Crashing


(Michael Asher) #1

We have a server that has been crashing pretty frequently. (Every Couple of days)

PBX Firmware:12.7.6-1904-1.sng7
Asterisk: 16.3.0

We have PJSIP setup for the extensions(port 5555) and we are using Chan_SIP(port 5060) for the trunking.

What appears to happen is Asterisk stops processing Chan_SIP and PJSIP requests. The carrier sees our SIP trunk go down but Asterisk still shows it up (with a sip show peers) I have Backtraces, PCAPs, Logs, Netstat output etc. from the last crash. Not sure how to share if someone can offer some input on that I would be glad to share.

The Full logs don’t seem to indicate much. They mostly just stop. HardDisk, CPU and RAM appear fine during the crash. Firewall and IPtables appear to be working correctly. Restarting the Chan_SIP Module does not fix the issue. Running fwconsole restart does fix the issue.

You can see the Recv-Q backing up.
Netstat:

netstat -plnu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
udp 213248 0 0.0.0.0:11596 0.0.0.0:* 17105/asterisk
udp 67456 0 0.0.0.0:11597 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:15860 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:15861 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:12100 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:12101 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:12300 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:12301 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:53 0.0.0.0:* 1283/dnsmasq
udp 0 0 0.0.0.0:69 0.0.0.0:* 1303/xinetd
udp 0 0 0.0.0.0:111 0.0.0.0:* 1/systemd
udp 0 0 0.0.0.0:123 0.0.0.0:* 1046/chronyd
udp 0 0 127.0.0.1:323 0.0.0.0:* 1046/chronyd
udp 0 0 0.0.0.0:4569 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:17304 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:17305 0.0.0.0:* 17105/asterisk
udp 213248 0 0.0.0.0:5060 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:58516 0.0.0.0:* 1022/avahi-daemon:
udp 0 0 0.0.0.0:5353 0.0.0.0:* 1022/avahi-daemon:
udp 0 0 0.0.0.0:5555 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:17932 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:17933 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:14176 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:14177 0.0.0.0:* 17105/asterisk
udp 213248 0 0.0.0.0:10182 0.0.0.0:* 17105/asterisk
udp 54400 0 0.0.0.0:10183 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:18874 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:18875 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:10730 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:10731 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:18938 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:18939 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:60000 0.0.0.0:* 6359/python
udp 0 0 0.0.0.0:19076 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:19077 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:19200 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:19201 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:15132 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:15133 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:11050 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:11051 0.0.0.0:* 17105/asterisk
udp 0 0 0.0.0.0:60261 0.0.0.0:* 17105/asterisk
udp6 2176 0 :::11290 :::* 17105/asterisk
udp6 4352 0 :::11291 :::* 17105/asterisk
udp6 0 0 :::11344 :::* 17105/asterisk
udp6 0 0 :::11345 :::* 17105/asterisk
udp6 0 0 :::56517 :::* 17105/asterisk
udp6 0 0 :::15592 :::* 17105/asterisk
udp6 0 0 :::15593 :::* 17105/asterisk
udp6 0 0 :::11500 :::* 17105/asterisk
udp6 0 0 :::11501 :::* 17105/asterisk
udp6 0 0 :::15662 :::* 17105/asterisk
udp6 0 0 :::15663 :::* 17105/asterisk
udp6 0 0 :::19814 :::* 17105/asterisk
udp6 0 0 :::19815 :::* 17105/asterisk
udp6 0 0 :::19848 :::* 17105/asterisk
udp6 0 0 :::19849 :::* 17105/asterisk
udp6 0 0 :::11836 :::* 17105/asterisk
udp6 0 0 :::11837 :::* 17105/asterisk
udp6 0 0 :::16172 :::* 17105/asterisk
udp6 0 0 :::16173 :::* 17105/asterisk
udp6 0 0 :::12316 :::* 17105/asterisk
udp6 0 0 :::12317 :::* 17105/asterisk
udp6 0 0 :::53 :::* 1283/dnsmasq
udp6 0 0 :::12386 :::* 17105/asterisk
udp6 0 0 :::12387 :::* 17105/asterisk
udp6 0 0 :::111 :::* 1/systemd
udp6 0 0 :::123 :::* 1046/chronyd
udp6 0 0 :::12428 :::* 17105/asterisk
udp6 0 0 :::12429 :::* 17105/asterisk
udp6 0 0 ::1:323 :::* 1046/chronyd
udp6 0 0 :::16728 :::* 17105/asterisk
udp6 0 0 :::16729 :::* 17105/asterisk
udp6 2176 0 :::16752 :::* 17105/asterisk
udp6 0 0 :::16753 :::* 17105/asterisk
udp6 0 0 :::16834 :::* 17105/asterisk
udp6 0 0 :::16835 :::* 17105/asterisk
udp6 0 0 :::12986 :::* 17105/asterisk
udp6 0 0 :::12987 :::* 17105/asterisk
udp6 0 0 :::13246 :::* 17105/asterisk
udp6 0 0 :::13247 :::* 17105/asterisk
udp6 8704 0 :::13452 :::* 17105/asterisk
udp6 0 0 :::13453 :::* 17105/asterisk
udp6 0 0 :::17634 :::* 17105/asterisk
udp6 0 0 :::17635 :::* 17105/asterisk
udp6 0 0 :::17692 :::* 17105/asterisk
udp6 0 0 :::17693 :::* 17105/asterisk
udp6 6528 0 :::17788 :::* 17105/asterisk
udp6 0 0 :::17789 :::* 17105/asterisk
udp6 0 0 :::17916 :::* 17105/asterisk
udp6 0 0 :::17917 :::* 17105/asterisk
udp6 0 0 :::18104 :::* 17105/asterisk
udp6 0 0 :::18105 :::* 17105/asterisk
udp6 0 0 :::10130 :::* 17105/asterisk
udp6 0 0 :::10131 :::* 17105/asterisk
udp6 0 0 :::10156 :::* 17105/asterisk
udp6 0 0 :::10157 :::* 17105/asterisk
udp6 0 0 :::18406 :::* 17105/asterisk
udp6 0 0 :::18407 :::* 17105/asterisk
udp6 0 0 :::10228 :::* 17105/asterisk
udp6 0 0 :::10229 :::* 17105/asterisk
udp6 0 0 :::10282 :::* 17105/asterisk
udp6 0 0 :::10283 :::* 17105/asterisk
udp6 0 0 :::14466 :::* 17105/asterisk
udp6 0 0 :::14467 :::* 17105/asterisk
udp6 0 0 :::10378 :::* 17105/asterisk
udp6 0 0 :::10379 :::* 17105/asterisk
udp6 0 0 :::18620 :::* 17105/asterisk
udp6 0 0 :::18621 :::* 17105/asterisk
udp6 0 0 :::14646 :::* 17105/asterisk
udp6 0 0 :::14647 :::* 17105/asterisk
udp6 6528 0 :::18904 :::* 17105/asterisk
udp6 0 0 :::18905 :::* 17105/asterisk
udp6 0 0 :::14962 :::* 17105/asterisk
udp6 0 0 :::14963 :::* 17105/asterisk
udp6 0 0 :::11018 :::* 17105/asterisk
udp6 0 0 :::11019 :::* 17105/asterisk
udp6 0 0 :::11156 :::* 17105/asterisk
udp6 0 0 :::11157 :::* 17105/asterisk


(Michael Asher) #2

Here is the core dump.

https://pastebin.com/0dAjPbxT


(Michael Asher) #3

Here is the full log from when the calls stopped to when fwconsole restart was ran.


(Michael Asher) #4

Here is the PCAP.

https://sync.aheliotech.com:510/shares/file/e3d5a642c28065/


(Michael Asher) #5

Task pocessors

https://pastebin.com/unEctU1T


(Michael Asher) #6

Anyone have any ideas?


(Communication Technologies) #7

I’m not sure, but wanted to recommend using the Asterisk forum as an option as well. They might be able to help if you do not get a response here.


(system) closed #8

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.