FreePBX | Register | Issues | Wiki | Portal | Support

Removal of Asterisk 13.26.0-1 and Asterisk 15.7.2-1


(Andrew Nagy) #1

Hello!

Quick update for you all, we’ve removed Asterisk 13.26.0-1 and 15.7.2-1 from our SNG7 repositories today after getting numerous complaints about Asterisk instability. We want to assure you that we do test all Asterisk instances but sometimes issues like these will slip through our own QA channel.

To downgrade your Asterisk 13 simply run yum downgrade asterisk13* this will place you on Asterisk 13.22

That said we think we have identified the issue and have released 13.26-3 and 15.7.2-3 to our testing repository. If you want to check this version of Asterisk simply run yum install sangoma-devel && yum upgrade and you’ll be placed on the version of Asterisk that we believe fixes this issue. If this does not work for you feel free to just run yum downgrade asterisk13* to be placed on a working version

Note This only affects the FreePBX Distro


Asterisk 13.26.0-1.sng7 hangs repeatedly shows SIP extentions still connected
SIP breaking in asterisk13-core-13.26.0
FreePBX 14.0.8.4, asterisk crashing, system is updated
Asterisk Freezes Hard after update problems
Weirdest problem ever (Aastra 6867i)
System stops responding every 3-5 days
Many incoming routes
How to Diagnose Strange Failure
Fresh SNG7 install, DAHDI fails to load
Intermittent IVR failure
Fclose 110 is not a valid stream resource
Operating System Updates 4/23/2019 never completes
FREE PBX backup and restore on another PBX
Applied updates now I have a flame next to asterisk on dashboard
⚠ Issue on all FreePBX updated - not working with Bria Push Service
TRUNK Dial failed due to CHANUNAVAIL HANGUPCAUSE: 1 - failing through to other trunks
FreePBX Randomly Stops processing calls
FreePBX becomes unresponsive every 5-6 days
All Extentions go unreachable once a day
Asterisk periodically unresponsive, only fix is reboot PBX
Server stopped working today
After Update Asterisk Hangs and Won't Shutdown
(Andrew Nagy) pinned globally #2

(Cyberdocwi) #3

Hello Andy,

Thank you for doing this: Making an obvious post about it, and immediate resolution on how to repair. One of the challenges I had when experiencing our failures was the proper search terms – asterisk crash wasn’t the best description to search against.

Best,
Christian


(Dave Burgess) #4

Sorry - that made me laugh. Searching for “Asterisk Crash” is a little like searching for “the internet”.


(Cyberdocwi) #5

That is quite alright. The computer itself didn’t lock up, or stop responding to pings. Linux was fully useful. The logs for me just stopped – /var/log/asterisk/full just stopped recording items, so I figured it was the asterisk process that died. FreePBX web management worked fine in the areas I looked, but SIPSTATION wouldn’t report any status details. Yet in the asterisk console (in linux), sip show connections revealed active SIP connections and links. My OpenNMS server wasn’t catching any traps, or detecting any issues. Then again, I haven’t deeply integrated OpenNMS into Asterisk yet, but SNMP, postfix, httpd, and all the other friends were actively working.

I was quite stuck on trying to construct a useful search string.


#6

Our phones in the store have been going offline ever since the install of these upgrades. I didn’t know what the issue was and had to keep rebooting the computer. It was cutting our customers off. There was a message in the log about some task reaching its limit. I’ve been wasting hours on this. I hope this back-out fixes it.


(Andrew Nagy) #7

FYI We identified this issue and it is resolved. It was resolved 3 days ago as previously stated

That said we think we have identified the issue and have released 13.26-3 and 15.7.2-3 to our testing repository.


#8

We are still experiencing this issue on a variety of customer servers and VPS’s.
We have now switched to Asterisk 13 certified on four problematic systems to see if this resolves the issue.
yum install sangoma-devel
yum upgrade asterisk-version-switch
asterisk-version-switch


(Jared Busch) #9

Yes, and you had what Asterisk versions? Why Asterisk 13 certified? Why not Asterisk 16. It is current and LTS. Lots of opaque logic here.

You complain that something broken, which was fixed, is still broken. Yet you provide zero information


#10

Hi Jared:
I’m sorry that my prior post didn’t satisfy your curiosity.
We had recently rolled out a new customer with a Cloud-Switch delivering calls to three office PBX’s with a unified call plan.
After updated the Cloud-VPS together with the three premises PBX’s to the latest releases, we were suffering from random Asterisk lockup.
This update included Asterisk 13.26-2 which as you know, was the LTS Version
This new LTS release, was causing systems that had previously been working brilliantly to become unresponsive.

We use a Custom Destination with alarms to notify us of any incoming call failures on inbound routes to our customer prem PBX’s so we were notified at the first undelivered call.

SAR showed that the systems were being starved of processor resources to the point they because un-responsive.
02:00:02 PM all 28.77 0.00 30.89 0.03 0.00 40.31
02:10:01 PM all 28.74 0.00 30.92 0.03 0.00 40.31
02:20:01 PM all 30.96 0.00 31.15 0.03 0.00 37.87
02:30:02 PM all 33.65 0.00 31.49 0.03 0.00 34.83
02:40:01 PM all 29.02 0.00 30.92 0.03 0.00 40.04
02:50:02 PM all 30.17 0.00 31.00 0.03 0.00 38.80
03:00:02 PM all 31.10 0.00 31.22 0.03 0.00 37.66
03:10:01 PM all 30.17 0.00 31.18 0.03 0.00 38.62
03:20:01 PM all 29.85 0.00 30.80 0.04 0.00 39.31
03:30:01 PM all 29.05 0.00 30.93 0.03 0.00 40.00
03:40:01 PM all 33.82 0.00 31.39 0.03 0.00 34.77
03:50:02 PM all 29.47 0.00 30.93 0.03 0.00 39.57
04:00:01 PM all 35.43 0.00 30.43 0.03 0.00 34.12
04:10:01 PM all 37.80 0.00 30.36 0.02 0.00 31.82
04:20:13 PM all 24.38 0.00 70.66 0.00 0.00 4.95
04:30:06 PM all 22.07 0.00 77.93 0.00 0.00 0.00

Our monitoring system would then have to execute a remote call to reboot the PBX.

04:36:20 PM LINUX RESTART

Rebooting appeared to be the only way to “break” the Asterisk lock up.
Everything was tried, fwconsole restart…… no joy
Killall -9 would not kill all of the cascaded running instances…


Commands to run for Diagnostics when Asterisk crashes again.
• ps auxffww > /root/process.txt 2>&1
• iostat > /root/iostat.txt 2>&1
• lsof > /root/lsof.txt 2>&1
• dmesg > /root/dmesg.txt 2>&1
• netstat -anp > /root/netstat-anp.txt 2>&1
Then run a packet capture:
• touch /root/asterisk-manager.pcap && tshark -i eth0 -f ‘tcp port 5038’ -w /root/asterisk-manager.pcap
Cancel the packet capture after it’s got at least 100 packets If after 2 minutes, it hasn’t captured any packets, please make note of that.
Ctrl-C should stop the packet capture
Then run the following:
tar jcvf /root/asterisk-crash-date -I.tar.bz2 /root/.txt /root/.pcap && rm -f /root/.txt /root/.pcap


We ran comprehensive diagnostics in an attempt to understand the problem but it kept coming back to a bug in the actual Asterisk LTS release.

I know Andrew recommended running the yum downgrade asterisk13*
However, I was enamoured by Andrew’s post of https://www.freepbx.org/sangoma-7-6-distro-ga/
Sangoma 7.6 Distro GA
Posted on May 10, 2019 by Andrew Nagy
“One more thing: Have you always longed for the day where you could install Asterisk Certified on Sangoma 7 and utilize it with FreePBX?
Well, today’s your day! If you’d like to test out Asterisk Certified on your Sangoma 7 FreePBX system you’ll just need to run the following commands (note this is only in the testing repo):”
yum install sangoma-devel
yum upgrade asterisk-version-switch
asterisk-version-switch
Then select Asterisk 13 Certified
So now we are happily running the Certified Asterisk 13.21.0 with happy customers and a happier support team.

Don.


(Aaron) #11

@tm1000 Is the issue present in 13.23? Or did you just go down to 13.22 to be safe?


(Lorne Gaetz) #12

This issue is only present in the distro rpm:

asterisk13.x86_64            13.26.0-1.sng7                         sng7-testing

Any version of 13.26 you download today will be newer than that, and does not have this issue. No other versions of 13 were affected.