All Phones Become "Unregistered"

General Help
#1

Hey all,

I am in a bit of a pickle. Since installing FreePBX, we have had a recurring issue where at a random time, all the phones will become “Unregistered” from the PBX. Anyone on the phone can continue the call, but all new in/outbound calls are denied. All of our phones are Polycom. the call center has 75 IP 331 phones, and admin has 30 VVX 410 phones. Asterisk is version 11.7.0 and FreePBX is 2.11.0.13

This issue happens at random, whether we are busy or slow, at night, after hours, or in the middle of the day. I have actively been looking at the asterisk -rvvv screen when it happens and it doesn’t show anything at all, except the people still trying to make calls. It is not gradual either, when the problem appears, all phones go offline all at once (besides those actively talking).

To date, the only way to bring the system back up is to do an amportal stop and then amportal start, after everyone is done with their call. Occasionally, we also have to stop/start MySQL too, but that is maybe 1 out of 10 times it crashes.

My network is pretty new and uses decent equipment. Phones connect to 1 of 3 Cisco Meraki POE Gigabit Switches. Those Meraki connect to a Cisco 3750 Core. The PBX is connected to that 3750 as well. We use VLans so the phones are on a 10.100.x.x and data is on 10.1.x.x and phones are provisioned via DHCP and Commercial EPM. The PBX server itself is a 3 yr old Dell with 2x Quad-Core Zeon PCU @ 2.5GHz, 72GB RAM, 2x 500GB 15k drives and 2x GB NIC cards. One connects directly to voice circuit the other to my network.

I have seen posts from people dating back to 2012 with similar issues, but not of them see to have a posted resolution… a Network Engineer friend has been looking at the polycom config files to see if there may be something in them causing it, but as of yet, hasn’t found anything. So while I wait for him, I figured I would post here.

#2

Couple of things.

Install MTY (yum install mtr) and then mty ip.address.of.phone Leave it up for a day and see what kind of packet loss you have.

Are you using the same network drop for data/voice and letting the phone do the untagging? I have seen broadcast storms do this before. If the phones are on trunk ports I would put one on an access port on the voice VLAN and see if it has the same issue.

#3

Tony, No, all the trunks are using IPv4 definitions.

SkyKing, I will give that a try… I can use any phone’s IP, right? And yes, All the phones share the same drop as the PCs. We were analog up until 2 yrs ago, and rerunning 100+ drops with a 2nd Cat5 was going to be $$$$$. All ports on the Meraki switches are set to Access by default. Only Port 48 to the Core is set up as Trunk.

#4

Sounds like Asterisk is locking up. Almost like the old DNS issue. Are you using FQDN for any of your trunks?

#5

Ok, if they are set to access you must be using some type of voice vlan command to enable 802.1q tags. I am saying an access port set to the voice vlan as the pvid.

#6

So we took a look at the Meraki switches, and QoS is enabled, but it wasn’t explicitly looking at our voice VLan. Rather, it was just looking for the SIP protocols. We just made a change so it looks for the Voice VLan.

Also, not sure if it helps more or not, but when the phones “unregister”, the admin web console also becomes unavailable…

#7