Phones keeps cutting in and out, any ideas?

I am having issues where the phones keep cutting in and out. We are getting the following errors in the log file.

Can anyone help?

[2015-01-09 13:02:00] VERBOSE[23558] features.c: == Spawn extension (macro-dial-one, h, 1) exited non-zero on ‘SIP/403-0000977c’
[2015-01-09 13:02:00] VERBOSE[23558] app_macro.c: == Spawn extension (macro-dial-one, s, 42) exited non-zero on ‘SIP/403-0000977c’ in macro ‘dial-one’
[2015-01-09 13:02:00] VERBOSE[23558] app_macro.c: == Spawn extension (macro-exten-vm, s, 7) exited non-zero on ‘SIP/403-0000977c’ in macro ‘exten-vm’
[2015-01-09 13:02:00] VERBOSE[23558] pbx.c: == Spawn extension (from-internal, 108, 2) exited non-zero on ‘SIP/403-0000977c’
[2015-01-09 13:02:18] VERBOSE[3498] chan_sip.c: – Registered SIP ‘406’ at 192.168.0.116:5060
[2015-01-09 13:02:18] NOTICE[3498] chan_sip.c: Peer ‘406’ is now Reachable. (66ms / 2000ms)
[2015-01-09 13:02:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘307’ at 192.168.0.126:5060
[2015-01-09 13:02:19] NOTICE[3498] chan_sip.c: Peer ‘307’ is now Reachable. (64ms / 2000ms)
[2015-01-09 13:02:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘800’ at 192.168.0.225:5060
[2015-01-09 13:02:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘306’ at 192.168.0.96:5060
[2015-01-09 13:02:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘500’ at 192.168.0.191:5060
[2015-01-09 13:02:19] NOTICE[3498] chan_sip.c: Peer ‘800’ is now Reachable. (70ms / 2000ms)
[2015-01-09 13:02:19] NOTICE[3498] chan_sip.c: Peer ‘306’ is now Reachable. (69ms / 2000ms)
[2015-01-09 13:02:19] NOTICE[3498] chan_sip.c: Peer ‘500’ is now Reachable. (70ms / 2000ms)
[2015-01-09 13:02:19] VERBOSE[23552] file.c: – <SIP/2424560312-0000977b> Playing ‘vm-intro.ulaw’ (language ‘en’)
[2015-01-09 13:02:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘309’ at 192.168.0.160:5060
[2015-01-09 13:02:19] NOTICE[3498] chan_sip.c: Peer ‘309’ is now Reachable. (68ms / 2000ms)
[2015-01-09 13:02:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘303’ at 192.168.0.158:5060
[2015-01-09 13:02:19] NOTICE[3498] chan_sip.c: Peer ‘303’ is now Reachable. (62ms / 2000ms)
[2015-01-09 13:02:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘100’ at 192.168.0.64:5060
[2015-01-09 13:02:19] NOTICE[3498] chan_sip.c: Peer ‘100’ is now Reachable. (70ms / 2000ms)
[2015-01-09 13:02:24] VERBOSE[23552] file.c: – <SIP/2424560312-0000977b> Playing ‘beep.ulaw’ (language ‘en’)
[2015-01-09 13:02:25] VERBOSE[23552] app_voicemail.c: – Recording the message
[2015-01-09 13:02:25] VERBOSE[23552] app.c: – x=0, open writing: /var/spool/asterisk/voicemail/default/100/tmp/Rmiz6b format: wav49, 0x4914ed8
[2015-01-09 13:02:25] VERBOSE[23552] app.c: – x=1, open writing: /var/spool/asterisk/voicemail/default/100/tmp/Rmiz6b format: wav, 0x482c918
[2015-01-09 13:04:20] NOTICE[3498] chan_sip.c: Peer ‘101’ is now UNREACHABLE! Last qualify: 16
[2015-01-09 13:04:29] NOTICE[3498] chan_sip.c: Peer ‘101’ is now Reachable. (16ms / 2000ms)
[2015-01-09 13:06:18] VERBOSE[3498] chan_sip.c: – Registered SIP ‘104’ at 192.168.0.226:5060
[2015-01-09 13:06:18] VERBOSE[3498] chan_sip.c: – Registered SIP ‘407’ at 192.168.0.181:5060
[2015-01-09 13:06:18] NOTICE[3498] chan_sip.c: Peer ‘104’ is now Reachable. (47ms / 2000ms)
[2015-01-09 13:06:18] NOTICE[3498] chan_sip.c: Peer ‘407’ is now Reachable. (47ms / 2000ms)
[2015-01-09 13:06:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘404’ at 192.168.0.230:5060
[2015-01-09 13:06:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘305’ at 192.168.0.222:5060
[2015-01-09 13:06:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘403’ at 192.168.0.223:5060
[2015-01-09 13:06:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘405’ at 192.168.0.220:5060
[2015-01-09 13:06:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘304’ at 192.168.0.214:5060
[2015-01-09 13:06:19] NOTICE[3498] chan_sip.c: Peer ‘404’ is now Reachable. (56ms / 2000ms)
[2015-01-09 13:06:19] NOTICE[3498] chan_sip.c: Peer ‘305’ is now Reachable. (54ms / 2000ms)
[2015-01-09 13:06:19] NOTICE[3498] chan_sip.c: Peer ‘403’ is now Reachable. (55ms / 2000ms)
[2015-01-09 13:06:19] NOTICE[3498] chan_sip.c: Peer ‘405’ is now Reachable. (55ms / 2000ms)
[2015-01-09 13:06:19] NOTICE[3498] chan_sip.c: Peer ‘304’ is now Reachable. (56ms / 2000ms)
[2015-01-09 13:06:19] VERBOSE[3498] chan_sip.c: – Registered SIP ‘103’ at 192.168.0.217:5060
[2015-01-09 13:06:19] NOTICE[3498] chan_sip.c: Peer ‘103’ is now Reachable. (39ms / 2000ms)
[2015-01-09 13:07:08] VERBOSE[3498] chan_sip.c: – Registered SIP ‘406’ at 192.168.0.116:5060
[2015-01-09 13:07:08] NOTICE[3498] chan_sip.c: Peer ‘406’ is now Reachable. (66ms / 2000ms)
[2015-01-09 13:07:09] VERBOSE[3498] chan_sip.c: – Registered SIP ‘307’ at 192.168.0.126:5060
[2015-01-09 13:07:09] NOTICE[3498] chan_sip.c: Peer ‘307’ is now Reachable. (69ms / 2000ms)
[2015-01-09 13:07:09] VERBOSE[3498] chan_sip.c: – Registered SIP ‘800’ at 192.168.0.225:5060
[2015-01-09 13:07:09] VERBOSE[3498] chan_sip.c: – Registered SIP ‘306’ at 192.168.0.96:5060
[2015-01-09 13:07:09] VERBOSE[3498] chan_sip.c: – Registered SIP ‘500’ at 192.168.0.191:5060
[2015-01-09 13:07:09] NOTICE[3498] chan_sip.c: Peer ‘800’ is now Reachable. (71ms / 2000ms)
[2015-01-09 13:07:09] NOTICE[3498] chan_sip.c: Peer ‘306’ is now Reachable. (69ms / 2000ms)
[2015-01-09 13:07:09] NOTICE[3498] chan_sip.c: Peer ‘500’ is now Reachable. (71ms / 2000ms)
[2015-01-09 13:07:09] VERBOSE[3498] chan_sip.c: – Registered SIP ‘309’ at 192.168.0.160:5060
[2015-01-09 13:07:09] NOTICE[3498] chan_sip.c: Peer ‘309’ is now Reachable. (68ms / 2000ms)
[2015-01-09 13:07:09] VERBOSE[3498] chan_sip.c: – Registered SIP ‘100’ at 192.168.0.64:5060
[2015-01-09 13:07:09] NOTICE[3498] chan_sip.c: Peer ‘100’ is now Reachable. (70ms / 2000ms)
[2015-01-09 13:07:09] VERBOSE[3498] chan_sip.c: – Registered SIP ‘303’ at 192.168.0.158:5060
[2015-01-09 13:07:09] NOTICE[3498] chan_sip.c: Peer ‘303’ is now Reachable. (61ms / 2000ms)
[2015-01-09 13:08:04] VERBOSE[3498] chan_sip.c: – Registered SIP ‘201’ at 192.168.0.206:5060
[2015-01-09 13:08:04] VERBOSE[3498] chan_sip.c: – Registered SIP ‘401’ at 192.168.0.221:5060
[2015-01-09 13:08:04] VERBOSE[3498] chan_sip.c: – Registered SIP ‘101’ at 192.168.0.201:5060
[2015-01-09 13:08:04] VERBOSE[3498] chan_sip.c: – Registered SIP ‘108’ at 192.168.0.203:5060
[2015-01-09 13:08:04] NOTICE[3498] chan_sip.c: Peer ‘201’ is now Reachable. (14ms / 2000ms)
[2015-01-09 13:08:04] VERBOSE[3498] chan_sip.c: – Registered SIP ‘301’ at 192.168.0.211:5060
[2015-01-09 13:08:04] NOTICE[3498] chan_sip.c: Peer ‘401’ is now Reachable. (15ms / 2000ms)
[2015-01-09 13:08:04] NOTICE[3498] chan_sip.c: Peer ‘101’ is now Reachable. (15ms / 2000ms)
[2015-01-09 13:08:04] NOTICE[3498] chan_sip.c: Peer ‘108’ is now Reachable. (15ms / 2000ms)
[2015-01-09 13:08:04] NOTICE[3498] chan_sip.c: Peer ‘301’ is now Reachable. (15ms / 2000ms)
[2015-01-09 13:12:12] VERBOSE[3498] netsock2.c: == Using SIP RTP TOS bits 184
[2015-01-09 13:12:12] VERBOSE[3498] netsock2.c: == Using SIP RTP CoS mark 5

You have a network problem. it’s a simple as that. You could start off by turning off ALL firewalls between your server and your phones.

Are the phones and phone system on the same subnet?
What is the time span between when they become reachable and when they go off line again?
Do your trunks go down as well?
Do all the phone become unavailable at the same time or are some of them still reachable?
When the phones are working is the sound quality poor, does the audio cut in and out?

You may have a switch problem, but without more info it is hard to tell.

The phones are in the same subnet
Not all the phones go down.
They go down for ten seconds
I was just told the audio quality is bad, sometime they hear an echo.

Any ideas?

It sounds like congestion on your network. You will need to diagnose that, start with tcpdump and/or iptraf on your server.

Thanks for the update, here’s what i just noticed on the log

[2015-01-13 12:42:08] NOTICE[3498] chan_sip.c: Peer ‘305’ is now UNREACHABLE! Last qualify: 54
[2015-01-13 12:42:08] NOTICE[3498] chan_sip.c: Peer ‘405’ is now UNREACHABLE! Last qualify: 56
[2015-01-13 12:42:08] NOTICE[3498] chan_sip.c: Peer ‘303’ is now UNREACHABLE! Last qualify: 57
[2015-01-13 12:42:08] NOTICE[3498] chan_sip.c: Peer ‘304’ is now UNREACHABLE! Last qualify: 57
[2015-01-13 12:42:08] NOTICE[3498] chan_sip.c: Peer ‘403’ is now UNREACHABLE! Last qualify: 56
[2015-01-13 12:42:08] NOTICE[3498] chan_sip.c: Peer ‘404’ is now UNREACHABLE! Last qualify: 57
[2015-01-13 12:42:08] NOTICE[3498] chan_sip.c: Peer ‘500’ is now UNREACHABLE! Last qualify: 57
[2015-01-13 12:42:08] NOTICE[3498] chan_sip.c: Peer ‘306’ is now UNREACHABLE! Last qualify: 57
[2015-01-13 12:42:18] NOTICE[3498] chan_sip.c: Peer ‘403’ is now Reachable. (56ms / 2000ms)
[2015-01-13 12:42:18] NOTICE[3498] chan_sip.c: Peer ‘305’ is now Reachable. (54ms / 2000ms)
[2015-01-13 12:42:18] NOTICE[3498] chan_sip.c: Peer ‘405’ is now Reachable. (55ms / 2000ms)
[2015-01-13 12:42:18] NOTICE[3498] chan_sip.c: Peer ‘404’ is now Reachable. (55ms / 2000ms)
[2015-01-13 12:42:18] NOTICE[3498] chan_sip.c: Peer ‘304’ is now Reachable. (57ms / 2000ms)
[2015-01-13 12:42:18] NOTICE[3498] chan_sip.c: Peer ‘303’ is now Reachable. (57ms / 2000ms)
[2015-01-13 12:42:18] NOTICE[3498] chan_sip.c: Peer ‘500’ is now Reachable. (58ms / 2000ms)
[2015-01-13 12:42:18] NOTICE[3498] chan_sip.c: Peer ‘306’ is now Reachable. (58ms / 2000ms)

As I suggested, you could:-

If you’re losing connection to everything (phones internally, and perhaps trunks externally) it could be a spanning tree issue. Do a constant ping to an inside host and another to an outside host (if windows, ping -t and on *nix, ping ). Let it run until you see the peers drop and check if you’re losing packets internally AND externally. If so, you are the victim of a spanning tree flapping issue. Always fun…

Do you have multiple switches?
If so, are the phone that go down on all on the same switch.

Are you using the same switch for voice and data traffic?
If you are, are your users moving large files around on the network?

You could have a bottleneck on you trunk ports between switches.
Can you check the port utilization on the switches?

You could also have a problem with one of your switches.
Can you check the CPU utilization on the switches?
Can you get an up-time report on the switch, maybe it is resetting.

You could also plug a laptop into one of the network cables from an affected phone. Ping the server and see if you have latency above 1ms. You shouldn’t have high latency because you are on a local network. Also check to see if you are dropping packets.

Good luck on your trouble shooting.