Hey Not sure if this helps, but it seems to drop the extensions one by one.
this is from the full log in /var/log/asterisk/
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[117720] res_pjsip/pjsip_options.c: Contact 228/sip:[email protected]:5060 has been deleted
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[117720] res_pjsip/pjsip_configuration.c: Endpoint 228 is now Unreachable
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[113560] res_pjsip/pjsip_options.c: Contact 236/sip:[email protected]:5060 has been deleted
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[113560] res_pjsip/pjsip_configuration.c: Endpoint 236 is now Unreachable
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[105197] res_pjsip/pjsip_options.c: Contact 237/sip:[email protected]:5060 has been deleted
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[105197] res_pjsip/pjsip_configuration.c: Endpoint 237 is now Unreachable
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[153253] res_pjsip/pjsip_options.c: Contact 243/sip:[email protected]:5060 has been deleted
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[153253] res_pjsip/pjsip_configuration.c: Endpoint 243 is now Unreachable
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[95294] res_pjsip/pjsip_options.c: Contact 247/sip:[email protected]:5060 has been deleted
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[95294] res_pjsip/pjsip_configuration.c: Endpoint 247 is now Unreachable
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[113560] res_pjsip/pjsip_options.c: Contact 251/sip:[email protected]:5060 has been deleted
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[113560] res_pjsip/pjsip_configuration.c: Endpoint 251 is now Unreachable
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[153253] res_pjsip/pjsip_options.c: Contact 254/sip:[email protected]:5060 has been deleted
/var/log/asterisk/full:[2021-03-23 13:00:11] VERBOSE[153253] res_pjsip/pjsip_configuration.c: Endpoint 254 is now Unreachable
Server is on the 20 vlan ( or what we call “voice vlan”)
All in office desk phones are on this network. Server has access to 20 and 21 vlan.
192.168.20.5/255.255.255.0
Soft phones for people working remotely are on 21 vlan. These have access to all ports the sever.
192.168.21.5/255.255.255.0
That uncauses a resource exhaustion, which this doesn’t really sound like. With all of the extensions losing connection at the same time, I’d be suspecting hardware, especially something like a bad switch or wonky power. If you are using POE from the switch, for example, I’d be really likely to look in that direction.
I will have a look at the switch logs to see if they are erroring out.
Checked the firewall and didn’t see and switch to the backup firewall or errors.
These are the only other warning / errors i have seen around the time of the crash.
[2021-04-06 16:20:55] WARNING[160839][C-000002cd] taskprocessor.c: The ‘subp:ast_channel_topic_all-000004bb’ task processor queue reached 500 scheduled tasks.
[2021-04-06 16:21:29] ERROR[14260] res_pjsip_header_funcs.c: No headers had been previously added to this session.
This is a symptom of the problem you solved with the MWI subscription type.
This, on the other hand, looks like a TCP failure. The most likely cause of that (that I can think of) would be the firewall resetting the TCP connections, a time-out somewhere, or the phones restarting the connections and the server trying to pick up in the middle of a conversation. All three of those could be caused by a bad switch module or a stateful firewall/switch losing its mind and forgetting who your phones are. Of course, as with most things IP, it could also be squirrels or a coincidence.
The more we talk, the more it sounds like bad infrastructure. I could be completely all wet on that, but I’d really like to suggest starting from the network core and working your way out.
At the beginning you say you are running Asterisk 13 - any reason you are not running version 16? I’m not sure how well pjsip works on the old Asterisk 13.