Hi, we’ve got several different freepbx servers, they were all updated to the same level, all modules are on the same version, and they are all managed identically with identical settings across the board. For some reason one of our servers has started randomly dropping 1 or 2 lines at a time out of about 130 total lines. I’ve tried installing new certs, migrating to a more powerful server and verifying that all settings are identical to our other servers. All servers are hosted at the same location, and the phones have been setup in multiple locations, the phones that drop off seem to be completely at random. In the logs I see the following error when a line drops. I’ve redacted our public ip, and domain for security reasons.
[2022-08-31 09:51:31] WARNING[6430] pjproject: SSL SSL_ERROR_SSL (Read): Level: 0 err: <336151576> len: 65535 peer: (redacted):11794
[2022-08-31 09:51:31] VERBOSE[29955] res_pjsip_registrar.c: Removed contact ‘sip:2307@(redacted):11794;transport=TLS;x-ast-orig-host=10.4.46.44:41167’ from AOR ‘2307’ due to shutdown
[2022-08-31 09:51:31] VERBOSE[21837] res_pjsip/pjsip_options.c: Contact 2307/sip:2307@(redacted):11794;transport=TLS;x-ast-orig-host=10.4.46.44:41167 has been deleted
[2022-08-31 09:51:31] VERBOSE[21837] res_pjsip/pjsip_configuration.c: Endpoint 2307 is now Unreachable
When I send an openssl -connect to our good servers on port 5061 (our pjsip port). I get a reply where there’s a tls session ticket. I also get a tls session ticket if I hit any open port on the affected server, with the exception of our pjsip port 5061, when I hit port 5061 no tls session ticket is generated. Here’s an example of a response from our good servers.
No client certificate CA names sent
Peer signing digest: SHA512
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
SSL handshake has read 6237 bytes and written 453 bytes
Verification error: self signed certificate in certificate chain
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: AC83452CC1E8071C6B86C9CB488109EBA22E2E35FA1B27F53201815E4D1E17DB
Session-ID-ctx:
Master-Key: 67C8EA4D0472000FC4D9237823A9F7DC3F76F0CD1DB5D9059C0F69E6AFB7012413D83892B55E0FAF4250DC0FEB196F91
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - de ed 0d 9b ba f5 5d 20-4f 50 86 3a b7 a3 d7 e5 …] OP.:…
0010 - 5a 4c 88 13 dc 35 20 49-9f 92 8b fd f3 99 65 c2 ZL…5 I…e.
0020 - 29 f1 72 a6 96 26 42 4b-24 24 39 21 68 5e 72 a7 ).r…&BK$$9!h^r.
0030 - fe 35 a8 06 c4 85 57 86-b9 34 98 94 06 bd be b3 .5…W…4…
0040 - 45 57 ee 61 75 a8 90 2f-96 a7 69 ab 50 5c 7d 89 EW.au…/…i.P}.
0050 - d9 02 45 70 ad 56 4c 10-e8 3f 50 3c fa 12 24 f7 …Ep.VL…?P<…$.
0060 - 49 b1 35 44 e7 cc 55 05-04 d7 44 81 50 a8 69 9c I.5D…U…D.P.i.
0070 - 68 3b 81 66 2c 6b 27 5e-55 62 53 32 df 8f f6 81 h;.f,k’^UbS2…
0080 - 62 3c bc ec a7 ab bf ab-0d 85 83 b1 58 5a 44 5f b<…XZD_
0090 - 8e 6e a7 20 56 51 94 b3-f3 39 a0 33 e6 0a 42 c9 .n. VQ…9.3…B.
00a0 - da b3 68 e7 52 45 07 45-60 34 30 3f ec 39 45 0d …h.RE.E`40?.9E.
00b0 - 65 2b bb dd 68 e5 9f de-70 d5 05 1b b1 fa 66 97 e+…h…p…f.
00c0 - 03 f7 bc ee 62 c1 8b 2c-20 de bf 3c 40 55 92 c4 …b…, …<@U…
00d0 - 34 b0 95 f4 10 10 37 5b-b5 9b e2 0b 79 da 48 b9 4…7[…y.H.
Start Time: 1661964448
Timeout : 7200 (sec)
Verify return code: 19 (self signed certificate in certificate chain)
Extended master secret: no
read:errno=0
Heres an example of a response from the server that’s having issues:
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
SSL handshake has read 28647 bytes and written 465 bytes
Verification error: self signed certificate in certificate chain
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: 370479A8BFAA774E24ABE3146C989A92D913D67FD684B4B843C67F49DDA3F6B3
Session-ID-ctx:
Master-Key: FA094B3CE2CC6D158EB56CDC9D1A1C4C30548AA6DC0A1DAE1C50DFFEC4777496C77902710DA7DDFDCD4FE8C11F6A1085
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1661964381
Timeout : 7200 (sec)
Verify return code: 19 (self signed certificate in certificate chain)
Extended master secret: no
closed
I’ve been trying to solve this for over a week and scouring the forums, I’ve updated certman, and sipsettings, I updated asterisk, I’ve tried everything I can think of but it’s a mystery to me what is going on.