Packets not being processed by Asterisk on Debian 12

Hi all,

After migrating to FPBX17 on Debian 12, we randomly notice that inbound SIP packets (INVITES/REGISTERS/…) suddenly are not being processed by Asterisk anymore.

Some more info:

  • in sngrep we see the packets arriving, but not in Asterisk
  • iptables firewall stopped & rules flushed
  • using PJSIP only
  • on Centos 7 we never saw this
  • asterisk still running and is not crashed
  • netstat still showing the listening ports

Does anyone experienced the same issue?

Thanks for helping out!

Did iptables have any forwarding rules when it was running ?

tcpdump, unlike sngrep, doesn’t filter anything you don’t ask it to

Negative… no forwarding rules setup.

When I fire the following command during the issue:

netstat -tulpn | grep asterisk

I get the following output (yes, we listen on 5588):

udp 427776 0 0.0.0.0:5588 0.0.0.0:* 2254422/asterisk

So this big number is the receive queue (“Recv-Q”) for the UDP socket… sitting in the kernel buffer waiting for Asterisk to read them.

Anyone has seen this before?

tcpdump will identify every packet going through the interface chosen, if that doesnt resolve the issue use wireshark (or tshark) to resolve any protocol issue, netstat’s utility is better found nowadays in ss

What do you mean with “if that doesn’t resolve the issue?”

I do see the traffic in sngrep & tcpdump (tcpdump -nn -i ens18 udp port 5588)…
Asterisk is just not processing the traffic.

But is Asterisk actually getting the traffic destined for UDP:5588

lsof -i 5588

would identify the ‘listener’ if any of such traffic (any protocol)

Yes sir (added a colon before the port):

lsof -i :5588
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
asterisk 2254422 asterisk 20u IPv4 42493713 0t0 UDP *:5588
asterisk 2254422 asterisk 21u IPv4 42496004 0t0 TCP *:5588 (LISTEN)

Then /var/log/asterisk/full should by default reflect any RX/TX activity of a coincident sngrep log to the box

When doing an inbound call for example, I see the INVITE in sngrep/tcpdump… but not in asterisk -rvvvvvvvv nor full log…

The Recv-Q is also not going down. So Asterisk is not reading them.

The trouble is we have seen you say you are getting the INVITE but you are not posting where the INVITE was sent to, in sngrep you can simply use the arrow keys to navigate and ‘drill down’ to what any transaction in the initiated protocol is doing, In your case, where the INVITE is being sent to.

On my FPBX instance (Debian 12) I’m running sngrep…
There I see an INVITE coming in from my trunk. But Asterisk is not processing it.

the arrow buttons will help you, until the INVITES get to the server, you have no chance

this screenshot is taken from the server itself? So they are arriving there…

Sounds like a deadlock.

https://docs.asterisk.org/Development/Debugging/Getting-a-Backtrace/#running-ast_coredumper-for-deadlocks-taskprocessor-backups-etc

Pretty well “No” to that, the content of the packet will say where it was sent (you didn’t inlude it) , the absense of a ‘green’ respose suggests that it was not sent to a device that replied . . .

Though I agree with @david55 about the deadlock, another possibility is that the firewall is not being properly turned off, or is getting turned back on by another process. At a time when Asterisk is not responding, at a root shell prompt type
iptables -vL
nft list ruleset
and post the output.

Sorry to say, but the screenshot is from sngrep running on the actual machine.
Meaning the packet does arrive on the actual box. Not going to include IP’s here on a public forum.

I already tried flushing the rules:

iptables -vL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

nft list ruleset

→ returns empty

Ok, will run the following cmd next time it happens:

/var/lib/asterisk/scripts/ast_coredumper --running --no-default-search --pid=XXX

Stupid question: What should I do with it afterwards? :slight_smile: