How to set up dual NICs with pjsip?

I have a working (version 14) freepbx system that is currently working. For reasons, I need to add a second NIC for a different LAN. Both NICs have public-internet-facing ip addresses, both are fully routable.

The setup was simple enough to get a second NIC’s internet responding, after a minimal amount of google:

  1. added an entry to /etc/iproute2/rt_tables (ie. “200 second”)
  2. add a default route to that table named “second” (ie. “ip route add default via table second”
  3. add a rule to select that route for source ip (ie. “ip route add from <eth1’s-ip> table second”)

After doing all that, then the server was able to respond to both ip addresses. I didn’t even have to restart ssh and could connect via the secondary ip; I could ping; httpd; etc.

I have verified that the pbx is at least purportedly LISTENING for connections on all interfaces via lsof:

asterisk 18106 asterisk 17u IPv4 156106016 0t0 UDP *:sip

but when I set up a sip client for an extension (all my extensions are pjsip), then it just times out. Nothing gets logged in asterisk. From what I can see, asterisk SHOULD be listening on all interfaces for sip connections (port 5060), but … nothing.

Help?

Are you running a firewall on the PBX? If so, probably need to update it for the new interface.

I’ve got some pretty strict rules on the server, yes. But those rules already explicitly allow all traffic with source ips in my local subnets.

Do REGISTER requests show in sngrep when an extension attempts to register? If so, it’s almost certain that FreePBX firewall is blocking them. Look at the active iptables rules.

If not, possibly a routing or firewall issue unrelated to FreePBX?

Part of why I’m confused is, I’ve already got another freepbx (running a slightly earlier version) that has dual NICs… AND is working. The biggest difference I can see between them is, this second server’s second nic is using a non-routed private ip in 10.x.x.x, and so only devices already in that subnet are connecting to that ip.

It (the working dual-nic server) doesn’t have ANY settings in it whatsoever in Settings / Asterisk SIP Settings that even mention the 10.x.x.x subnet

Very literally, the very first two entries in my firewall are:

ACCEPT tcp – .0/22 0.0.0.0/0
ACCEPT udp – .0/22 0.0.0.0/0

where both the freepbx server and the client device are within that /22 subnet.

I’d previously never heard of “sngrep”. But running that while my sip client tries to connect to the ip on eth1:

[ ] 5 REGISTER -ext-@-eth1-ip-address- -ext-@-eth1-ip-address- 16

OK, so either iptables is blocking the REGISTER, or the reply is being blocked or misrouted. At the Asterisk command prompt, type
pjsip set logger on
and see whether REGISTER requests are received and if so, whether replies are being sent to the correct address and port.

You mentioned that you did it without restarting anything…

Does Asterisk know about this IP?

With the most-sensitive info stripped:

[2021-06-15 14:16:53] VERBOSE[18184] res_pjsip_logger.c: <--- Received SIP request (544 bytes) from UDP:public.ip.of.client:13124 --->
REGISTER sip:ip.address.for.eth1 SIP/2.0
Via: SIP/2.0/UDP private.ip.of.client:8278;branch=z9hG4bKMLQ6o27vfsHGxv7c;rport
Contact: <sip:[email protected]:8278;rinstance=62B29F5C>;expires=600;video
Max-Forwards: 70
From: <sip:[email protected]>;tag=402005BCB37C61507E3EC7DCF75DAAE8
Allow: ACK, BYE, CANCEL, INFO, INVITE, NOTIFY, OPTIONS, REFER
Supported: replaces, path
To: <sip:[email protected]>
Expires: 600
Call-ID: B7A01D51F6EF8A6E772E0A46729BD80AF7DE98BD
CSeq: 4270 REGISTER
User-Agent: Acrobits Softphone Business/3.8.26
Content-Length: 0


[2021-06-15 14:16:53] VERBOSE[20755] res_pjsip_logger.c: <--- Transmitting SIP response (527 bytes) to UDP:public.ip.of.client:13124 --->
SIP/2.0 401 Unauthorized
Via: SIP/2.0/UDP private.ip.of.client:8278;rport=13124;received=public.ip.of.client;branch=z9hG4bKMLQ6o27vfsHGxv7c
Call-ID: B7A01D51F6EF8A6E772E0A46729BD80AF7DE98BD
From: <sip:[email protected]>;tag=402005BCB37C61507E3EC7DCF75DAAE8
To: <sip:[email protected]>;tag=z9hG4bKMLQ6o27vfsHGxv7c
CSeq: 4270 REGISTER
WWW-Authenticate: Digest realm="asterisk",nonce="1623781013/0ee434b433ec8c90f9afe20f1f6ab74c",opaque="7f07a1db6c47f125",algorithm=md5,qop="auth"
Server: FPBX-14.0.16.4(13.38.1)
Content-Length:  0


[2021-06-15 14:16:55] VERBOSE[18184] res_pjsip_logger.c: <--- Received SIP request (544 bytes) from UDP:public.ip.of.client:13124 --->
REGISTER sip:ip.address.for.eth1 SIP/2.0
Via: SIP/2.0/UDP private.ip.of.client:8278;branch=z9hG4bKMLQ6o27vfsHGxv7c;rport
Contact: <sip:[email protected]:8278;rinstance=62B29F5C>;expires=600;video
Max-Forwards: 70
From: <sip:[email protected]>;tag=402005BCB37C61507E3EC7DCF75DAAE8
Allow: ACK, BYE, CANCEL, INFO, INVITE, NOTIFY, OPTIONS, REFER
Supported: replaces, path
To: <sip:[email protected]>
Expires: 600
Call-ID: B7A01D51F6EF8A6E772E0A46729BD80AF7DE98BD
CSeq: 4270 REGISTER
User-Agent: Acrobits Softphone Business/3.8.26
Content-Length: 0


[2021-06-15 14:16:55] VERBOSE[4371] res_pjsip_logger.c: <--- Transmitting SIP response (527 bytes) to UDP:public.ip.of.client:13124 --->
SIP/2.0 401 Unauthorized
Via: SIP/2.0/UDP private.ip.of.client:8278;rport=13124;received=public.ip.of.client;branch=z9hG4bKMLQ6o27vfsHGxv7c
Call-ID: B7A01D51F6EF8A6E772E0A46729BD80AF7DE98BD
From: <sip:[email protected]>;tag=402005BCB37C61507E3EC7DCF75DAAE8
To: <sip:[email protected]>;tag=z9hG4bKMLQ6o27vfsHGxv7c
CSeq: 4270 REGISTER
WWW-Authenticate: Digest realm="asterisk",nonce="1623781015/f2c02b5bb720162dac9f84342252c313",opaque="5d4baf687a289089",algorithm=md5,qop="auth"
Server: FPBX-14.0.16.4(13.38.1)
Content-Length:  0

If it is a private IP, nothign is sensative.

While I haven’t rebooted the whole server since setting up the ip address, I have indeed restarted the asterisk service itself.

The ip.address.of.eth1 and public.ip.of.client are both real-world routed ip addresses, so yup I’m redacting those.

OK, so the client didn’t ‘hear’ the 401 and just kept retransmitting the REGISTER without an Authorization header. AFAICT Asterisk sent it to the correct IP and port. Possibilities:

  1. Outgoing packet (the 401) blocked by iptables.
  2. OS level routing issue (went out wrong interface, not sent to proper gateway, etc.)
  3. Blocked by hardware firewall in path back to client.
  4. Client ignored the reply, e.g. because it was mangled by a SIP ALG in the path.

Expand the REGISTER in sngrep to see whether replies were correctly sent (use tcpdump instead if the results are unclear). If it looks clean there, run Wireshark on the client machine to see whether replies are coming in and whether they have been butchered.

My client device is an iphone running a softphone. There’s no “wireshark” possible there whatsoever.

The issue is easily related to the “policy based routing”, especially if the udp packets from the freepbx are being immediately misdirected. But I cannot “lab” that up, and I’m reeeeeeally hoping that somebody else in here can confirm that there isn’t some weird bug in how asterisk generates the udp responses.

I note now that I managed to try getting my pjsip extension to log in using “tcp” instead of “udp” for the sip protocol – and that immediately worked. The call itself failed, I presume because the rtp is udp-based.

On Wi-Fi or mobile data? If Wi-Fi, a softphone on a laptop connected to the same network will probably fail the same way. If so, you have good network tools available. If mobile data, set the iPhone as a hotspot, connect the laptop to that and test. This will be double NATted, which might cause an unrelated problem, though it usually works quite well.

Alternatively, you should be able to get a SIP trace from the softphone. In Groundwire, which I believe is similar, go to Settings → Preferences → Troubleshooting log.

Useful info. If the RTP failed the same way, I’d expect outbound audio to work and inbound to fail. Possibly, port number rewriting caused Asterisk to ignore the RTP. You should still be able to see the RTP at the PBX using tcpdump.

I have still gotten nowhere, does ANYBODY have advice?

With multiple public IP addresses you will need to define a transport for each one rather than binding to 0.0.0.0.

(edited because I was mistaken)

In Asterisk SIP Settings - pjsip tab, disable the 0.0.0.0 transports and enable the transports bound to each of your public IP addresses. To see these you have to toggle Show Advanced Settings in this tab.

Probably generally true of people here, but it assumes that you are appearing on the internet as though you were two separate sites, rather a single multi-homed site.

If you are doing this for general internet resilience, you need to obtain an autonomous system number and implement border gateway protocol. This probably something that only serious business ISP’s will support, but the company I worked for before I retired did so when it only had about 50 staff.

This means that the IP address from either ISP will route to you through either ISP.

(I think a lot of these dual NIC cases are really about an extranet (for VoIP) and the internet, rather than two internet connections.)

Does making this change not kick in immediately upon pressing the “Apply” button? Because after doing this 1- I didn’t get any improvement (ie. only the tcp connectivity was functional), and 2- the “lsof -ni” still shows that asterisk is binding to “0.0.0.0” not the ips directly.

I’ll have to wait until some time tomorrow if I have to manually shut down the asterisk service and restart it.