Different Port for Endpoing

We’re on FreePBX 13.0.25 and running Asterisk 13.5.0

We’ve recently ran into the need to have some of our endpoint NATing, instead of over VPN. Though it is easy to open up port 5060 on our firewall for a given source IP, we of course have issues when multiple endpoints are coming from that IP. I’ve added the following to iptables to overcome that, and then of course change the port on the endpoint:

iptables -t nat -A PREROUTING -i eth0 -p udp --dport 5161 -j REDIRECT --to-port 5060

This, I’ve noticed, if I make a different redirect port for each endpoint, and then open that port on my firewall, will keep each endpoint properly connect. What’s frustrating is, I can’t just allow port 5161 (in this case) to be the only allowed port from a given source IP. I must ALSO allow 5060 in order for the connection to work.

The reason this is a problem is, we have some roaming endpoints. I would like to allow port 5161 from ANY source IP, so that the endpoint can roam from ISP to ISP, but I’ve found that the device doesn’t properly work until I allow 5060 as well, which I definitely don’t want to allow from “any” source.

It seems initial traffic from the endpoint to the server goes via 5161, but then at some point needs to establish on 5060. This fixes the “NAT hole closing” problem, but requires me to keep 5060 wide open, which I really don’t want to do.

Suggestions? Is there a better way to write that iptables rule so it stays as port 5161 for a given endpoint?