This is usually caused by badly behaving routers that remap the outbound port randomly for no good reason. What brand/model router are you using for NAT?
Your router is most likely using symmetric nat. You will most likely be unable to change that behavior. I know that Sonicwall uses this and I am sure many more do.
Symmetric NAT
Each request from the same internal IP address and port to a specific destination IP address and port is mapped to a unique external source IP address and port, if the same internal host sends a packet even with the same source address and port but to a different destination, a different mapping is used.
Only an external host that receives a packet from an internal host can send a packet back.
So I guess it is symmetric NAT. Flowroute said that could be problematic, not having the port show up at Flowroute as 5060, now it is 56641. Anyone know if it is in fact a problem?
The way I get around it is to increase UDP timeouts to 300 seconds. This will keep the signaling port open provided you have qualify enabled, otherwise it will constantly changed once the UDP port times out. Sonicwall is able to do this per rule, but I am not sure about TPLINK.