ATA choosing random port

Hi everyone,
So far my experience with asterisk/freepbx has been pretty great.
I have been having this problem for a while now though and am hoping someone can shed some light onto this for me.

We are running asterisk 1.4.24 and freepbx 2.5.1, almost all of the ATA’s we are using are grandstreams.

The ata’s are connecting from remote locations on the internet and they are behind NAT.

We configure the firewall to allow outgoing and incoming connections to the IP of the ATA and the the port ranges 5060 & 10000-20000. This works most of the time.

When we check the peer with “sip show peer xxx” we see the reg. contact showing port 5060 or something in the 10000-20000 range

However sometimes for seemingly no reason the port changes to something out in the middle of nowhere, I have seen. 600035, 65065, 7082, 32768… and when the port changes the peer becomes UNREACHABLE

Is there anyway to force what port range the ATA can use?
Is this a problem with on the ATA side or the server side?
I am not even entirely sure what this port represents, is it the port used for the SIP or the RTP?

Hopefully there is a freepbx/asterisk expert out there that can help me

Thanks in advance!!