UDP ports Phone VS Asterisk

So I may have misread your original post.

It sounds like 2 separate (but possibly related) issues. 1st is the SIP time outs, 2nd is dropped audio.

Are the SIP timeouts occurring during registration, call setup, or during a call? What is SIP Debug telling you about these?

The dropped audio seems more likely a lack of QoS across the VPN (these are all calls between sites, correct). But if the SIP timeouts are happening during the call, that could possibly cause the dropped audio. What are your RTCP statistics saying about call quality?