Remote Extension with VPN and Jitter!

Hi Members again, i have a question regarding Asterisk behind Vyatta and QoS. The scenario is Internet -->Vyatta–>Asterisk
Vyatta is serving as a VPN server also for Remote Users to connect with local extensions.

Now the problem is when i register extension from remote location, every thing works fine but when i try to call i can hear only from one side, but from the other side voice is not consistent. “I guess Jitter”

I have applied Vyatta QoS , but same problem. VPN i am using is PPTP.
This might be due to TCP or else, Any suggestions please?