Remote sip extension problem

I am trying to setup remote sip extention but it cant connect. On both ends I have simple ISP routers ZXV10 H201L on server side and ZXV10 W300 on client side. Server and phone are both behind NAT.

I tried to connect with hardphone and softphone, in both cases cant connect. With SIP client X-Lite I got error “Problem at server (sip error 408)”.

With setup I did:

  1. Added Sip extension and also put NAT Yes
  2. I configured Asterisk SIP settings
  3. I fowarded ports on router

I could upgrade routers or buy new router and phone with VPN client. But any help or idea to make this work with current setup?