Trying moving to chan_sip. PJSIP has come a long ways but still reports of weird issues like this and others.