We use freepbx for our voip lines, and configured it using live IP provided by our ISP. But, at times our freepbx server takes lots of bandwidth which is creating a major issue. Also, a important thing I should mention that recently in our router QOS service, I have set xlite (this is the softphone we use) as a high priority. Is there any other issue than QOS things which create this issue.
My guess is that have an unpatched ntp service running, (fixes available on the intertubes), I wasn’t aware xlite had a specific QOS, you would have to be more specific as to what you did but also it is pretty well guaranteed that QOS is not your problem.