Kernel packet priority

I am trying here because Sangoma support so far has been less than helpful. I have two issues and one of then is a major one and I think they are linked. First, my question is, does someone know if there is a command to the kernel to enable or tweek the packet priority in the kernel? Reason is the RTP streams coming from the kernel (via a tshark off the interface) have a high jitter rate, no packet loss but there are httpd processes that are taking priority and causing high latency and jitter to the rtp streams. Received streams to the kernel have low latency and jitter, so its not a network issue.

Now for the more details, this is a HA setup with 700+ phones (1000+ users) on it, running on a Sangoma branded hardware. (version 13), Sangoma phones. They are using AD to sync about half of the phones as users, phones that do not have a person attached to them are local users, but phones that have a person are synced by the AD, so I have about 500+ users that have phones and 300+ users that are virtual that do not have a phone.That being said there seems to be a big issue where the directory forces a sync every time you save a user or its linked extension, and that sync takes about 10 min or so and burns up a lot of CPU time. As you cam imagine its quite a pain to administer where every time you save a user or extension it takes 10+ minutes before you can do anything in the system all. (and this process sometimes degrades voice quality) Besides that the larger issue (if you can’t say the first issue is not large enough) is UCP. When a user logs in a httpd process starts and burns 30% to 99% of CPU, per connection, while the connection is active. Just logging in and doing nothing and its burning up one whole CPU. (server is a 8 core). When you do a wireshark there is a constant stream of data from the server to the client of directory data (list of the users along with all their contact information) in JSON format. That would explain the high httpd process and the fact that after a couple of these are open the RTP streams start to degrade rapidly with high jitter. If the httpd process is trying to send gobs of data and filling up all the buffers and blocking while sending this large amount of unnecessary data, it would explain why the RTP is having high jitter, but I do not understand why as the NIC connection is 1GB and it rarely is seeing about 500mbs throughput, so I am guessing this is a kernel issue not properly having enough buffer, and/or not prioritizing the RTP, or httpd is doing something to prevent a high throughput. First can anyone explain to be why UCP is sending all this data for no reason and can anyone tell me how to shove the RTP to the top of the kernel priority so I stop having voice quality issues. Thanks!

You can increase the buffer size but from your description I don’t know if that will help:

Are you using RestApps from the phones? I’ve seen an issue before where RTP dropped when a user used the apps from the phone, but that was a long time ago and I would assume fixed.

Thanks for the link, that and further research I issued the following -

echo 'net.core.wmem_max=26214400' >> /etc/sysctl.conf
echo 'net.core.rmem_max=26214400' >> /etc/sysctl.conf
echo 'net.core.wmem_default=26214400' >> /etc/sysctl.conf
echo 'net.core.rmem_default=26214400' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_rmem= 10240 87380 12582912' >> /etc/sysctl.conf
echo 'net.ipv4.tcp_wmem= 10240 87380 12582912' >> /etc/sysctl.conf
sysctl -p

That did not seem to make a lot of difference, so after more research I ran across using tc qdisc, so I setup tc qdisc on eth0 -

tc qdisc add dev eth0 root handle 10: prio
tc filter add dev eth0 parent 10: protocol ip prio 1 u32 match ip protocol 17 0xff flowid 10:1
tc filter add dev eth0 parent 10: prio 3 protocol all u32 match u32 0 0 flowid 10:3

I think that may of helped, it seems like the rtp now has a jitter below 10 rather than 20 or greater. Still the jitter to the interface is still under 1 and out of the interface is around 2 to 5, but that is a improvement. Since this is a HA setup, there is a float interface of eth0:ext-ip, I have not tried setting tcqdisc on that yet, I’m not sure if that will make any difference. From what I read the Linux kernel by default will favor TCP as the typical use for Linux would be file / mail / web server and that is all TCP. Linux is not a RTOS (Real TIme Operating System), but there should be a way to tune it to favor UDP over TCP specifically for voice like asterisk. The issue is if I a prioritizing UDP, then I may have higher resends for TCP, so I may consider adding interfaces, add a interface just for UCP and admin, one for the phone traffic and one just for the SIP trunks. I am thinking that may give better voice reliability. That would mean adding them to both servers and adding two more float IP addresses though pcs (Pacemaker).

fyi, i had posted about the AD directory sync issue a while ago CPU utiliziation spikes......gulp and i put a ticket in on it here https://issues.freepbx.org/browse/FREEPBX-19974

Thanks, glad to know I am not the only one having issues with this.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.