PJSIP issues and possible memory leaks?

Is anyone having weird issues with PJSIP? Since I migrated from ASterisk 13/Freepbx 12 I’ve had nothing but extensions dropping off. I was running on a 1024 RAM on Vultr and upped it to 2048 ram last night and TOP is showing me free memory keeps dropping:

top - 10:21:35 up 12:16, 1 user, load average: 0.02, 0.13, 0.15
Tasks: 107 total, 1 running, 106 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.3 us, 0.3 sy, 0.0 ni, 99.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 1883708 total, 212440 free, 778244 used, 893024 buff/cache
KiB Swap: 2097148 total, 2094288 free, 2860 used. 790064 avail Mem

This started off as about 650000 free last night. looking at a asterisk debug I see pjsip running out of memory. Before I go over to Asterisk on this has anyone been seeing anything weird?

Thanks leon

some of the asterisk debug

[2017-09-06 09:45:48] ERROR[7284] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 702
[2017-09-06 09:45:48] ERROR[3640] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 702
[2017-09-06 09:45:48] ERROR[3640] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 702
[2017-09-06 09:45:48] ERROR[3640] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 702
[2017-09-06 09:45:48] ERROR[3640] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 702
[2017-09-06 09:45:48] ERROR[7285] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 702
[2017-09-06 09:45:48] ERROR[7285] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 702
[2017-09-06 09:45:48] ERROR[7285] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 702
[2017-09-06 09:45:48] ERROR[7285] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 702
[2017-09-06 09:45:48] ERROR[7285] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 702
[2017-09-06 09:45:48] ERROR[2803] pjproject: ssl0x7f0b5c0ccd40 Renegotiation failed: Not enough memory (PJ_ENOMEM)
[2017-09-06 09:45:48] VERBOSE[2809] res_pjsip/pjsip_configuration.c: Contact 702/sips:[email protected]:45384;transport=TLS is now Unreachable. RTT: 0.000 msec
[2017-09-06 09:54:06] VERBOSE[8236] res_pjsip/pjsip_configuration.c: Contact 702/sips:[email protected]:38964;transport=TLS has been deleted
[2017-09-06 09:56:06] VERBOSE[8236] res_pjsip/pjsip_configuration.c: Contact 702/sips:[email protected]:45367;transport=TLS has been deleted
[2017-09-06 09:59:30] VERBOSE[7602] pbx_variables.c: Setting global variable ‘SIPDOMAIN’ to ‘pbx.backwoodswireless.net

BTW I uninstalled Zulu, Symphony and other things that would seem to be optional in order to debug this.

ldz

This should be resolved in

13.7.0, 14.0.0

If you have a version equal to or greater than the above see:

https://issues.asterisk.org/jira/browse/ASTERISK-25653

that issue is closed. I will open a ticket over there thanks james

PS: I’m on the latest codebase

“suspended” no activity. I think that means it can be reopened if someone is willing to get them what they need.

right but they said if someone else was having similar to open a new issue which I did and referred to that one in the text. I’m corresponding with them already on it. very weird never had these issues on 13/12

thanks leon

I see. Yeah I am a “headline reader” didn’t see that.

I also updated the superfecta thread here can you please take a looksee?

Yeha I read the headlines too unless it strikes my fancy :slight_smile:

Hi folks…I opened a ticket with Asterisk for this

https://issues.asterisk.org/jira/browse/ASTERISK-27258

But they need me to send them the output of memory show memory commands from the CLI which is NOT in the binaries you provided.

They are requesting, “You have to rebuild Asterisk [2] from source and enable the Asterisk compiler flag MALLOC_DEBUG using menuselect to get the CLI “memory show summary” commands.”

There is a definite memory leak and abnormal operation with PJSIP.

Please advise how we can get there from here.

Thanks leon

You will need to compile asterisk from source with the provided options. This is not something we can provide unfortunately but is in our wiki on how to compile it if needed.

UGH wouldnt this be something needed to for normal debugging of production stuff? WHy isn’t that part of the build?

1 Like

I’ve just searched the wiki and can not find it can you please provide a link?

It’s not compiled in RPMs that way as it’s not optimized when you do that and affects the call volume that the PBX can handle.

well can you provide a test gen so I can, or others, run the tests asterisk wants me to do on this problem? I would think this would be a simple thing to do to the build scripts

No we can not and no it’s not simple.

ok so where are the instructions mentioned in the wiki as I could not find them??

Hi!

I think I would use something like

https://wiki.freepbx.org/display/FOP/Installing+FreePBX+13+on+CentOS+7#InstallingFreePBX13onCentOS7-CompileandinstallAsterisk

(ie the Compile Asterisk section of the a tutorial that tells you how to install FreePBX on different distros… Pick the one closest to your situation…)

Good luck and 73

(I am not an ham, my father is… This is how I recognized what your username is even though the prefixes and format slightly change from one place to the next…)

Nick

Hi there…yup am a ham and I’m in Pennsylvania :slight_smile:

ok the link you gave was 13 so I found the 14 stuff and will attempt it in the morning

thanks 73 leon

what is your dad’s callsign and where is he?

I know it’s not part of the normal FreePBX stuff, but one of the Chan-SCCP-B Wikis I wrote talks briefly about adding options for debugging (and conferencing, IIRC) to the Asterisk kernel. Perhaps a quick perusal of the Wiki at GitHub there might get you what you need?

Hi Leon!

I wasn’t sure what version you were running (that’s why I said to pick the one closest to your situation )…

You are welcome!

I will send it to you by PM…

You are an ham, I think you will understand why I have reservations about posting someone else’s callsign in a public forum…

Good luck and have a nice day!

Nick