PJSIP PJ_ENOMEM Error

General Help
1

Hi Folks,

Every week or so I have to restart my FreePBX server because I receive the error messages below. Is there a setting for PJSIP that needs to be increased so I don’t receive these messages?

My system is has approx 60 extensions, 40 of them Digium D70 phones and 20 of them Mitel 6869i running TLS/SRTP. My FreePBX version is 13.0.192.9. Asterisk is version 13.15.0. The modules are on the stable track, not edge. Physical server, not virtualized with two Xeon E5-2650 V4 processors and 16GB of ram.

Thanks for your help.

RAM Usage:
[root@pbx ~]# free
total used free shared buffers cached
Mem: 16206164 3102828 13103336 448 257676 1631600
-/+ buffers/cache: 1213552 14992612
Swap: 8175612 0 8175612

Error messages:
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9928141’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9922141’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9925141’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9925142’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9928142’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9928143’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9922143’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9925143’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9928144’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9925144’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9928145’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9925145’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9925147’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9928147’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9925148’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] ERROR[6312] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 148
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9922148’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] ERROR[6312] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 148
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9928148’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] ERROR[6312] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 148
[2017-06-11 11:24:42] ERROR[6313] res_pjsip.c: Error 70007 ‘Not enough memory (PJ_ENOMEM)’ sending NOTIFY request to endpoint 148
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9928150’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9922150’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9925150’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9928153’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9925153’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9928154’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9922154’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9925154’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9928161’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9925161’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension '9925162’ priority -1 to restapps-device-hints
[2017-06-11 11:24:42] VERBOSE[6639] pbx.c: Added extension ‘9928162’ priority -1 to restapps-device-hints
[2017-06-11 11:24:43] VERBOSE[6639] pbx.c: Added extension '9928170’ priority -1 to restapps-device-hints
[2017-06-11 11:24:43] VERBOSE[6639] pbx.c: Added extension ‘9925170’ priority -1 to restapps-device-hints
[2017-06-11 11:24:43] VERBOSE[6639] pbx.c: Added extension '9928172’ priority -1 to restapps-device-hints
[2017-06-11 11:24:43] VERBOSE[6639] pbx.c: Added extension ‘9922172’ priority -1 to restapps-device-hints
[2017-06-11 11:24:43] VERBOSE[6639] pbx.c: Added extension '9925172’ priority -1 to restapps-device-hints
[2017-06-11 11:24:43] VERBOSE[6639] pbx.c: Added extension ‘9928199’ priority -1 to restapps-device-hints
[2017-06-11 11:24:43] VERBOSE[6639] pbx.c: Added extension '9922199’ priority -1 to restapps-device-hints
[2017-06-11 11:24:43] VERBOSE[6639] pbx.c: Added extension ‘9925199’ priority -1 to restapps-device-hints
[2017-06-11 11:24:43] VERBOSE[6639] pbx.c: Added extension '9928202’ priority -1 to restapps-device-hints

2

Please take out the verbose messages.

3

Hi Andrew,

Are you saying that it is something that can be ignored because the verbose messages are just being “overly wordy” and the system should function properly?

4

Well it looks like your system is running out of memory but it was hard to see without your message being wrap as code and with all of the verbose messages.

5

Hi Andrew,

Here are the messages without the extra verbosity.

[2017-06-11 11:24:42] ERROR[6312] res_pjsip.c: Error 70007 'Not enough memory (PJ_ENOMEM)' sending NOTIFY request to endpoint 148
[2017-06-11 11:24:42] ERROR[6312] res_pjsip.c: Error 70007 'Not enough memory (PJ_ENOMEM)' sending NOTIFY request to endpoint 148
[2017-06-11 11:24:42] ERROR[6312] res_pjsip.c: Error 70007 'Not enough memory (PJ_ENOMEM)' sending NOTIFY request to endpoint 148
[2017-06-11 11:24:42] ERROR[6313] res_pjsip.c: Error 70007 'Not enough memory (PJ_ENOMEM)' sending NOTIFY request to endpoint 148
6

In case it is of any use, when PJSIP throws those errors, the phones using PJSIP fail to register.

Additional info:

The system is approx 1 year old and processes approx 100 calls a day.

7

The error is simply Not enough memory. PJSIP has run out of memory and won’t do anything.

How much RAM do you have on the system

8

That’s almost nothing for a machine of that size. There may be a bug in the Asterisk version you’re running - How long has this been happening?

9

There is 16GB of ram. Below is its usage.

[root@pbx ~]# free
total used free shared buffers cached
Mem: 16206164 3102828 13103336 448 257676 1631600
-/+ buffers/cache: 1213552 14992612
Swap: 8175612 0 8175612

10

Hi Rob,

This has been going on for months. I usually don’t let it get to the point of failure as I know I have about 7 days of use before the errors start, so the system gets restarted before that. Time constraints have delayed me from giving this matter the attention it needs.

11

One more item to note, this system was an install from the FreePBX distro, it was not installed in some other manner. There is no additional software added to the machine. All modules are at the latest stable track.

12

The ‘free’ output you posted earlier is showing plenty of free memory. How long is it going to be until you think it starts playing up again? We’ll need to see the output of ‘free’, as well as ‘ps -auxw’ when it’s erroring, unfortunately.

13

About July 4th if left unattended.

1 Like
14

Well, this is going to be an Asterisk issue (most likely), but if you could wait until it IS erroring, and then run these commands, that’ll help us diagnose what the problem is:

ps -auxww | pastebin
free -h | pastebin
tail -500 /var/log/asterisk/full | pastebin
asterisk -rx 'core show channels' | pastebin

That will give you 4 links that you can paste here (or, send to either @tm1000 or I as a private message) and that will be amazing. Sorry we can’t do much more now 8-(

15

Many many thanks Rob. Will do.

1 Like