FreePBX 12.0.76 started acting weird on Ubuntu 14.04

FreePBX
1

Recently I noticed that my FreePBX is acting weird. It was a FreePBX 12 running on Ubuntu 12.04. The web ui response time is terrible slow + multiple menus (like Backup and Restore, Extensions etc) “greyed out”. I configured full backup for every night but my last backup is from 25th of August. I did not noticed anything sooner because every extension worked properly and I not logged in the webui for days.

I installed another system on Ubuntu 14.04 and destroyed the old one (which was a mistake probably) and restored everything from the last backup, but the result is the same.

Current config:
Ubuntu 14.04 64bit
FreePBX 12.0.76
Asterisk 12.8.2

Module list:

root@sc-asterisk:/home/username# amportal a ma list

Please wait…

no repos specified, using: [standard] from last GUI settings

Module Version Status

announcement 2.11.0.4 Enabled
arimanager 12.0.4 Enabled
asteriskinfo 12.0.2 Enabled
backup 12.0.19 Enabled
blacklist 12.0.3 Enabled
builtin Enabled
callforward 12.0.7 Enabled
callrecording 12.0.3 Enabled
callwaiting 12.0.3 Enabled
campon 2.11.0.2 Enabled
cdr 12.0.22 Enabled
certman 12.0.5 Enabled
cidlookup 2.11.1.12 Enabled
conferences 12.0.16 Enabled
contactmanager 12.0.3 Enabled
core 12.0.39 Enabled
customappsreg 12.0.3.2 Enabled
cxpanel 4.1.8 Enabled
dashboard 12.0.31.2 Enabled
daynight 2.11.0.6 Enabled
digium_phones 2.11.2.5 Disabled
digiumaddoninstaller Not Installed (Locally available)
donotdisturb 12.0.3 Enabled
featurecodeadmin 12.0.2 Enabled
findmefollow 12.0.11 Enabled
framework 12.0.76 Enabled
iaxsettings 2.11.0.3 Enabled
infoservices 12.0.3.2 Enabled
ivr 2.11.0.11 Enabled
logfiles 12.0.6 Enabled
manager 12.0.2 Enabled
music 12.0.1 Enabled
paging 12.0.14 Enabled
parking 12.0.12.1 Enabled
presencestate 12.0.30 Enabled
printextensions 2.11.0.1 Enabled
recordings 12.0.8 Enabled
ringgroups 12.0.3.2 Enabled
setcid 2.11.0.4 Enabled
sipsettings 12.0.14 Enabled
sipstation 12.0.8.6 Enabled
timeconditions 12.0.8 Enabled
ucp 12.0.24 Enabled
userman 12.0.26 Enabled
voicemail 12.0.40 Enabled

Only error messages (which I think irrelevant) from freepbx.log:

[2015-Aug-28 12:55:29] [ERROR] (core/install.php:569) - Failed to add index to extensions field in the users table
[2015-Aug-28 12:56:53] [ERROR] (BMO/Notifications.class.php:493) - [NOTIFICATION]-[framework]-[ASTCONFAPPMISSING] - No Conference Room App (Neither app_meetme nor app_confbridge is configured in Asterisk, conferencing, paging and other functionality will not work properly)

Error messages from /var/log/apache2/error.log:

[Mon Aug 31 18:07:52.431199 2015] [:error] [pid 28843] [client 10.1.10.33:40047] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130, referer: http : //10.1.10.32/admin/config.php
[Wed Sep 02 16:53:35.767156 2015] [:error] [pid 26199] [client 10.1.10.33:42031] script ‘/var/www/html/config.php’ not found or unable to stat
[Wed Sep 02 17:00:02.143674 2015] [:error] [pid 8966] [client 10.1.10.33:42456] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130, referer: http : //10.1.10.32/admin/config.php
[Wed Sep 02 17:51:41.587003 2015] [:error] [pid 9109] [client 10.1.10.33:46545] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130, referer: http : //10.1.10.32/admin/config.php
[Wed Sep 02 18:07:03.706297 2015] [:error] [pid 9110] [client 10.1.10.33:47872] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/libraries/BMO/Codecs.class.php on line 70, referer: http : //10.1.10.32/admin/config.php?display=extensions
[Wed Sep 02 18:09:04.165025 2015] [:error] [pid 9107] [client 10.1.10.33:47962] PHP Fatal error: Call to a member function send_request() on a non-object in /var/www/html/admin/modules/asteriskinfo/page.asteriskinfo.php on line 141, referer: http : //10.1.10.32/admin/config.php?display=extensions
[Wed Sep 02 18:28:28.680883 2015] [:error] [pid 15970] [client 10.1.10.33:49579] PHP Strict Standards: Only variables should be passed by reference in /var/www/html/admin/modules/dashboard/Dashboard.class.php on line 128, referer: http : //10.1.10.32/admin/config.php

Firsts I only see the error about the diginum_phone function, so I disabled that module. Now the new errors appeared.

I even tried (on another server) to install FreePBX 12 then upgrade to 13, restore the backup. Every menu worked fine, but when I clicked on an extension I got mysql error message. (long select command appeared in red).

Then I tried to install FreePBX 13 on Ubuntu 14.04 from scratch using this documentation:
http : //wiki.freepbx.org/display/HTGS/Installing+FreePBX+13+on+Ubuntu+Server+14.04.2+LTS

But I got error during this step: Compile and install pjproject when run the “make deb” command

make[2]: Leaving directory /usr/src/pjproject-2.4/pjsip/build' make -f /usr/src/pjproject-2.4/build/rules.mak APP=PJSUA2_LIB app=pjsua2-lib depend make[2]: Entering directory /usr/src/pjproject-2.4/pjsip/build’
.pjsua2-lib-x86_64-unknown-linux-gnu.depend:1: *** missing separator. Stop.
make[2]: Leaving directory /usr/src/pjproject-2.4/pjsip/build' make[1]: *** [depend] Error 2 make[1]: Leaving directory /usr/src/pjproject-2.4/pjsip/build’
make: *** [dep] Error 1

So right now I have multiple server and none of them working 100% properly. Any help will be appreciated.

2

FreePBX can’t talk to Asterisk. You either changed the password in /etc/asterisk/manager.conf or it doesnt match whats in Advanced Settings

3

Thank you very much for your help.

Since the webUi is basically unusable (it is soo slow) is there any CLI way to change FreePBX “advanced” settings to the same what I have in the manager.conf file?

Regars,

4

Run (as root):
amportal a s

5

Hi,

Again, thank you very much for your help.

Unfortunately it did not work. But it seems something not quite right with the config (but I am not an expert). I logged to the asterisk database and checked the ampusers table. It has 2 records but none of them are in the config files. (after running the command).

So I inserted a new record by hand with the username and password from the config file (I used this link info to generate the password hash: https : //vps.powerpbx.org/clients/knowledgebase/12/How-to-change-FreePBX-admin-password-from-command-line.html (sorry I am not allowed to enter a url)).

But I have still having the same issue: slow response time, greyed out menus + the following error message on the login page retrieve_conf failed, config not applied → Reload failed because retrieve_conf encountered an error: 1

  • I can see a flame icon next to the “Asterisk” label and if I move the mouse over to the label I see the text: “Asterisk Manager Interface (astman) failure”

after amportal restart this is in the apache2 error log:

[Mon Aug 31 16:10:09.172481 2015] [:error] [pid 26199] [client 10.1.10.33:57722] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130, referer: http : //10.1.10.32/admin/config.php
[Mon Aug 31 16:10:23.403909 2015] [:error] [pid 26201] [client 10.1.10.33:57739] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130
[Mon Aug 31 16:11:09.258312 2015] [:error] [pid 26198] [client 10.1.10.33:57724] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130
[Mon Aug 31 16:12:09.340512 2015] [:error] [pid 26200] [client 10.1.10.33:57727] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130
[Mon Aug 31 16:46:59.989278 2015] [:error] [pid 28840] [client 10.1.10.33:60226] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130, referer: http : //10.1.10.32/admin/config.php
[Mon Aug 31 18:07:52.431199 2015] [:error] [pid 28843] [client 10.1.10.33:40047] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130, referer: http : //10.1.10.32/admin/config.php
[Wed Sep 02 16:53:35.767156 2015] [:error] [pid 26199] [client 10.1.10.33:42031] script ‘/var/www/html/config.php’ not found or unable to stat
[Wed Sep 02 17:00:02.143674 2015] [:error] [pid 8966] [client 10.1.10.33:42456] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130, referer: http : //10.1.10.32/admin/config.php
[Wed Sep 02 17:51:41.587003 2015] [:error] [pid 9109] [client 10.1.10.33:46545] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/modules/digium_phones/functions.inc.php on line 130, referer: http : //10.1.10.32/admin/config.php
[Wed Sep 02 18:07:03.706297 2015] [:error] [pid 9110] [client 10.1.10.33:47872] PHP Fatal error: Call to a member function connected() on a non-object in /var/www/html/admin/libraries/BMO/Codecs.class.php on line 70, referer: http : //10.1.10.32/admin/config.php?display=extensions
[Wed Sep 02 18:09:04.165025 2015] [:error] [pid 9107] [client 10.1.10.33:47962] PHP Fatal error: Call to a member function send_request() on a non-object in /var/www/html/admin/modules/asteriskinfo/page.asteriskinfo.php on line 141, referer: http : //10.1.10.32/admin/config.php?display=extensions
[Wed Sep 02 18:28:28.680883 2015] [:error] [pid 15970] [client 10.1.10.33:49579] PHP Strict Standards: Only variables should be passed by reference in /var/www/html/admin/modules/dashboard/Dashboard.class.php on line 128, referer: http : //10.1.10.32/admin/config.php
[Wed Sep 02 19:07:45.015071 2015] [:error] [pid 17634] [client 10.1.10.11:49825] script ‘/var/www/html/config.php’ not found or unable to stat

6

Ampusers has nothing to do with Asterisk Manager. Please don’t go off searching the internet if you want help here. It really makes things more frustrating for those actually trying to help you when you go off and do something else entirely.

It would also have been helpful for me to know what the command said/did when you ran it. But you didn’t provide that. The article you linked to is for FreePBX Administration Users. Not Asterisk Manager users.

7

Are you sure your machine isn’t pointing to a broken DNS server? That will cause apache to hang, as well as other odd problems.

8

This is because the asterisk manager user is wrong. That’s why the menu items are greyed out. It’s timing out. Might also be a local host resolve issue in resolve.conf

9

Hi,

Thanks again your help. I promise I won’t do any partisan activity anymore I just felt that if I don’t know something what you recommend then I should google about it and try to do it. So I did not want to ask too many questions.

Back to my issues:

root@sc-asterisk:/home/username# amportal a s

Please wait...

Manager ReSynced, restarting Asterisk

STOPPING ASTERISK
Waiting for Asterisk to Stop Gracefully...

Asterisk Stopped 

SETTING FILE PERMISSIONS................Done
Removing any dangling symlinks
Dangling symlinks removed

STARTING ASTERISK
Asterisk Started
Reloading Asterisk because of Dynamic Hints

Also I waited to the webui to load the Advanced Settings menu (10+ min) and checked the “Asterisk Manager” username and password there. It is the same than the manager.conf file

Additional info:

The server is a VM running on KVM machine and it has only 1 GB RAM. I have around 30 extension.

root@sc-asterisk:/home/username# free -m
             total       used       free     shared    buffers     cached
Mem:           994        858        135         24        152        450
-/+ buffers/cache:        255        738
Swap:          975         14        961
10

@tm1000 @xrobau

Hi

Thank you very much for your help. I checked the name resolving. These were my outputs:

root@sc-asterisk:/home/username# ping localhost
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.045 ms
64 bytes from localhost (127.0.0.1): icmp_seq=3 ttl=64 time=0.046 ms
^C
--- localhost ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.045/0.050/0.060/0.009 ms
root@sc-asterisk:/home/username# ping google.com
PING google.com (216.58.192.14) 56(84) bytes of data.
64 bytes from nuq04s29-in-f14.1e100.net (216.58.192.14): icmp_seq=1 ttl=55 time=11.6 ms
64 bytes from nuq04s29-in-f14.1e100.net (216.58.192.14): icmp_seq=2 ttl=55 time=14.5 ms
^C
--- google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 11.638/13.097/14.556/1.459 ms
root@sc-asterisk:/home/username# cat /etc/resolv.conf 
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
search company.com
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8


root@sc-asterisk:/home/username# dig google.com

; <<>> DiG 9.9.5-3ubuntu0.4-Ubuntu <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12621
;; flags: qr rd ra; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;google.com.			IN	A

;; ANSWER SECTION:
google.com.		55	IN	A	74.125.239.99
google.com.		55	IN	A	74.125.239.101
google.com.		55	IN	A	74.125.239.97
google.com.		55	IN	A	74.125.239.110
google.com.		55	IN	A	74.125.239.98
google.com.		55	IN	A	74.125.239.100
google.com.		55	IN	A	74.125.239.96
google.com.		55	IN	A	74.125.239.102
google.com.		55	IN	A	74.125.239.103
google.com.		55	IN	A	74.125.239.105
google.com.		55	IN	A	74.125.239.104

;; Query time: 13 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Thu Sep 03 06:52:49 UTC 2015
;; MSG SIZE  rcvd: 215

It was a little bit strange for me that the /etc/resolv.conf has duplicated entries. So I checked the /etc/network/interfaces file:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 10.1.10.32
        netmask 255.255.255.0
        network 10.1.10.0
        broadcast 10.1.10.255
        gateway 10.1.10.1
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 8.8.8.8
        dns-search company.com

We don’t have dedicated DNS nameserver in the office so I removed the last line and restated the machine. (ifdown and ifup removed the company.com from the resolv.conf but I still had 2 identically same entries (with comments)). But the result is the same:

cat /etc/resolv.conf 
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8

Could this be my problem?

Update:
@xrobau

I checked and the /etc/resolvconf/resolv.conf.d/tail already had the comments and the 8.8.8.8 nameserver entry. I deleted it and did an ifdown eth0 && ifup eth0 and now /etc/resolv.conf has only 1 entry. + I did an amportal restart:

amportal restart

Please wait...


STOPPING ASTERISK
Waiting for Asterisk to Stop Gracefully...

Asterisk Stopped 

SETTING FILE PERMISSIONS................Done
Removing any dangling symlinks
Dangling symlinks removed

STARTING ASTERISK
Asterisk Started
Reloading Asterisk because of Dynamic Hints

Plus I noticed if I click on the Applications >> Extensions menu then eventually I will get a page with this error message:

[WARNING]: Menu Item: Extensions is disabled because asterisk is not running

Restart Asterisk and then refresh the browser in order to try accessing this menu item again.
11

This is because your machine can’t talk to Asterisk. This could be any number of reasons. Normally it errors quickly, which is what makes me think you have a DNS error. Check your Asterisk Manager settings in Advanced Settings

i7AdRw1.png837×277 26.5 KB

They should match /etc/asterisk/manager.conf

12

HI,

Again, thank you very much for your reply. The username and the password is matching. But on my FreePBX webui I don’t have any option to define Manager Host, Port and Timeout information. I only have “Asterisk Builtin mini-HTTP server” > HTTP Bind Port which is configured for 8088.

Here is my manager.conf file:

;
; AMI - Asterisk Manager interface
;
; FreePBX needs this to be enabled. Note that if you enable it on a different IP, you need
; to assure that this can't be reached from un-authorized hosts with the ACL settings (permit/deny).
; Also, remember to configure non-default port or IP-addresses in amportal.conf.
;
; The AMI connection is used both by the portal and the operator's panel in FreePBX.
;
; FreePBX assumes an AMI connection to localhost:5038 by default.
;
[general]
enabled = yes
port = 5038
bindaddr = 0.0.0.0
displayconnects=no ;only effects 1.6+

[username]
secret = password
deny=0.0.0.0/0.0.0.0
permit=127.0.0.1/255.255.255.0
read = system,call,log,verbose,command,agent,user,config,command,dtmf,reporting,cdr,dialplan,originate,message
write = system,call,log,verbose,command,agent,user,config,command,dtmf,reporting,cdr,dialplan,originate,message
writetimeout = 5000

#include manager_additional.conf
#include manager_custom.conf

Here are the listening ports:

netstat -tulnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      26312/sshd      
tcp        0      0 0.0.0.0:8088            0.0.0.0:*               LISTEN      2158/asterisk   
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      682/mysqld      
tcp        0      0 0.0.0.0:5038            0.0.0.0:*               LISTEN      2158/asterisk   
tcp6       0      0 :::22                   :::*                    LISTEN      26312/sshd      
tcp6       0      0 :::80                   :::*                    LISTEN      1124/apache2    
udp        0      0 0.0.0.0:4569            0.0.0.0:*                           2158/asterisk   
udp        0      0 10.1.10.32:9000         0.0.0.0:*                           2158/asterisk

I changed the default 5060 to 9000 because of reasons…

13

I don’t know. All I can say is that your Apache service can’t connect to Asterisk… If you’re sure that your asterisk manager username and password are correct, then I am stumped.

The only other things I can think of is that your machine has somehow had selinux enabled, or something similar, or, someone clever has installed a firewall on the box that doesn’t have ‘lo allow all’ in it.

Check iptables-save?

14

Hi,

I do have firewall rules but I think it is not a issue based on my netstat and nmap output:
netstat:

netstat -tulnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      26312/sshd      
tcp        0      0 0.0.0.0:8088            0.0.0.0:*               LISTEN      2158/asterisk   
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      682/mysqld      
tcp        0      0 0.0.0.0:5038            0.0.0.0:*               LISTEN      2158/asterisk   
tcp6       0      0 :::22                   :::*                    LISTEN      26312/sshd      
tcp6       0      0 :::80                   :::*                    LISTEN      1124/apache2    
udp        0      0 0.0.0.0:4569            0.0.0.0:*                           2158/asterisk   
udp        0      0 10.1.10.32:9000         0.0.0.0:*                           2158/asterisk

Nmap:

root@sc-asterisk:/home/username# nmap -Pn -p22,8088,3306,5038 -sV localhost
Starting Nmap 6.40 ( http://nmap.org ) at 2015-09-04 16:09 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000041s latency).
Other addresses for localhost (not scanned): 127.0.0.1
PORT     STATE SERVICE  VERSION
22/tcp   open  ssh      (protocol 2.0)
3306/tcp open  mysql    MySQL 5.5.44-0ubuntu0.14.04.1
5038/tcp open  asterisk Asterisk Call Manager 2.5.0
8088/tcp open  http     Asterisk 12.8.2
1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at http://www.insecure.org/cgi-bin/servicefp-submit.cgi :
SF-Port22-TCP:V=6.40%I=7%D=9/4%Time=55E9C249%P=x86_64-pc-linux-gnu%r(NULL,
SF:2B,"SSH-2\.0-OpenSSH_6\.6\.1p1\x20Ubuntu-2ubuntu2\.3\r\n");
Service Info: Device: PBX

That’s being said, when I running a netcat localhost 5038 or nmap -p5038 localhost command in one terminal and tcpdump -n dst port 5038 in another I can not see any traffic. (not sure because tcpdump not capturing lo interface traffic by default or why)

I repeated these tests with the firewall turned off and the result was the same.

update:
I checked the man page of tcpdump and by default it is monitoring on eth0. So after running tcpdump -i lo -n dst port 5038 I can see that packets are coming to the port when I using netcat or open the webui.

Previously I noticed that during my attempt to create another machine everything works fine until that point where I restoring my backup. During the Restore process I can decide what do I want to restore:
-files (in the /etc and /var directories )
-PBX configs
-CDRs config (or something like that)

If I remember well (but I have to test it again), when I selecting all of them except the PBX then the webui is still working, but I don’t have my old extension information in the system.
If I enable the PBX on FreePBX 12 then this gryed out problem comes up.
If I update the FreePBX from 12 to 13 first then import it then menus are working, I can see all the extensions, but when I click to edit any of them I the big red SQL Select error message.

If you think this VM is hopeless and you are still with me and up to the challenge then we can try solve the import / select issue on PBX 13.

I will be more than happy if I can solve this stuff.