Apply Configuration Changes - Error and locks up system

Hello,

We are using FreePBX Version 2.7.0.1 and CentOS 5.4 (Final). The system has worked flawlessly for us until today. I logged into the gui to add an extension for a new employee, I noticed the Apply Configuration Changes bar was displayed, wasn’t sure why, figured we must have forgotten to press it after the last change a week ago. So, I pressed it, Continued with reload, the frogy was catching the fly for roughly 1 minute, then I get an error which read:

Error: Did not receive valid response from server

The GUI completely locked up. I was able to SSH in and restart httpd. The Apply Configuration Changes was still displaying at the top. Since it was early and no one was in the office yet, I rebooted the server. It came back up, logged in, the PCC button was still there, I clicked it and watched in asterisk -r to see what happened. It looked like it completed the reload and everything was working, except the GUI was locked up again, and I had to restart httpd again.

Any ideas on what might be causing this? It happens in Firefox and in IE. Any assistance is greatly appreciated!

now it doesn’t error out, but it just sets there on the “Please wait, reloading” page. If someone has any ideas, please share, this is extremely frustrating.

Check your log file, check your CPU utilization when clicking on Apply Changes (use top).
Check your MySQL for errors:

mysqlcheck -u root -p --check asterisk

Check that there is enough memory with free -m

Hi Mikael,

Thank you for your response. Here are the outputs from top while applying:

top - 07:28:44 up 23:39, 3 users, load average: 0.36, 0.09, 0.03
Tasks: 95 total, 1 running, 94 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2%us, 0.8%sy, 0.0%ni, 95.8%id, 3.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2051104k total, 888256k used, 1162848k free, 148888k buffers
Swap: 1015800k total, 0k used, 1015800k free, 312664k cached

There are no errors in MySQL.

Output from free -m:

         total       used       free     shared    buffers     cached

Mem: 2003 866 1136 0 145 305
-/+ buffers/cache: 415 1587
Swap: 991 0 991

Thank you again for your assistance.

I did notice when I click apply, the log shows all the settings being applied, etc, I didn’t see any errors. The apply feature finally did error out with the same error as before.

The problem still lingers, I’m not sure what else to do at this point.

Ok, this issue is really starting to cause a problem, each time the apply button is pressed, it drops all calls and asterisk has to be restarted to get calls flowing again. The apply configuration changes button is always displaying, even if there are no changes to be applied. Still getting the error when applying also.

I’ve seen this happen when the system is busy, but I’ve never had to restart httpd, just refresh the page (F5). Sometimes the orange bar reappears, but clicking again clears everything up.

And a question for Mikael: What is the underlying command the GUI runs when the Orange Bar is pressed. (I’ve got it here somewhere but…)

Is it just the Asterisk CLI reload command?

Perhaps doing the reload outside the GUI might be a good troubleshooting tool.

Bill

Hi Bill,

Thanks for the response. We’ve done that a thousand times, the bar is still there. This system is pretty bored most of the time. It’s worked great for us until here recently, I’m not quite sure why. The bar won’t go away, and if someone goes in and clicks it, it drops all the calls (if there are any). You would think that when you add an extension or make any change and click “Save”, that it would do what it needs to do to apply the changes rather than having to do multiple steps to “apply /save” the configuration.

I think that might be left over from earlier telephony days. I administered several early electronic PBX’s in the 1970’s. (Dimension, Horizion, Merlil, as well as an ESSX system). They all followed the same scenario. You did your upgrades and then at a later time, usually in the wee hours of the morning, the system was updated.

I can see the logic of this.

  1. Updating the working config files can be a resource intensive operation.
  2. Many of the early systems required a system restart to commit the changes.
    Finally, the way asterisk is set up, ALL configuration files are reloaded no matter how small the change, so it makes sense to wait until you’ve done all your configuration before applying the changes.

BF

I’m wondering if part of the problem (at least the error) might be because the Browser is timing out waiting for a response from the server. In Firefox, it’s fairly easy to extend the timeout time. Just google “firefox timeout extend” and see if there’s any clue there.

This won’t solve the underlying problem of the server taking so long to respond, but might let you keep control of your browser.

Bill

Hi Bill,

I have the same problem in IE or any other browser also, so I don’t think it is browser related. The system always thinks there’s something that needs to be applied, which I believe is why the apply config changes button is always there, but I have no clue why it’s happening.

Do you have PHPMySQLadmin installed? You can browse the database and see if the flag is set.

If you know how to do SQL queries from the cmd line you can use the mysql tool. I don’t know the table name off the top of my head. If you browse the database it is intuitively obvious.

Will that solve the issue where it drops all the calls when applying, too or just make the bar disappear? :wink:

All browsers have a timeout that kicks in if it’s waiting too long. What I’m suggesting that you increase the length of that timer. No, it won’t prevent the system bogging down during the reload, but it might allow you to keep control of the browser and at least clear the Apply Bar.

Bill

I’ll give it a try, willing to try just about anything at this point. But it’s strange that this just suddenly started happening, whereas it was perfectly fine for several months. The system is no longer bogging down, which is good, but it does drop the calls during the apply, and that error still pops up.

Just an update to this. Today I upgraded the MOH module and miraculously the problem is resolved! Not sure if the MOH module had anything to do with the issues or not, but glad it’s fixed!

If I make changes and click on Apply Configuration Changes for a third time, it will slow down and finally lock up.

Tried to SSH. Used TOP to see the processes, but still waiting for the prompt to show.

First thought it was fail2ban which sometimes shows 85% for several seconds.

This is my third install of PIAF, each with different hard drives. I looked at the screen, and it says that the CPU was stopped for 10-seconds.

I’m going to try to spread out the changes that I make to see if the few changes in just a few minutes might be the problem. It might still be reloading when the GUI says it is done.

Upgraded only the recommended items.

  • PBX in a Flash Version : 1.7.5.5
  • FreePBX Version : 2.6.0.3
  • Running Asterisk Version : LOADING
  • Asterisk Source Version : 1.4.21.2
  • Zaptel Source Version : 1.4.12.1
  • Libpri Source Version : 1.4.11.4
  • Addons Source Version : 1.4.12

CentOS release 5.5 (Final) :32 Bit Kernel: 2.6.18-194.17.4.el5