This morning got a call from one of our producers that calls were not coming in for a show that usually gets hundreds each hour. I tried dialing and got a reject message on the PRI. Logged on the gui and got the dashboard up but nothing else would load. the browser just hung. I was able to get on the CLI no activity was showing up in the verbose mode. also wasn’t able to place out from SIP extensions. Top -c didn’t show any high processor usage so at that point I just rebooted and everything came back to life.
Which logs might show what happened? the asterisk log didn’t and neither did the messages log.
Well, /var/log/messages should be the place to look. However, if a process just stone-cold “hung”, there may not have been a message associated.
But… let’s look at what we do know.
You got a ROT from the PRI… so that service couldn’t handle the request. Since you didn’t get a RingNoAnswer, that indicates to me that the PRI was actually in an alarm state, and not just going unanswered.
The GUI loaded, but nothing else; the browser just hung.
You didn’t have high processor usage, so it doesn’t sound like something was “running away”.
I would say to check /var/log/mysqld.log and see if you see anything there. Since multiple systems were affected (your PRI handler, the dashboard, calls) which are all different processes, the common thing to all of that would be mysql.
I just had the same problem on a server with 2 E1 links. I did a downgrade to 13.22 to see if that fixes the problem. Version 13.26 was installed on Friday and there was a problem on Saturday and again today.
Note: Apparently version 13.26 has a bug related to DAHDI or PRI. Once we downgraded to 13.22 everything was fine and the server is up and running.