Kernel panic after FreePBX 13 Upgrade

Just upgraded to FreePBX Distro 10.13.66-6.
Note: We are virtualized on Hyper V 2012 R2 with linux integration services installed (and have been successfully for well over a year)

On the Grub loader, theres the following:

“SHMZ (2.6.32-504.8.1.E16.X86_64)”
“PBX (2.6.32-431.E.16.X86_64)”

Before the upgrade, there was only one option on the grub menu, but I can’t remeber which one. I’m assuming the second on the list. Well, wbviously the first options is what loads automatically, but after the CentOS loading screen, the following comes up:

“Kernal panic - not syncing: Attempted to kill init!”
“Pid: 1, comm: init Not tainted 2.6.32-504.8.1.e16.x86_64 #1
[and then some more stuff starting with “Call Trace:”]

If I restart and select the second grub option, the OS loads fine, and all seems up to date in FreePBX. Question is, why was the new option added to the grub menu, and why won’t it work?

watching this for a response, had the same issue. just running on the 431 for now.

The “more stuff” would provide a clue as to what broke it.

Hi Dicko, indeed. See image below.

And the Grub loader:

It’s a filesystem problem, what the problem is I can’t say, presumably between the virtualized kernel and the underlying hardware. Try another Virtualization method (KVM would be solid, perhaps even a virtualbox under your windoze, but Microsoft are a little dumb as yet about native VM’s :wink: and I am sure Schmooze have better things to do than work around MS ) as a rough guess, turn off disk cacheing and enable disk write-backs if those are options.

Sadly, we don’t test against Hyper V. It has had no end of issues with CentOS and Linux in general. I realise that some people are required to use HyperV, but it’s not something we officially support.

However, unofficially, it’s pretty easy to get some debugging as to why it’s crashing. When you’re at the GRUB screen:

Hit ‘e’ to edit the kernel arguments, and then select the kernel line and hit ‘e’ again. You want to remove the ‘quiet’ from the end of that line.

Hit enter, and then b, and it’ll boot up with much more information, and you’ll see WHY it’s crashing.

(If you don’t get enough, reboot again and remove ‘rhgb’ as well)

Hi guys,

The two PBX’s that I upgraded are in production, so it’s taken me some time find a good time to take them offline. Anyway, long story short: PROBLEM RESOLVED.

The Hyper V Integration Services for Linux version I had installed on the VM was for “RHEL65”. When I uninstalled it, I was able to boot just fine with the first boot option, without integration services installed. But due to some NIC problems I’ve had in the past when integration services are not installed, I wanted to try RHEL66, which I did, and it worked! So, my conclusion is that the CentOS 6.5 integration services drivers were causing the VM to crash.

Here’s the procedure:

You first need to boot into the second boot option, which should allow FreePBX to load. Login and run the following:

rpm –qa | grep microsoft

rpm –e microsoft-hyper-v-<version string from step 1> kmod-microsoft-hyper-v-<version string from step 1>

(i.e. rpm –e microsoft-hyper-v-3.4.20120727 kmod-microsoft-hyper-v-3.4-1.20120727)

*be patient, and wait for command prompt to run the reboot command
*also, note that you cannot “upgrade” to RHEL66, as a bug exists where the /dev/cdrom device disappears after installing integration services (Microsoft says the resolution is to uninstall integration services, lol), therefore you must uninstall, reboot, and reinstall the newer version

Upon rebooting, I FIRST booted into the second boot option, though I am not sure this is necessary. I then ran reboot again. Then I let it boot into the first boot option by default, and wala!

Once booted, the /dev/cdrom device will be back. You can then run mount /dev/cdrom /media followed by cd /media/RHEL66 followed by ./install.sh to install the correct version of integration services (assuming you have attached the ISO to the VM’s optical drive of course).

That’s it! I think for me, upon the next major upgrade, I will uninstall integration services beforehand (or at least make a snapshot/checkpoint in case it wont boot, which I unwisely forgot to do, one of the main benefits to virtualizing).

Thanks a lot. this saved me a world of problems.

1 Like

Glad I took the time to post it.

And thanks all you pros out there for your input too.