Hi,
After I had removed the 2 hard drives raid array and then reconnecting them (order of SATA cable may not be the same), the bios see the hard drives but the freepbx isn’t booting at all. THE bios keeps saying to hook up a boot device. I checked the bios and on the bios, it sees only 1 of the 2 drive in the raid array.
Is there someone who knows what’s going on?
How are your raid devices managed ? What hardware is involved in the raid devices apart from the hard drives?
There is not hardware raid. The Raid is managed by Freepbx.
Something I just realized after booting over and over again. A message appeared that seems to be something that was written into the EFI bios by FreeNAS. and seems to be looking for the NAS thumb drive. Would this cause the issue because I used the computer to try to recover a NAS data by copying it to another drive. The computer does boot with the FreeNAS drive though.
Then read up on mdstat diagnosing and repair,
Basically ‘md’ looks for raid information on each hard disk it finds and tries to mount the most recently used drive so be careful what order and which drive
grub needs to be installed on all drives if using raid1 so sata order might be pertinent
Yes, the drives were setup as RAID1.
How do I read the boot order of the RAID if I can’t even get to the drive?
But the UEFI OS seems to have a path to grub. FreeNAS used Grub, I think. Do you think that the path was loaded into the UEFI CMOS?
Your FreePBX box also uses grub, If raid-1 then boot with only one drive installed, if that doesn’t work then boot with only the other drive installed, if neither work, boot with a recovery system that has mdadm and rescan your disks /re-assemble your md’s and whatever else it takes
Edit: Nevermind, I missed the post about being software RAID. Just start over, save yourself the headache.
Original reply for posterity:
Of course order matters.
Why on earth would you think it was an intelligent decision to remove your boot device?
Assuming you had a real RAID1 on a real RAID controller card designed for hot swap, you will easily be able to boot to a single disk, as long as said disk is where it was to start with.
Either one, it doesn’t matter.
It was RAID1 setup when Freepbx 14 was installed.
I’m now using the SG7 (freepbx) recovery USB (created from ISO) to see if I can figure out how to reassemble it. If you got some commands I can run to get it to reassemble, that would great.
ps: lesson learned. I thought it was like FreeNAS which doesn’t care what order.
mdadm raid-1 devices don’t care which interface the hardrives are attached to but your bios does, if it tries to boot a disk without grub installed on it it will fail, you can usually choose the harddrive to boot from as you start the machine, but when you get the raids rebuilt make sure to check for grub on everything
man mdadm
gives you the basics, google has examples and howtos
This is easiest – yes – to simply reinstall.
But, this server is configured properly and the RAID1 array is not degraded (ran mdadm)
Yes, I have a backup from late December, 2019.
I have used backup & restore before in 13 but it was so bad that I was better off recreating everything – something I would prefer not this time. Unless –
is version 14 backup better? Better in that it will hav all the voice recordings we made for our IVR intact?
What are the caveats I should be aware of?
Thanks Dicko. I have been Googling a lot. 
When I go into UEFI BIOS, I can see only 1 of 2 drives. So I select the one that’s show and I get the message that it can’t boot (I forget the exact words). BUT, as per Servant suggestion, I am reinstalling Freepbx 14 on a separate drive (not the raid1 array) to see if restoring the backup will work or not. Based on what I’ve read, is it possible that Grub is installed on only one drive and I’m lucky enough that the other drive, which has Grub isn’t available?
Coming back, to the UEFI bios problem…I see that it will boot any USB thumb sticks!!! Freepx, FreeNAS, but the RAID1.
“exact words” are almost always important
From the recovery disk boot what does
cat /proc/mdstat
And
cat /proc/partitions
Return?
Thanks Dicko. Here’s the result. I still think that it is fixable because to reinstall and restore backup is a riskier options than to try to make the raid array work again.
So /dev/sdb has had it’s partition table wiped, and unfortunately possibly it’s grub install
sfdisk -d /dev/sda |sfdisk /dev/sdb
will create a partition tables on sdb that is the same as sda Unfortunately sdb is smaller than sda so there might be problems later
you can add back the /dev/sdb partitions to the raids with
mdadm --manage /dev/md0 --add /dev/sdb2
mdadm --manage /dev/md1 --add /dev/sdb3
then
watch cat /proc/mdstat
will show you the progress as the raids are resynced,
presumably /dev/sda1 (and /dev/sdb1) were the boot partition you will have to handle that later
when done you should be able to
chroot /mnt/sysimage
Then you likely 1) never set it up right and 2) never tested it or you would have not done 1.
14 clearly tells you in a pop up on mouseover. Don’t recall what 13 said, but I know you were informed.
Not to mention that you can clearly see what is included and not.
I’m not so sure… It sounded like he had a different disk in there.
Then he needs to fix that, all his screen shot says is that /dev/sdb has no partition table.
I find it hard to understand that sub has no partition because the PC was powered off normally (i.e. no power failure or sudden crash or sudden power outage (it’s hooked an UPS). So to have a partition missing escapes logic.
The hard drives are each 500GB so seeing sdb at 480GB, it appears it wasn’t setup properly??? I don’t see any logic there either because when it was first installed, I ran cat /proc/mdstat and it was perfect.
We’ll see how it goes when I try to restore in version 14 because it was a full back up as per your screenshots. I learned my lesson in 13. A full backup wasn’t really a full back up in v13 though.
Bleakly, it doesn’t really matter whether you understand why or not. It is just a fact in your world, you hosed /dev/sdb . . .
As was otherwise noted, booting a random disk for no understandable reason is NEVER a good idea unless you understand ABSOLUTELY FULLY the ramifications.
So, You Broke it, some are trying to help you fix it, please adjust your attitude . . .
