We had a massive storm yesterday and ended up with a brownout situation, then the power failed completely for several hours. The UPSs were drained and things shut down. This morning, the PBX was in a split brain condition, with mysql running on the primary and everything else on the secondary. Unfortunately, asterisk refuses to start on the primary.
So far, I’ve tried looking for corrupt files, file system problems, power cycling it, swearing at it, looking at config & log files, etc - nothing has helped. I’m to the point where I’m making no progress & am pretty much out of ideas of what to try. Unless someone has suggestions, I’m at the point of reinstalling the one node, then rebuilding the cluster.
With all the little tweaks I’ve made since getting the system, many of which are not stored on the mirrored drive space, I’d rather not wipe the drive… I know that I’m going to miss something and end up being bitten by it. Both PBX systems currently have 480 gig SSDs in them - I replaced the 240 gig that they came with almost immediately as they were nowhere near large enough to store the amount of call recordings that sales told me they would. (A sales person exaggerate? NEVER! ) Even after doubling the size, they still only barely hold the 3 months management wants.
Because I’m looking to replace the drive in one already, I was thinking it’d be a good opportunity to increase the size of the drives in both systems. I cannot find a matching 480 gig drive locally, but 1T drives are readily available. Unfortunately, I do not know how the reinstall would go having the new drive be 1T while the currently active node is 480 gig.
My assumption is that I’d install PBXact on the new drive, join the cluster, let it sync, then have the freshly installed system take over as the active node and repeat the whole process for the other system. Does anyone have any experience with this or input for me? I read through the VG resizing document on the wiki, but that assumes you’re starting with a healthy cluster and that the hardware on the two machines exactly matches.