Want to create a a script to automatically upgrade distro

I’m trying to build a bash script that automatically do upgrades to the distro via the cron tab every month or two. When I manually run the script it goes through everything fine and reboots at the end and it seems to go through all the available upgrades (listed manually in a separate file) but at the end it gets stuck in an infinite reboot loop. Has anyone else written something like this?

Here is what the script does:

step 1) initialize a few variables to check the version of the PBX, check the newest available version, and some possibly extraneous stuff like checking the runlevel (for which rcX.d folder to put a S(tart) or K(ill) script in).

step 2) if the version is the same as the newest available version remove the lines I add to rc.local at the end of the script or rcX.d as well as this script and the version list and then exit the script. If it’s not the newest version then either add start script to rc${runlevel}.d OR add the /upgradescript line to rc.local (I’ve tried it both ways, I comment one or the other out of the script so I’m not changing both at the same time).

step 3) check for /upgradescripts as suggested in the upgrade wiki page. Not necessary admittedly.

step 4) check to make sure the line is in the crontab and if not add it.

step 5) get the version name for the next one on the list and do the wget, chmod, and ./ commands. I had a problem with my script moving on before the downloaded upgrade script from Schmooze finished upgrading so I added && and some menial task (&& echo “done upgrading, moving on”) to make it wait for the other script to finish. That works, at least the first time through.

step 6) check again if we’re on the newest version. If yes remove files and remove lines from rc.local, sleep for 1s to make sure it finishes removing the files and lines, and then shutdown -r 15 “the script thinks we ARE at the newest level”. If not at the current level it does shutdown -r 15 “we are NOT at the newest level, make sure this should be restarting. ctrl C to cancel shutdown”. I added the text so I can see which shutdown command was being triggered.

I’m not sure what all is happening since watching the console doesn’t show me much. It shuts down and I see that white line moving across the screen. As soon as it gets all the way across I see a brief glimpse of the upgrade routine going through all the steps and then it shuts down again. Since I don’t have time to get my ssh session to reload I’ve been leaving a tab open in my internet browser and refreshing the admin/config.php?display=sysadmin page and can watch the version number ticking upwards but once it gets to the last item on the list it just keeps rebooting without a good way to stop it.

Other relevant info:
-I’m testing this on virtual machines that I can quickly recreate when I break them.
-I’ve tested on both the 32 and 64 bit versions.
-So far I’ve only tested starting with distro levels in the 5.211.65 track
-I’ve tested separately each of the steps in my script and they seem to do their job. Just not sure what’s causing this reboot loop.

Any insight is appreciated, thanks.

Sysadmin pro does automagic upgrading and a bunch of other stuff for $25. You could reinvent the wheel but it all depends on what you feel your time is worth

I should have seen that response coming. What spurred this project is that we have a couple of clients on OLD systems (1.818-62 track; these are people with the PBX in house and don’t like paying for support) and I was hoping to just start the script and let it run through everything up to current. Will Schmooze do all the upgrades from the past couple years for $25? If so I might take you up on that.

Will your script be able to check if the system is running properly after the upgrade & roll back if it isn’t?

I personally wouldn’t automate any upgrades on servers, you really need to be there to make sure it’s done right.

1 Like

Not automatically, but since we’ll have a backup on our FTP server we could (if need be) reinstall the old distro and restore the backup.

Alright, after digging through the System Admin module on a PBXact machine I see some handy features like more notifications settings and sections for DDNS and VPN. I’ll give it a try on a newer system first and see how it works.

I still would like to know what would trigger a reboot loop though for any other script that I end up writing that I want to start on boot.

Just in case anyone is curious I looked at this again recently and figured it out. It was really stupid actually, there was a typo (an extra period) in the version list so when it got to that upgrade it kept downloading a script that didn’t exist, run through the motions and reboot because it wasn’t on the latest version. I put in some extra checks to kill the script if something goes wrong.

Also, it seems to work regardless of which route you take with the starting of the script on boot: adding a line to /etc/rc.d/rc.local or adding a whole script to /etc/rc$runlevel.d/ .

Because this is kind of a dangerous script the next step before deploying this is to automatically take a snapshot of the VM before the upgrade, and delete the snapshot after the second or third iteration (presumably we’ll catch any mistakes by the time we do another set of upgrades).