Full backup on bootable disk

Dear all,

I’m working on a script to hot backup full disk on a new bootable disk.

  • One sata disk is in the server with original Freepbx install with 3 partitions /boot /root and swap disk is SDA
  • I plug a second disk on USB port recognized as SDB (this is a usual scenario)

I have make a script to upload on / , chmod 777 and run

echo "# MAKE EMPTY DISK #"
sudo dd if=/dev/zero of=/dev/sdb bs=512 count=1

echo "# MAKE PARTITION TABLE #"
sudo parted -s /dev/sdb mktable msdos

echo "# CREATE PARTITIONS BOOT #"
sudo parted -s /dev/sdb mkpart primary ext4 0% 350M
sudo parted -s /dev/sdb set 1 boot on

echon"# CREATE PARTITIONS SWAP #"
sudo parted -s /dev/sdb mkpart primary linux-swap 350MB 1GB

echo "# CREATE PARTITIONS ROOT #"
sudo parted -s /dev/sdb mkpart primary ext4 1GB 100%

echo "# FORMAT PARTITIONS #"
sudo mkfs.ext4 /dev/sdb1
sudo mkswap /dev/sdb2
sudo mkfs.ext4 /dev/sdb3

echo "############################################"
echo "################ QUESTION ? ################"
echo "#################### ! #####################"
echo "########## SHOW DISK INFORMATIONS ##########"
sudo fdisk -l
echo “Check if you can see the 3 sdb partitions above, if yes enter y and the backup process will start”

read -p "If you can see sdb partitions above, enter Y or y to continu otherwise press N or n to exit " -n 1 -r
if [[ $REPLY =~ ^[Nn]$ ]]
then
echo ''
echo 'Cancelled’
exit
fi
echo "# MOUNT PARTITIONS #"
sudo mkdir /mnt/target
sudo mount /dev/sdb3 /mnt/target/
sudo mkdir /mnt/target/boot
sudo mount /dev/sdb1 /mnt/target/boot/

echo "# STOP ASTERISK AND MYSQL #"
sudo amportal stop
sudo /etc/init.d/mysql stop

echo "# CLEAN AND EMPTY LOGS #"
rm /var/log/.0
rm /var/log/
.1
rm /var/log/.gz
rm /var/log/
.tgz
rm /var/log//.0
rm /var/log//.1
rm /var/log//.2
rm /var/log//.3
rm /var/log//.4
rm /var/log//.5
rm /var/log//.6
rm /var/log//.7
rm /var/log//.8
rm /var/log//.9
rm /var/log//.gz
rm /var/log//.tgz

for i in find /var/log/ -name '*'
do
cat /dev/null > $i
done

echo "# READY FOR BACKUP IT WILL START IN 5 Seconds #"
sleep 5

sudo rsync -axu --progress / /boot /mnt/target

echo "# BACKUP DONE FIX fstab and grub UUID - REMOVE network cards Fixed MAC #"
sleep 2

sda1u=$(lsblk -no NAME,UUID /dev/sda1 | awk {‘print $2’})
sdb1u=$(lsblk -no NAME,UUID /dev/sdb1 | awk {‘print $2’})
sda2u=$(lsblk -no NAME,UUID /dev/sda2 | awk {‘print $2’})
sdb2u=$(lsblk -no NAME,UUID /dev/sdb2 | awk {‘print $2’})
sda3u=$(lsblk -no NAME,UUID /dev/sda3 | awk {‘print $2’})
sdb3u=$(lsblk -no NAME,UUID /dev/sdb3 | awk {‘print $2’})

sudo sed -i s/$sda1u/$sdb1u/ /mnt/target/etc/fstab

sudo sed -i s/$sda2u/$sdb3u/ /mnt/target/etc/fstab

sudo sed -i s/$sda3u/$sdb2u/ /mnt/target/etc/fstab

sudo sed -i s/$sda2u/$sdb3u/ /mnt/target/boot/grub/grub.conf

sudo sed -i s/HWADDR=.// /etc/sysconfig/network-scripts/ifcfg-eth0
sed -i "/^[ \t]
$/d" /etc/sysconfig/network-scripts/ifcfg-eth0
sudo sed -i s/HWADDR=.// /etc/sysconfig/network-scripts/ifcfg-eth1
sed -i "/^[ \t]
$/d" /etc/sysconfig/network-scripts/ifcfg-eth1

unlink /mnt/target/etc/udev/rules.d/70-persistent-net.rules

echo "# COPY MBR #"
dd if=/dev/sda of=/tmp/mbrsda.bak bs=512 count=1
dd if=/tmp/mbrsda.bak of=/dev/sdb bs=446 count=1

echo "# FINISH NEXT STEP IS TO MANUALY ENTER the COMMAND #"
echo " grub "
echo "Then press enter "
echo "and past the following code: "
echo "#######################"
echo "device (hd0) /dev/sda"
echo "device (hd1) /dev/sdb"
echo "root (hd0,0)"
echo "setup (hd0)"
echo "root (hd1,0)"
echo "setup (hd1)"
echo "quit"
echo "#######################"
echo " press enter and you can shutdown and reboot with new disk "

The problem is the end must be done by hand in the grub CLI
and I would like to send the grub CLI commands in the shell script, is it possible ?

Also my side all tests are successful but I have friends reporting some Kernel panic on new disk
with:
mount: you must specify the filesystem type
kernel panic not syncing pid 1 comm init not tainted 2.6.32-431.el6.x86_64 #1

Any idea ? I realy whant to setup an universal script to clone disk

I would like to specify

  • I don’t want to use dd because I want universal backup on any media size
  • I don’t want to use any clone tool because i want a hot backup able to work with a CRON
  • I don’t want to use RAID features as I want a disk able to boot on a new hardware in case of hardware failure
    (this is the reason why the script erase the NIC UUID)

You might want to look into mondorescue

@Dicko
I can see you’r a rescue specialist in other post
You can see globally that my script do what mondo do with additional feature to fix the UUID problems
about UUID I have try first to replace UUID with /dev/sdax on grub and it’s not working for me.
Only the end off the job is a problem for me “the grub CLI command” how to do it with a shell script ?

In another world, I don’t want to reinvent the wheel, if you have a howto for mondo and 2.6.32-431.el6.x86_64
I’m interested.
I have take a look in mondo FTP and not see what to do to script a full backup of my freepbx distro.

I wouldn’t worry about the disk uuid but feel free to use any alias in /dev/disk/* by-id , by label or just /dev/sda1 etc. do that in grub (un comment out (ouch!!) GRUB_DISABLE_LINUX_UUID=true ) and edit fstab before mondoing, also delete the interfaces in /etc/udev’ persistent network rules. as they will be rebuilt on the next reboot, both the original machine and the mondo’ed version. You can also comment out the MACADDRESS in your ethnic definition files, it is unnecessary .

man mondoarchive

Lays out exactly how to script it for a cron job without the TUI look to the options -E to ignore your log files, (they will be rebuilt), maybe the -U and -d for writing a bootable USB drive , -i to make local iso images that can be dd’ed , -n for nfs files , you would boot from the mindi.iso for that., -g to make it cron happy there are lots of example usage’s ta the bottom.

Thanks,
I will try mundo

A couple of practical suggestions that evolved, you might also want to exclude /var/spool/asterisk as it’s content becomes stale quickly and tends to be large, especially the monitored calls , I use rsnapshot to back that up remotely on a timely fashion, maybe you want to add the mysq and sqlite3 l databases.

Personally I mostly moved to VM’s in an HA configuration. Much quicker and no loss of restored ‘state’ .

For small "under the sink’’ systems however you can’t beat mondo with a bootable usb stick that you (or your non-stupid client) can restore to last Friday after hardware failure or being compromised. Also another cron job that ‘images’ to an off-site iso nightly for if you need to do a truck-roll.

Finally I have make a try again to remove the UUID in grub and replace with /dev/sdax target partition and it work, so it’s a good new.

Also I think I can answer to myself for the end of script without using the grub CLI after a day brain storming
I think the solution is with chroot with mounting the original /dev on the target disk to build the grub and MBR on target disk.
By this way I think I can finish the shell script making a full bootable backup by just invoking the script.

I finally prefer using my own tool because I know exactly what it do :slight_smile:

Personally I do think you have “reinvented the wheel” but if you are happy with your solution, then go with it.

Don’t expect anyone else to trust you though, it will only work for your small deployment, grub2 nor grub nor lilo is not part of linux, the Mondo guys have been doing it for years, lvm, raid all covered, so far no ZFS, does yours?

But otherwise, you need to support LVM’s as most FreePBXers will have that installed, (a bad choice but there you go) go bust your chops. and add support for that :slight_smile:

I agree i try to reinvent the wheel, but for some reasons

My job is to make software for idiot as I was an idiot before and surely still now (but less).
Right I make some pro softwares this is my job.
When some customer or integrator call me and ask me a question how to do that or that
if I can answer just click here and take a coffee believe me, everybody is happy.

I make software versions for multiple platforms and Linux distro: raspberry version Debian ARM7, rooter embedded Debian ARM9, X86 both Debian and Centos, all my Debian versions have a bootable backup easy procedure for idiot.

If you re-read your above answers, imagine that an idiot can’t understand and do what you explain
editing many files, changing xxx, save, deleting, … before mundo

OK, if it works for you . . .

But, as to reinventing the wheel and that you posted here in a place where 90% or more installed from an iso that will be basically RHEL5 or 6 , if you want to learn, then perhaps do something like:-

if [ ! -f /etc/yum.repos.d/mondoarchive.repo ];then  
	wget ftp://ftp.mondorescue.org/rhel/6/x86_64/mondorescue.pubkey;gpg --import mondorescue.pubkey
	wget ftp://ftp.mondorescue.org/rhel/6/x86_64/mondorescue.repo -O /etc/yum.repos.d/mondoarchive.repo;	yum -y install lzop mondo 
fi

Obviously use the os, os level architecture as appropriate

when that has finished you can expore the mindi bash script, which basically does all the prepreparion of the booting and partitioning schema.

less $(which mindi)

It would help you to know how Bruno et al take care of various boot loaders , file systems, md’s and LVM’s,
You will need that to be ubiquitous over all your stated needs, mondo supports most every linux on intel and amd ,.

http://trac.mondorescue.org/

might help you adding it’s functionality to “whatever” as to embedded OS’s on routers et al, I think.

Anyway for those that want it you could

sed -i 's|^HW|#HW|'  /etc/sysconfig/network-scripts/ifcfg-*
sed -i 's|^UU|#UU|'  /etc/sysconfig/network-scripts/ifcfg-*
sed -i 's\^SU\#SU\g'  /etc/udev/rules.d/70-persistent-net.rules

before running or croning something like, for a local iso image

 mkdir -p /backup;$( which mondoarchive) -OiV -E "/backup|/var/log/|/var/spool/asterisk/"  -d /backup -s 4480M -p  `hostname`-`date +%Y-%m-%d

for a NFS

$(which mondoarchive) -OVn nfs://192.168.1.2/home/nfs -d /Monday -E "/mnt/nfs|/var/log/|/var/spool/asterisk/"

for a USB

$(which mondoarchive) -OUV -d /dev/sdb -s 16g -L

etc, etc,

The disk UUID’s will be fixed up after nukeing into the new machine, with a bit of less idiocy you can use expert mode and move between single disks and raid’s of many flavors, and file systems of sata/scsi,ide or even vda for virtualization

If you want to remove UUID’s then maybe something like

sed -i 's|UUID=.* |/dev/sda2 |g' /boot/grub/grub.conf
sed -i 's|^UUID=.* / |/dev/sda2\t\t /|g'  /etc/fstab
sed -i  's|^UUID=.* swap|/dev/sda3\t\tswap\t\t\t    |g' /etc/fstab
sed -i 's|^UUID=.* /boot|/dev/sda1\t\t\/boot|g' /etc/fstab

for a RedHat type OS before imaging.

It would be easy to chain all that into a script, but we are not allowed to do that here, so I will leave that as an exercise for the student :wink: