Asteriskcdrdb Backup Problem

We recently implemented full backups of our FreePBX server but we are getting errors from the asteriskcdrdb portion. I have looked around quite a bit online and haven’t found any solutions regarding this specific issue. I have placed the backup logs below:

Full Backup Log>
February 2, 2017, 12:00 am - Backup Lock acquired!
February 2, 2017, 12:00 am - Running pre-backup hooks…
February 2, 2017, 12:00 am - Adding items…
February 2, 2017, 12:08 am - Backup failed dumping SQL database [asteriskcdrdb] to file [/var/spool/asterisk/tmp/backup-3/mysql-4.sql], you have a corrupted backup from server [localhost].
February 2, 2017, 12:08 am - Building manifest…
February 2, 2017, 12:08 am - Creating backup…
February 2, 2017, 12:15 am - Storing backup…
February 2, 2017, 12:15 am - Running post-backup hooks…
February 2, 2017, 12:15 am - Backup successfully completed!

I then tried to do a backup of ONLY the CDR data and got this>
February 2, 2017, 11:36 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: Initializing Backup 12
February 2, 2017, 11:36 am - Backup Lock acquired!
February 2, 2017, 11:36 am - Running pre-backup hooks…
February 2, 2017, 11:36 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: Backup Lock acquired!
February 2, 2017, 11:36 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: Running pre-backup hooks…
February 2, 2017, 11:36 am - Adding items…
February 2, 2017, 11:36 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: Adding items…
February 2, 2017, 11:43 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: /bin/grep: write error
February 2, 2017, 11:43 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: mysqldump: Got errno 32 on write
February 2, 2017, 11:43 am - Backup failed dumping SQL database [asteriskcdrdb] to file [/var/spool/asterisk/tmp/backup-12/mysql-4.sql], you have a corrupted backup from server [localhost].
February 2, 2017, 11:43 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: Backup failed dumping SQL database [asteriskcdrdb] to file [/var/spool/asterisk/tmp/backup-12/mysql-4.sql], you have a corrupted backup from server [localhost].
February 2, 2017, 11:43 am - Building manifest…
February 2, 2017, 11:43 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: Building manifest…
February 2, 2017, 11:43 am - Creating backup…
February 2, 2017, 11:43 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: Creating backup…
February 2, 2017, 11:43 am - Storing backup…
February 2, 2017, 11:43 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: Storing backup…
February 2, 2017, 11:43 am - Running post-backup hooks…
February 2, 2017, 11:43 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: Running post-backup hooks…
February 2, 2017, 11:43 am - Backup successfully completed!
February 2, 2017, 11:43 am - id: bb2ac0b8da1f64a3498af147ba43fc10
data: Backup successfully completed!

We are doing this so we can eventually setup a failover server, but we need the CDR data backed up. Does anyone have any suggestions on how to fix this issue?

Looks like database corruption

mysqlcheck --repair --all-databases

I would (from painful recent experience) suggest that Lorne’s suggestion be amended slightly:

  1. Figure out how big your biggest table is.

  2. Make sure you have enough free disk space to hold a duplicate copy of your largest table.

  3. Run ‘mysqlcheck --repair --all-databases’

Not taking care of the first part just means that mysqlcheck will stuff your disk so full that asterisk stops working until you desperately free up some disk space (may I suggest gzipping the files in ‘/var/log/asterisk’? The Zulu logs, in particular, are quite large.)

Tom

Your machine is running out of disk space

Yes, it was. In large part because Zulu has diarrhea of the mouth when it comes to logging (4.4 GB per week!), combined with the fact that the default logrotate scheme does not compress the log files (~350MB after gzip).

After resolving those problems, all is well.