Backup Module Ideas - and a Few OTTS Seats Left

Blog News
1

The backup module has been a long time member of the available FreePBX modules. It has been well received and has helped many users recover from potential disasters, and for that we are all very grateful. It has also been plagued by various issues and has been one of those modules that many felt could really use an overhaul but at the same time, it's not the most exciting module that attracts interest from the volunteer development team.

Well Tony over at Schmooze Communications has decided that its day has come and volunteered to move it to the top of their priority and get going on a rewrite. So kudos to them for taking the initiative and now it's your opportunity to help drive the requirements and features to help make sure that this is the last backup module that we need to write, and that it can serve the needs of the varying distributions of FreePBX out there so that there is a common backup ability available amongst FreePBX distributions AND not only can it address the backup needs that might be particular to a distribution, but it can also help users move between FreePBX distributions as their needs (or evaluation process) changes.

There are a number of Backup Module feature requests outstanding that I'm sure will make for a good start, but I suspect that there are plenty of other good ideas that are not captured. So please provide feedback, whether opening a new feature requests or just responding to this post which I'm sure Schmooze will keep their eye on.

Open Telephony Training Seats Left

Our next OTTS event is coming up at Digium Headquarters in just over a couple weeks (March 3rd) and there are still a few seats left. If you are looking for an in depth training in FreePBX along with the other essential telephony and Linux knowledge required to be proficient in this space, this is the place to do it. In addition, learn the much needed sales and marketing knowledge from our successful team who has sold hundreds of systems and can help you learn now to beat the competition while maintaining very strong profit margins.

If you would like to join us, register here and use discount code LASTMINUTE to receive one of two $500 discounts that we have just released to fill up the remainder of the class!

Philippe - On behalf of the FreePBX Team!

2

To me the best type of backup module would be one where it would list the various things that can be backed up and have checkboxes next to them, with “check all” and “uncheck all” buttons available.

The one time we tried to use this module was when we were moving from one distribution to another - the problem was that it moved certain unwanted files that screwed up the new system, when all we really wanted was data like extension settings and route and trunk settings. I don’t know enough about how the backup module works to say exactly what caused this problem, but my point is that if you decide to change from, say, PBX in a Flash to Elastix (or vise versa) you should be able to backup just your settings, custom sounds files, *_custom.conf files, etc. without touching other files. Similarly, if you are creating a new Asterisk box and your current box has something like FreePBX 2.2.x on it and you put the latest versions of everything on the new box, there should still be a way to export settings from the old box to the new.

One thing I would suggest is exporting the settings in some sort of XML file format so that people can read (and, if proficient enough, even edit) them. Not only would this minimize incompatibilities between versions, but if a newer version had a couple of new fields that you wanted to pre-populate in a particular way before re-importing the files, you could do it using a simple word processing program with search and replace and/or macro capability.

3

As far as restoring from older versions of FreePBX that is a desirable feature and something I just discussed with Schmooze this morning. FreePBX already has the ability to do all the migration, so it’s simply a matter of tapping into that ability - basically restoring the old data and then triggering all the migration code to handle any schema and other changes that may be necessary and that is handled by the migration code that comes with FreePBX.

4

If someone is going to work on the backup module, here is something that could use some attention. I had to quit doing database backups because of the following problem:

To backup the astdb, dumpastdb.php calls $astman->database_show(). In php-asmanager.php database_show() calls command("database show ") which calls send_request(‘Command’, array(‘Command’=>"database show ")) which calls wait_response(). Since wait_response($allow_timeout=false) is the default value, any error while processing the return value of "database show " can cause an endless loop. I had been seeing this problem every once in a while with my production FreePBX 2.5.1.1 system, but once I had more than 700 users, this call to “database show” causes an endless loop 100% of the time. I tested this with Asterisk 1.4.11, 1.4.20, and 1.4.23, but always got the same result. I also can cause the same endless loop by entering “database show” through the Asterisk CLI module in FreePBX. On the other hand, entering “database show” at the actual Asterisk CLI is very responsive.

I am using Ubuntu Server 8.04 (2.6.20-15-server kernel) with PHP Version 5.2.4-2ubuntu5.4.

5

I, for one, would love the ability to backup to another machine on the network, or even the internet, straight from the backup module. I don’t know enough about how Asterisk or FreePBX works to know whether this is feasible or not, but there it is…

6

Maybe some consideration should be given to a more open back up concept. If we could do something similar to PFSense, where the backup creates and XML file that can then be both updated with an editor and/or moved to system with a different level of code on it this could be very useful.

I have often run into the issue, since this is a phone system, where I have not touched a box in a long time, and when it did come time to upgrade move from say TB 1.2 to PiaF 1.3 the Backup restore was just no help :slight_smile:

With a good XML construct or other tool, it would be possiable to “upgrade” the backup file by passing it through a script and then applying/restoring this against a current distribution.

Also, being able to add the ability to execute a script at the end of a valid backup to move the resulting file(s) to an archive repository would be of significant value.

7

Hi

Thanks for stepping up to the plate on this one.

Given that the database has a list of all the modules ad their versions in it, and tar’d up, the asterisk database is pretty tiny, it would be good if everytime that a module reload was done (orange bar clicked) it did a MySQL dump on the database, tar’d and zipped it up and sent it to a remote location via SSH or FTP, in the same way that backups are sent now. Tests show its only a few kb, so it should not impact on voice quality if a VoIP call was in progress.

This could mean that a great proportion of the restore could be done from this file, by letting it go and get the modules it needs, which it knows from the database, from the FreePBX repositiory on restore.

This could form the the basis of a roll-back facility as well.

This would only be part of the functionality of backup, clearly there are some config files, CDR, Voicemail and custom voice prompts. But it could be argued that CDR can be lost in many companies with little or no impact on the business (if they do impact on the business, then they should be dealt with under special arrangements), voicemails may have been emailed, so those are available, and config files and custom voice recordings are pretty static, so a major backup should take care of those.

So a combination of a backup of everything to do with asterisk and FreePBX, similar to the way it happens at the moment every day, week or month as appropriate, coupled with an asterisk database backup each time the orange bar is clicked should put a replacement PBX back on its feet in exactly the same config as it was when it fell over, and is pretty foolproof.

The service of providing remote up to the minute offsite backups would make a valuable addition to a VAR’s portfolio, helping to justify his support fee.

Within the FreePBX backup, it would also be good to include other custom directories which may have been added outside the webroot/admin folder.

Yours

Joe

8

Tony that’s great news!

My Wishlist:
All the above ideas… plus.
Off machine backup methods for connection mediums like:
SFTP (My personal preference)
Samba Share (if installed)
Offsite Internet option to something like Amazon’s S3
Encryption Technology

A place to add new single files and another to add folders to add to the backup routine checkbox list.

A place for modules to be added for future additions such as SugarCRM or something else like that where they can have a safe place to cleanly import their own backup processes even if they have to call other processes to start and stop services first.

An email option to report issues and success notifications would be nice.

A notification issue screen would be good too if there was ever troubles, maybe on the FreePBX System Status > FreePBX Notices?

A restore tattle tale function that’ll look at a selected backup to report on what check boxes were actually selected and what version it came from might be useful for laypeople to be used before choosing to blindly restore from something maybe someone else created.

I am watching TV with my wife right now so this probably isn’t as complete a list as I could have made. But this is the gist.

Good luck and I’d be happy to use whatever you come up with! Let me know if you need some testing done.

William…

9

http://pfSense.org has an excellent backup system (derived from m0n0wall). A key benefit of this is the single XML file that can be immediately downloaded and tweaked / edited / used elsewhere as needed. This inherently works across versions (backwards compatible anyway) and is also handy for moving things from one system to another. Pfsense added the ability to download sections, and for modules to contribute to the backup file.

Imagine being able to download all extensions, or trunks or IVR settings in a single click from a simple page. (And then tweak the file(s) and upload them somewhere else).

I think a system like this will also greatly facilitate people helping each other. E.g. “Here is the trunk config I used for provider XYZ. Just restore this XML file and change the user/pw to your account.”

It doesn’t work so well for binaries (a zip file format perhaps?) but for where it does work it is excellent.

cheers, michael

10

Expanding on this request, the ability to backup to more than one location (i.e. local and a remote ftp site) would be excellent. We have lots of customers running FreePBX based systems and the ability to automatically have a copy of their latest config stored on our servers would save a lot of time.

A related feature would be the ability to set an expiry data on the backup so you can keep, say, the last two weeks and not have to go in and manually remove old backups.

Graham

11

expanding on GWalmsley request,

Yes a way to at least localy remove older backups (We don’t do remote backups) but we need the removal of older backups to be smarter the just removing everything older then x days. It would be great to be able to setup a schedule like the following:

We remove backups older then 60 days, EXCEPT for the 1st and 16th of any month. For backups that are older then a year we remove the 16th of the month and keep the first of the month for historical reasons. At this point we do a manual process, I cut a pair of CD/DVD’s of the old stuff once every 6 to 9 months and store them away.

Yes it’s a bit extreme but our CEO and CFO are known for saving voicemails for a long time and then once in a rare while deleting them (only when I complain and threaten several times). Three times now I’ve been asked to recover a voicemail from 6 or more months ago that they suddenly need. So this has become important for us.

12

I, for one, would love the ability to backup to another machine on the network, or even the internet, straight from the backup module.

One workaround for this is to mount a remote file system at /var/lib/asterisk/backups.

I too would like to see this as part of the backup system. Also, a way to schedule deletion of old backups based on multiple criteria.

13

offsite backup removal is in 2.7