Freepbx astdb replacement functions could allow clustered astdb-like SQL database

Hi,

FreePBX makes extensive use of astdb via Set(DB), etc.

In my setup I have two freepbx servers configured as a “master-master cluster” which uses DUNDi to lookup sip extensions on each box.

In my scenario, any sip extension can register on either one of these servers.
Suppose extension 101 dials *78 which sets DND on (in asterisk) and sets a DB key value on pbx server 1. Then suppose server 1 goes down for some reason and extension 101 auto-registers on server 2. If someone calls 101 then the phone will start to ring but should report BUSY.

So in a more general view, I was wondering if all DB calls made by FeePBX could/should be customizable. Instead of using astdb, FreePBX could use an SQL database or at least allow the administrator to choose between astdb or sql. So instead of using Set(DB) FreePBX could use a generic function/macro which in turn would call either DB or something else the admin would define (eg. func_odbc…). Same for reads and deletes. Please take a look at http://www.asteriskpbx.org/func_odbc.

The major advantage of using an SQL database like MySQL or Postgresql is that you can easily setup a master-master cluster.

Has anyone dealt with this before?
Does someone think this could be of interest?

we would like to see and option to replace the astdb at some point with an external DB. Not just for clustering, it is also useful to be read/write values without going through the manager from outside. If I’m not mistaken, I think callweaver has replaced the internal berkley db with sqlite. My question for this is if it is not better to literally replace the astdb at the asterisk level with something else so the dialplan would not have to change. The challenge I have observed in the past of which this seems to have the same issue is the need to open the database before using it in every channel and then potentially close the connection - without doing such, you run out of db handles. This is the case in MySQL. By transplanting it at the Asterisk level, it could all be done without any change to the dialplan at all. Have you looked to see if anyone has done that?

One of the reasons for the Callweaver fork of Asterisk 1.2 is the replacement of astdb with sqlite.

I don’t know however if sqlite can do master-master clusters like MySQL. Never used it.

I didn’t know about the db handles issue and that may just be what’s keeping Digium from switching.

I guess I’ll have to wait and look for a custom solution until then.

Thanks for your comments.

file started to use sqlite3 instead of astdb: http://svn.digium.com/view/asterisk?view=rev&revision=75398 But he hasn’t done anything for a while though. Callweaver had some seriuos issues with sqlite: http://www.callweaver.org/ticket/201

Maybe func_odbc would be the way to go. Might be able to do it as a amportal.conf setting, and twist the dialplan to use func_odbc instead of astdb. But for “normal” users astdb is more than enough, but i had an project last year that would really benefitted from an mysql db instead.

/niklas

pnlarsson,
have you scanned to see if anyone has created a replacement for astdb within Asterisk. I still feel that doing such transparently would be an ideal way to address it. One of the issues that I believe exist with all of the odbc, mysql, etc. dialplan level alternatives is that each channel must open and subsequently close an individual db connection which means you can run out of handles either because of too many simultaneous connections or because connections are not properly closed (‘handle leaks’).

The difference between using asterisk-addons’ MySQL module and using MySQL via unixODBC/MyODBC (or any other engine through func_odbc) is that in the first case you have to worry about closing the connections but in the second case the database connection is permanent. I don’t know how “reliable” the asterisk system would then be and I don’t know if there would be a noticeable performance penalty.
So I guess the “db handle leaks” would not be an issue with func_odbc but it would require changing the dialplan and replacing DB calls witgh func_odbc calls.
However, I don’t think anyone’s working on “replacing astdb transparently” within Asterisk.
However, 1.6 is introducing flexibility within Realtime.
Realtime users will be able to INSERT and DELETE custom entries at will but, still, it would require changes to the dialplan and replace the DB calls with something like Set(REALTIME(…)=…).

well it’s good that they have created a single persistent connection to the database, that takes on headache away. The changes in the dialplan would not really be too difficult to get over and there is probably even a way that we could handle it automatically in the auto-generated code. However - given the number of outside applications that exist between FreePBX itself and it’s access through the manager, ARI, XML applications, etc. - it still makes more sense in my mind to do it at the Asterisk level. And I suspect it would not be too hard to simply modify the API calls that are currently going to the AstDB code, and put a fairly thin translation layer to have them use the func_odbc instead.

I even came across some old code of an odbc version of app_db which I suppose is obsolete.

http://www.voip-info.org/wiki/view/Asterisk+app_dbodbc
http://downloads.netmonks.ca/app_dbodbc.c

Anyway, I’m sure Asterisk can be modified at a lower level so that it can be configured to use either DB or func_odbc. However, even if we suppose the developers take that into consideration, it will never make it into a pre-1.6 system.

pnlarsson’s suggestion of defining the function to use in amportal.conf or elsewhere is interesting and can be applied to any * version and wouldn’t require waiting for the base code to come up, if it ever does.

Anyway, it’s worth thinking about it, although not a priority.

I’m not worried about the official Asterisk changing. It’s easy enough if to patch and rebuild Asterisk if this were important to someone. My biggest reasoning behind favoring this approach is that it auto-magically keeps anyone’s add on code, custom development, XML phone applications, etc, etc. running. In my opinion, it is the cleanest and most bug free way of making the change. Once you get a few very simple functions working (about half a dozen), you fix everything both in and out of your control.

The problem with func_odbc is that it’s relatively new and it has been backported to 1.4 and 1.2. However, it seems that the 1.2 and 1.4 backports differ (even in functionality according to the author). They are supposed to work but they are unsupported (at least for 1.2).
Also, the 1.2 func_odbc backport (that’s what I’m using/testing) would probably have to be modified even more extensively as it has only two major built-in functions wich could be considered equivalent to ast_db_get and ast_db_put. It lacks equivalents to ast_db_del and ast_db_deltree (unless of course one passes the custom SQL command).
I don’t know about 1.4 and 1.6.
I might be wrong but it looks more challenging than I thought.

hmm, it just occurred to me that replacing all DB calls with func_odbc calls at the asterisk layer will not do what I need.
In my specific case, I actually want Asterisk to use 2 different databases (whether astdb or not).
The core database would have to be server-specific (no replication - no cluster) because it would record things such as SIP client registration. Chan_sip has a call such as:

ast_db_put("SIP/Registry", p->name, data);
manager_event(EVENT_FLAG_SYSTEM, "PeerStatus", "Peer: SIP/%s\r\nPeerStatus: Registered\r\n", p->name);

Correct me if I’m wrong but I don’t want SIP/Registry to be written to a clustered DB otherwise both Asterisk servers will think that the client is registered to itself.
However, I DO want specific dialplan events (such as the *78 DND db setting) to be recorded on a clustered DB as this data needs to be shared by 2 or more servers.