FreePBX 15 to 17 restore breaks Endpoint Manager (EPM) (again)

I found this thread and this thread about the same thing, but they’ve been idle for the draconianly-short 7 day timeout, so I can’t reply to either.

A FreePBX 15 server I manage died a few days ago – catastrophic storage failure. So I restored its most recent backup to a new build of FreePBX 17. Among a litany of other issues, I discovered that EPM no longer works right. When I try to create a template (I haven’t tried much else), I get a “whoops” error, that indicates that a column is missing.

In order to figure out what columns should exist, I had to create a brand-new FreePBX 17 server, to see what columns exist in the endpoint% tables, and then diff that against what I currently had (as a result of the FreePBX 15 → 17 migration).

In my case, the following SQL statements cause the table definitions to match what the fresh-FreePBX-17 database had:

ALTER TABLE endpoint_basefiles MODIFY `model` varchar(2000);
ALTER TABLE endpoint_customExt ADD COLUMN `transport` varchar(20);
ALTER TABLE endpoint_templates ADD COLUMN `plarNumber` varchar(255);
ALTER TABLE endpoint_templates ADD COLUMN `plarTimeout` varchar(2) DEFAULT 0;
ALTER TABLE endpoint_templates ADD COLUMN `ldapFNameFilter` varchar(100);
ALTER TABLE endpoint_templates ADD COLUMN `ldapLNameFilter` varchar(100);
ALTER TABLE endpoint_templates ADD COLUMN `ldapSearchFilter` varchar(100);
ALTER TABLE endpoint_templates ADD COLUMN `ldapDialStringsToggle` varchar(2);
ALTER TABLE endpoint_templates ADD COLUMN `ldapDialStringsConfig` varchar(4096);
ALTER TABLE endpoint_templates ADD COLUMN `phonesWebUIToggle` varchar(2);
ALTER TABLE endpoint_timezones MODIFY `snom` varchar(255);
ALTER TABLE endpoint_timezones ADD COLUMN `fanvil` varchar(5);

I noticed a comment in one of the previous threads that the previous version (15 something) was out of date, and to upgrade to the latest, before taking a backup. I don’t know what version of EPM the FreePBX 15 server was running, but it should have been the latest version; the backup file doesn’t seem to record installed module versions anywhere, at least that I’ve found so far. And while that comment was made with the best of intentions for that specific situation, it’s not acceptable from a software development perspective. And from a sysadmin perspective, it is downright crazy that modules don’t verify that their schema is as expected. (And fix it, if needed, or at least make it obvious exactly what’s wrong.)

Creating these columns in the manner I have doesn’t necessarily accomplish the exact same net effect as what the developers might have intended; if they were executed in a different order or with different syntax, the defaults might be populated differently, or there might have been computed values, or… who knows, there’s no end of possibilities here.

Does anyone know what the database migrations actually should be? Is there anything more than merely creating the missing columns (and resizing existing ones)?

Hi @aNullValue sure will check asap.

1 Like

I believe I know what the issue is here. Based on what I’ve been able to find these fields were added at some point later in v15, either with a later update or as a result for changes in v16. On the limited amount of v15 boxes I could get into I found that only ones that were using the “free” license (i.e. only supporting Sangoma phones) these fields existed. I also found that on boxes that had EPM as a paid license any box that let the initial support term run out (thus can’t update to new versions) didn’t have these fields or not all of them. On the one box that was current on its support fees, they existed.

So my theory is that if you have licensed the EPM module but did not keep your support renew current you are stuck at a specific version (varies depending on the end of annual support) and if that is a version that does not have the updated table structure the Restore process is choking on it.

If that is the case the Restore process logic is assuming that every thing being restored is on current releases of the module in that version branch. Unfortunately, this doesn’t account for restoring backups of commercial modules that are on the most recent version their license allows. So as an example with completely made up version numbers:

If user got EPM and they don’t pay the support renewal they are now stuck at v15.1.13 because that was the most current version released before update support expired. Now in v15.2.4 new table structures are used for anything that version or higher. Last release of v15 branch is 15.10.13.

Now the user is going to backup and restore to v17 but oh no, the restore process expects that EPM v15 should be 15.10.13 (or at least higher than 15.2.4) and doesn’t actually check for lower versions, it doesn’t deal with the fact the lower versions don’t have things expected by v17 and thus pukes on itself.

Since my server no longer exists, I can’t swear to the version number we were running, but 1) we were up-to-date on licensing, and 2) we were running the most recent version that was offered by the module upgrade UI. So while I follow your logic, I’m not 100% confident that it’s correct. I don’t think that the restore process is choking on it; I think that it’s succeeding, and just not caring at all what the schema is. But I could definitely be wrong.

But that might be a distinction without a difference. The recommendation to make sure that the old-server is up-to-date implies that the migrations only run during the upgrade process, which is a fundamentally flawed concept.

Well the v17 restore process has had numerous tickets opened against it plus numerous posts, especially in regards to restoring v14 and v15 systems not working right. Things are missing and errors are thrown.

So there is some correlation with the older the version being restored and the amount of errors being thrown.

2 Likes

The things that don’t work are not esoteric here, @kgupta, it’s really basic, easily caught stuff. Again, I cannot fathom how this tested well enough to be released. You are forcing your install base, us included, to abandon the product due to years-long severe code quality and support issues.

1 Like

The issue has been identified, and the root cause appears to be that the backup contains both the MySQL dump and ALTER TABLE instructions, which are attempting to modify the v17 MySQL schema for the Endpoint module. This is causing errors, particularly when new fields got added or database tables have been modified in v17.

The workaround for this is to reinstall the module after restoring the backup. This will ensure that the v17 MySQL schema is correctly applied. @aNullValue try fwconsole ma install endpoint.

Ideally, the MySQL dump in the backup should only contain data and not schema modification commands. However, since this issue originates from older versions of Backup and it’s challenging for users to update their FreePBX systems to the latest version for backup purposes, we are implementing a fix in v17. This fix will ensure that, during the restore process, the database schema is automatically adjusted to be compatible with v17 once the restore is completed.

Best Regards,
Kapil

3 Likes

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.