I am finding more and more that updates are breaking things in my install. I want to be up to date for security purposes but it just get’s worse and worse. I updated modules last Friday and the result was this.
I get a red “Error” on every admin page nothing else.
The UCP node throws “Whoops\Exception\ErrorExceptionUndefined offset: 1
File:/var/www/html/admin/modules/ucpnode/Ucpnode.class.php:156” on the dashboard and my system status disappears and just shows “There was an error asking for Overview. Check the error logs for more information”
IVR Direct Dial is still enabled but has stopped functioning. (this has been the major cause for concern I have been hammered with support calls on that one)
This has been the result on two separate servers I manage. I have backups and could restore the VMs but I prefer to get the updates working.
I have continually updated this server from it’s original release 3 years go. Maybe do I start from scratch with a new install?
I’m just becoming increasingly frustrated by broken updates that have seemed to have been cropping up over the last several months. At one point Voicemail was completely broke until a subsequent patch was sent out a day or two later.
Has anyone else experienced the above issues in question and what is my best course of action you think?
Our new release system was put in place to catch this
It has been in force for over a month. If you have “edge” enabled (not the default) you may experience brokenness. For production systems, you should not enable “edge”.
As developers, we run with “works here”. When we release things are tested on our systems in our environments. They are then tested by a QA person in their environment. We cannot account for all environments or scenarios. The edge track allows for a period of expanded testing where more users with more environments further validate changes. If it makes it through this buffer we typically assume any issue to be an edge case or isolated incident. We attempt to catch as many of these in that buffer but no release system is fool-proof. If you find bugs feel free to report them at http://issues.freepbx.org. Edge or release if we don’t have bug ticket then there is no bug.
I am not in “edge” mode on either of the two system. I will report the bugs and hope for the best. It just seems since Sangoma entered the picture the updates that break things have become more and more frequent. Maybe I’m the only one that feels that way, if so I’m ok with that, it could be our system. Like I said it’s an old release that has been upgraded through time.
i guess unfortunately i have to agree. we have issues with nearly every system now. the certificate manager does not install (gets duplicate CA error - new since the latest set of updated using module admin), we have seen that firewall appears to stop running (this seems to occur without any intervention on our part, we just start seeing tons of fail2ban messages) updating to the latest fixes this), trying to access the intrusion detection from the gui results in a crash (updating to the latest fixes this), backups don’t run any more (this is new since we upgraded using module admin. we do not run edge. these are all systems now running 10.13.66-9 Freepbx version FreePBX 13.0.113
Exception: DateTime::__construct(): Failed to parse time string (-- 00:00) at position 0 (-): Unexpected character in file /var/www/html/admin/modules/backup/vendor/touki/ftp/lib/Touki/FTP/FilesystemFactory.php on line 64
Stack trace:
Yes this is because your FTP server is returning bad time values.
I’m sorry you both feel thing way. Certificate manager actually went through the edge testing and met the requirements and was pushed through. Same for backup and restore.
With backup and restore there are thousands of different FTP setups and configurations out there. It’s hard to set even a small subset of those issues.
Let us know what you’d like us to do to help gain back your trust.
unfortunately the last four systems we updated experienced the same issues. on all four, we had previously run through the https setup (system admin) for the gui, so i suspect that there must be some relationship between the fact we had this setup for the web gui and installing the certificate manager i just don’t know what just yet. as to backups, i won’t argue about many different ftp configurations, but the back up routine was running on freepbx gui 13.0.109 and after the upgrade to .113 it no longer worked. our ftp config has not changed in donkey’s years and of course there could be something wrong with it, but the fact that all the other pbx’s running on prior levels work given me pause to consider where the error might be. as a data point the ftp server is a bog standard windows 2008 server (i know this is old). i just tested a few other servers that were running the .109 gui and they work just fine, but every system that has moved to .113 fails with the same error on the back up. i am open to any suggestions here.
Did you upgrade Certificate Manager. I ask because the issue you are saying is still there was confirmed resolved by the original reporter. I dont think you upgraded.
That’s great that it worked fine for you. Did you know we had several bug reports over the last year where it didn’t work at all for certain people. Which is why we’ve tried fixing it for them. Unfortunately it ended up breaking for you. FTP is becoming a real headache for us. At this point I am going to suggest removing it.
We have a bug report on this and without feedback and before seeing this the assumption is that this was a Windows FTP server implementation. I have been reworking the FTP stuff and it is a nightmare. We have however closed about a dozen other bugs while reworking ftp. It seems the library we use has some special handlers for windows so I need to either add a windows setting or somehow detect windows and change up the connection mid stream. This is on my plate and something I will be looking at. I need to get a windows ftp server that acts this way so I can test any fixes I come up with.
certificate manager - very interesting - we updated several systems at about 6:30 am today. it failed every time on the certificate manager so we disabled it and completed the upgrades. i just went back to those systems and did the cert manager update and it worked. so my guess is there must be something in the dependencies that is not right
backups - i have a windows server that i am happy to give you access to if you need to test. i can set up a user id/password and directory.
The fix was most-likely in edge waiting to be released and tested. I pushed it out this morning in stable though it hasn’t gone through anymore testing.
All of the above really. I personally didn’t really start having issues until I started uplifting my Distro through system admin. After that I started having more and more module problems with each release. My server that I built far more recently on an already newer distro I’ve had only a handful of issues with. The IVR DID in this instance spans both.
the only thing i would recommend is make sure your backup and restore module is not later than 13.0.21.8. if you go to .9 and are using a windows server to store your backups, it no longer works. the .8 release has a bug in it that it does not delete the older versions but at least the backup it self does work and does transfer and store the file correctly.
a couple of other data points
reboot after an update - i know some guys will shoot me for saying this, we have learned that all to often something does not work correctly after an update and rebooting the system fixes it.
if you get odd errors during an upgrade run fwconsole chown and then try the update again. Again i don’t know why but sometime we will see a bunch of symlink errors on the dashboard. the chown usually fixes them