System updates gets exception even after reboot

@Marbled we released a new wanpipe and dahdi this morning. Check it out.

Iā€™m a bit worried. I just noticed these updates, and updated them from the GUI. After a reboot, Iā€™m now getting these errors:

Last login: Fri Oct  6 15:06:05 2017 from 192.168.0.35

Exception: SQLSTATE[HY000] [2002] Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)::SQLSTATE[HY000] [2002] Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) in file /var/www/html/admin/libraries/utility.functions.php on line 204
Stack trace:
  1. Exception->() /var/www/html/admin/libraries/utility.functions.php:204
  2. die_freepbx() /var/www/html/admin/libraries/BMO/Database.class.php:154
  3. PDOException->() /var/www/html/admin/libraries/BMO/Database.class.php:149
  4. PDO->__construct() /var/www/html/admin/libraries/BMO/Database.class.php:149
  5. FreePBX\Database->__construct() /var/www/html/admin/libraries/BMO/FreePBX.class.php:69
  6. FreePBX->__construct() /var/www/html/admin/bootstrap.php:151
  7. require_once() /etc/freepbx.conf:9
  8. include_once() /var/lib/asterisk/bin/fwconsole:12
Exception: SQLSTATE[HY000] [2002] Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)::SQLSTATE[HY000] [2002] Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) in file /var/www/html/admin/libraries/utility.functions.php on line 204
Stack trace:
  1. Exception->() /var/www/html/admin/libraries/utility.functions.php:204
  2. die_freepbx() /var/www/html/admin/libraries/BMO/Database.class.php:154
  3. PDOException->() /var/www/html/admin/libraries/BMO/Database.class.php:149
  4. PDO->__construct() /var/www/html/admin/libraries/BMO/Database.class.php:149
  5. FreePBX\Database->__construct() /var/www/html/admin/libraries/BMO/FreePBX.class.php:69
  6. FreePBX->__construct() /var/www/html/admin/bootstrap.php:151
  7. require_once() /etc/freepbx.conf:9
  8. include_once() /var/lib/asterisk/bin/fwconsole:12
[root@freepbx ~]#

Start mysql (mariadb)

Ok. My system might have been doing this previously. Iā€™m using putty to connect to the server. If I wait a while before connecting with putty, everything seems to come up ok and I donā€™t see that error. I guess itā€™s still being generated in the background. Unfortunately, what I have noticed, is that DAHDi is not loading on reboot once again :frowning:

Right. You need to do what was talked about in the other tickets that exist on this issue. It has to be started after network. This is an upstream upgrade. We didnā€™t do anything here.

Iā€™m sorry, Iā€™ve not seen that specific ticket about starting DAHDi after network. Could you point me in the right direction? TIA.

https://issues.freepbx.org/browse/FREEPBX-15491?focusedCommentId=110997&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-110997

The solution to this thread should be:

yum remove fail2ban-server
yum remove ql2400-firmware
yum remove ql2500-firmware
yum upgrade
reboot
2 Likes

Hi Andrew!

Thank you, will do it tonightā€¦

My guinea pig for this is my home PBX and I cannot currently remote it (long storyā€¦) so this will have to wait until I get homeā€¦

Have a nice day!

Nick

I modified the wanrouter start & stop levels in /etc/wanpipe/wancfg_zaptel/wancfg_zaptel.pl but that didnā€™t make any difference.

I then downgraded the two wanpipe files (kmod-wanpipe.x86_64 wanpipe.x86_64) back to version 7.0.20-9.sng7 (from 7.0.20.13-1.sng7) which I was using before todays upgrade, and DAHDi is starting on reboot again!

Note that Iā€™ve kept the DAHDi file (kmod-dahdi-linux.x86_64) at the latest version 2.11.1-11.sng7.

GM I just got back from a evening in philly and just catching up. I did s SYSTEM UPDATE and found these available, should I update or should I wait?

RPM Name New Version Installed Version
kmod-dahdi-linux.x86_64 2.11.1-11.sng7 2.11.1.9.sng7
kmod-wanpipe.x86_64 7.0.20.13-1.sng7 7.0.20.9.sng7
wanpipe.x86_64 7.0.20.13-1.sng7 7.0.20.9.sng7

The only purpose of the downgrade was to verify that DAHDi would work on boot, as you had suggested that the 3 upgraded files would have had no impact on this.

I would like to be clear that on FreePBX13, everything worked fine. Upgrade to FreePBX14, DAHDi would not start on reboot. Upgrade to new kernel yesterday and DAHDi started to work on reboot. Upgraded wanpipe module to the latest version today and DAHDi stopped working again.

Iā€™m very happy to upgrade again, and work with you to establish exactly whatā€™s going wrong.

Edit: @tm1000, Oh! youā€™re post that Iā€™ve just replied to has vanished.

Andrew i have followed your solution and it did work. thanks for that. however i have now lost connection to the web interface. i am getting an ERR_CONNECTION_REFUSED on the browser, although i am on the local network.
calls are processed normally.
any ideas please, since i have not yet found on the forum a solution that works, and i would not really like to experiment.

thanks

Hi!

For this particular fix you not only need to have DAHDI hardware but actually have to have Sangoma hardware which, if I recall correctly, is not what you have in your system (unless I am mistaken you use Digium inspired hardware, no?).

I am not sure this fix would still work after the recent updates thoughā€¦ Chances are it might notā€¦

What I did in my case was switch to a kernel those package supportedā€¦ All it required was to edit the grub.cfg file (IIRC) and indicate that the default was the previous kernelā€¦

Now I have a question, do you have Sangoma hardware in that box or not???

I was sure that you didnā€™t so the fact that you have or donā€™t have the right Wanpipe drivers should not matter as these are for Sangoma hardwareā€¦ This seems to suggest that if there is a problem with the Wanpipe drivers even people which donā€™t have Sangoma hardware are affected which should not be the caseā€¦

Non-Sangoma hardware should not depend on whether Sangoma hardware drivers are initializing correctly or notā€¦

Some of those problem seems to be related to the underlying OS changing enough that the step at which the DAHDI hardware should initialized is no longer the same as it was in the pastā€¦

With Sangoma hardware if the network is not up when the driver is loaded it fails with a timeout after something like 4 minutesā€¦

Another problem in your case seems to be that the DAHDI driver initialization seems to have a big dependency on the Sangoma Wanpipe drivers initialization, something it should not haveā€¦

Thereā€™s something unclear in your description but it looks like what got things working for you was that the Wanpipe drivers were broken yesterday nightā€¦ Itā€™s almost as if thatā€™s what is needed for your Digium derivative card to be initialized properlyā€¦

I think (gut feeling) that under normal circumstances the Sangoma Wanpipe drivers need to be initialized first, then DAHDI but that DAHDI initilization should not depend on whether the Wanpipe drivers have been properly initialized and, right now, it seems to depend on it in some wayā€¦

From what you posted it sounds like what is making things work for you is having broken Wanpipe drivers with your Digium card derivative.

What is making things work for me is, obviously, having working Wanpipe for my Sangoma cardā€¦

Something somewhere in those scripts is enforcing a strong dependency between Wanpipe drivers initialization and DAHDI driver initialization which definitely should not be thereā€¦

Good luck and have a nice day!

Nick

Hi Leon!

I donā€™t remember what hardware you haveā€¦

Do you have any DAHDI hardware and if so, which?

My guess is that if you have DAHDI hardware itā€™s not hardware which use the Sangoma Wanpipe drivers as you would have the same problem I doā€¦

If you donā€™t have any DAHDI hardware at all I think you can update those safely but if you have anything else wait until both chaser and me report that things are working for themā€¦

Good luck and have a nice day!

Nick

@Marbled

Many thanks for the detailed feedback.

I can confirm that I do NOT have any Sangoma hardware. I was pointed to the wanrouter start / stop level mod by @tm1000, which is why I tried that.

I accept that the hardware I am using is NOT digium manufactured, but there is another member (@glew) who is reporting exactly the same symptoms as me, and he has indicated that he is using genuine digium hardware.

Iā€™m not sure if heā€™s tried his system with the ā€˜brokenā€™ wanpipe driver. I saw him posting today asking how to go about the big update that came out the other day. However, I guess that if he does the update now, heā€™ll miss out on the broken wanpipe driver, so he still might not get DAHDi to work (unless he then downgrades wanpipe to the broken version, as I have done).

You can remove wanpipe if you wish. Itā€™s not needed since you donā€™t have wanpipe.

Ok. Iā€™ll try that and see if it works. Thanks

Edit: Oooh. Iā€™m not sure thatā€™s going to workā€¦

[root@freepbx ~]# yum remove kmod-wanpipe.x86_64 wanpipe.x86_64
Loaded plugins: fastestmirror, kmod, versionlock
Resolving Dependencies
--> Running transaction check
---> Package kmod-wanpipe.x86_64 0:7.0.20-9.sng7 will be erased
--> Processing Dependency: kmod-wanpipe for package: freepbx-14.1-1.sng7.noarch
---> Package wanpipe.x86_64 0:7.0.20-9.sng7 will be erased
--> Running transaction check
---> Package freepbx.noarch 0:14.1-1.sng7 will be erased
--> Processing Dependency: freepbx for package: sangoma-pbx-1708-1.sng7.noarch
--> Running transaction check
---> Package sangoma-pbx.noarch 0:1708-1.sng7 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================
 Package                       Arch                    Version                         Repository                  Size
========================================================================================================================
Removing:
 kmod-wanpipe                  x86_64                  7.0.20-9.sng7                   @sng-pkgs                   36 M
 wanpipe                       x86_64                  7.0.20-9.sng7                   @sng-pkgs                   29 M
Removing for dependencies:
 freepbx                       noarch                  14.1-1.sng7                     @sng-pkgs                  163 M
 sangoma-pbx                   noarch                  1708-1.sng7                     @sng-pkgs                  8.0 k

Transaction Summary
========================================================================================================================
Remove  2 Packages (+2 Dependent packages)

Installed size: 228 M
Is this ok [y/N]:

What hardware do you have

Can you run

Uname -r

[root@freepbx ~]# uname -r
3.10.0-693.2.2.el7.x86_64
[root@freepbx ~]#

Itā€™s running on a Dell Optiplex 755