Sangoma 7 upgrade script failing


(Alkis) #47

Hi Bryan

Have you had some time to check the logs?

Thanks in advance
BR


(Bryan Walters) #48

@alkisalgianakis Yes, I’ve been commenting on the tickets I’m aware of and trying to see how we can make the update process better moving forward.


(Alkis) #49

Of course, you have been so helpful. I appreciate all your guidance and efforts so far.
Thanks a lor


(Marbled) #50

Hi Bryan!

I posted more info on my ticket…

I was able to get unstuck but I am now stuck again…

I still had /var/run/post-sngupdate, I moved it out of the way…

I ran /usr/sbin/post_upgrade but it failed with yet other broken dependencies…

The main one, unsurprisingly, was nodejs-devel…

I ran

yum upgrade --skip-broken

manually and this updated quite a few things…

Unfortunately after that

/usr/sbin/post_upgrade

no longer wanted to work because it thought it had already run so I removed the file used to track this…

(I figured I didn’t have much to lose at this point…)

Oh yes, I forgot to say that, I am not sure when, network connectivity was restored…

Now it looks like the new repos don’t provide kmod-dahdi (which my Sangoma A200 needs) and I still have the el6 and they seem to complain big time about the new kernel…

Oh, I forgot to mention I had Sangoma hardware, an A200, in that system… I must confess that at one point I was beginning this might be one of the reasons I lost network connectivity since, when the drivers are loaded, it appears like a network card and maybe it was confusing something in the new distro network stuff…

This would also have explained why your in-house tests might not have experienced the problem I and other did, ie not having that kind of hardware in the system.

Though I guess you did test with Sangoma hardware, yes? Yes, you most likely did…

Now, any suggestions about what to do about kmod-dahdi?

Do I have a missing repo somewhere, do I need to replace it with something else?

Thank you and have a nice day!

Nick


(Marbled) #51

Hi!

We are at at least two people who have DAHDI hardware and had network connectivity problems, me and the person who opened the FREEPBX-15489 ticket (he mentions it in this thread: FreePBX 13 -> 14 Upgrade Script kills DAHDI ).

Now in his case the virtualized network card seemed to have something to do with his network connectivity problems while in my case I wondered if my Sangoma A200 might actually have something to do with my network connectivity problems since it appears as a network card of sorts when its drivers are loaded.

Now I did have a small configuration problem at first, something which was not causing any problems before the update but it was fixed quite early and I still had connectivity problems so I am wondering what else could have played a part in this?

Maybe it’s unrelated to the problems we have but @ar_gen_tum, @ldkeen, @nsumner, @alkisalgianakis do you have DAHDI hardware or were you using a VM and did you have DNS/network connectivity problems as well or only dependency problems?

Is there anything which links our problems together…

I was able to get further and now I definitely have a problem related to having DAHDI hardware, the kmod-DAHDI package is not the right one for the kernel and doesn’t appear to be available in the repo… I believe this is what happened to the person who created the FREEPBX-15489 ticket…

Thank you and have a nice day!

Nick


(Marbled) #52

OK, I removed gnome-disk-utility-libs-2.30.1-2.el6.x86_64…

I had seen it in my logs but I thought it wasn’t critical to fix all broken dependencies and I was hoping it might get fixed in a later step of the upgrade…

(Nodejs I knew it needed to be removed and replaced so it was a no-brainer…)

It looks like the upgrade script doesn’t tolerate any broken dependencies in the list package it tries to update…

I removed it and now it went muuuuuch further…

As for DAHDI, I saw this in the script output now:

######### Updating Dahdi for this kernel #########

I do not know yet if it is working though…

I see it recompiled a few things, redownloaded modules and it’s now going through “repairs”…

######### Repairing PHP Version specific modules #########
######### Repairing areminder #########
######### Repairing broadcast #########

This is where it is right now…

I am keeping my fingers crossed hoping it will complete successfully… :open_mouth:

I will keep you posted…

Have a nice day!

Nick


How to Upgrade FreePBX Distro 13 to 14
(Mike Waldron) #53

If you use any VMware specific NIC, i.e. VMXNET the upgrade will fail. You must use E1000 for it to succeed.

I did as well notice that my Wildcard TDM was detected as an ethernet controller with lspci.


Upgrade to v14 error > Network
Install of FreePBX distro fails on VMware
(Marbled) #54

Hi Mike!

So both cards appear as network cards… I wonder if that played any part in our problems…

I am not running this in a VM so that’s not what we have in common…

Thank you and have a nice day!

Nick


(Marbled) #55

OK…

@GameGamer43 had asked me to run /usr/sbin/post_upgrade manually (and remove the previous script) which I did…

It looks like when the script is run that way it needs some help in the form of pressing “Enter” when it processes modules you don’t have (either that or I am starting to hallucinate… :scream:

When it completed I went on the web interface and it worked!

(Pretty nice actually, even nice than FreePBX 13…)

Now like @waldrondigital, DAHDI did not work…

I try to do a fwconsole restart and I got this (in RED…>)

########################################################################
#                          Sangoma Wanpipe                             #
#        Dahdi/Zaptel/SMG/TDMAPI/BOOT Configuration Script             #
#                             v2.51                                    #
#                     Sangoma Technologies Inc.                        #
#                        Copyright(c) 2016.                            #
########################################################################

Warning: Dahdi modules not found: Wanpipe not build for DAHDI
Error: Dahdi and/or Zaptel modules not found! Please install DAHDI and compile Wanpipe for DAHDI!
Starting WAN Router...


wanrouter: Wanpipe Module: wanpipe not found !!!!
        WANPIPE drivers must be compiled as modules
        Check kernel configuration in /usr/src/linux/.config: 
             CONFIG_WAN_ROUTER=m
             CONFIG_VENDOR_SANGOMA=m



Error Output:
================
modinfo: ERROR: Module wanpipe not found.
modinfo: ERROR: Module wanpipe not found.

So it doesn’t find the right Sangoma Wanpipe drivers…

There is a package called wanpipe-7.0.20-8.sng7.x86_64 but it’s apparently unhappy with it so I guess we need an update, @GameGamer43?

Now, I also have another problem, when I rebooted I was no longer able to access the web interface but I am looking into it (after I have eaten something, I am starving… :wink: ).

Apache is running and I see nothing in its logs so I am beginning to wonder if the firewall got activated or if fail2ban is playing tricks with me…

(My PBX is not in the same network segment than I am so either programs might be unhappy with that…)

edit: The firewall is not even running and is still at version 13.0.6…)

Good luck to you all and have a nice day!

Nick


(Alkis) #56

Dear Nick
No network problems or dahdi hardware for me. It is just that upgrade will not complete.

We have run the upgrade on a similar machine which was not properly set up only installed and registered distro 6 to later upgrade to 7.
All worked fine.

However on a operational machine the upgrade did not complete and we are still trying.

All the best with your issues. I will be watching the thread.

Alkis


(Marbled) #57

Hi Alkis!

Thank you!

So it looks like we all share some broken package problems but the added network problems some of us had might be related to our telephony hardware (Sangoma and Digium) which looks like network cards to the system.

In my case the step you had problem with I believe (redownloading commercial packages) ran apparently correctly except for the fact it was waiting for me to confirm by pressing “Enter” for each module…

In my ticket both Andrew and Bryan mentioned that this was because I had disabled the commercial modules I was not using. They have an updated script that is going into code review/QA but in the mean time the “fix” is to enable all modules (fwconsole ma enableall).

I was running it manually as Bryan had asked me to in my ticket so I was able to provide the currently needed keyboard input.

I saw the updated FreePBX web interface.

It was reporting my trunks to be online and Asterisk was, apparently, running fine…

It did report DAHDI (my Sangoma A200 card needs that) as not working) which is why I rebooted almost immediately hoping this would fix it…

It did not, looks like there’s a problem with the package I have, some sort of mismatch with the kernel most likely and no updated package…

I now have 2 main problems (3 if you count a slight cosmetical one):

  • There’s something very wrong with the DAHDI drivers… I am not sure where the dependency between kernel, DAHDI drivers and Sangoma Wanpipe drivers is broken though…

  • There’s apparently something funky going on with the default iptables rules which is blocking access to the FreePBX web interface in both HTTP (port 80) and HTTPS (443)… I know it’s iptables related as clearing all iptables rules fix my problem. While I know the different port assignments pretty well I am quite far from an iptables guru. The firewall software I use don’t require me to use its “peculiar” rule format…

as for the last “cosmetic” problem

When I go from the Dashboard to DAHDI Config (surprisingly this doesn’t happen with everything, my guess is that it’s timing related…) I see a message pull down from the top of the screen (in a pretty “interesting” color) and this message says “undefined”.

I also see the same message if I click on the interface name in Live Network Usage.

My guess is that this is probably a message that tells the user to wait but something is obviously missing somewhere…

I guess I am masochistic because even though I have an almost working system now I will rerun the script as Bryan asked me to (but this time with all the modules enabled so that it doesn’t stop…)

As for your problems, Bryan said that they have an updated script that is going into code review/QA so it might be best to work for it in your case.

If however you decide to try again I suggest you enable all modules with "fwconsole ma enableall* as Bryan instructed me to and make sure, you have no broken dependencies in your logs… The easiest way I have found to check this is to do a “yum upgrade” (but don’t accept the upgrade) once the script loops and remove all (make a list, just in case and make sure you have backups) the packages it reports as broken,

You want to remove everything before it suggest to add the “–skip-broken” switch. It might complain about other problems which appear to be dependency problems as well, but this is what you want to remove, nothing else…

Do make sure you have a backup if there is anything important on that system and be ready to wait quite a while for it to complete…

Thank you very much and have a nice day!

Nick


(Andrew Nagy) #58

All of the problems you have mentioned are because the system thinks your Dahdi cards are network cards. From firewall to the dashboard network interface issues.


#59

Sangoma’s wanpipe/wanrouter driver will always look like a network driver because the Sangoma driver can do a lot more than Dahdi, but unless you need and can condition those point to point connections, it would never be a problem, it will/should never get in the way of ip routing/connections at the dahdi level, which only is an intermediary from the native driver as far as FreePBX and below should see because without extraordinary setup, there will never be a IP route from or to it, unless you want to do point to point fractional ip connections,

To learn more about it, perhaps study

So when you setup Sangoma hardware be careful to ONLY go the dahdi way, all channels will then go to Asterisk/Dahdi. And make sure your underlying OS is bybassing the unroutable Sangoma network interface


(Serggio) #60

I’m using FreePBX on VM and trying updating on VM’s copy. No DAHDI hardware installed, no DNS/network connectivity problems.


(Marbled) #61

Hi Andrew,

Right now, my Sangoma A200 card cannot be blamed for anything as it doesn’t appear like a network card to the system because the drivers are unable to start.

I believe the root of that specific problem is that the following package

kmod-dahdi-linux-2.11.1-22_centos6.2.6.32_642.6.2.el6.x86_64.x86_64

which, it seems probably contains DAHDI drivers, was not replaced by one compatible with the current kernel (2.6.32 was the old kernel IIRC so this package is kernel dependent).

Now there’s something I don’t quite get here… Unless my upgraded distro doesn’t use the same repos as the normal Sangoma 7 distro that would mean that this package is not available to it as well, that would mean that no DAHDI card, my Sangoma A200 included, would work out of the box with a new install of the Sangoma 7 distro…

(Or at least an updated install, it’s possible the current problem might be the result of updates…)

Now that would be, well, “interesting” to say the least…

There’s probably a more logical explanation than that, hopefuly…

As for the card appearing like a network card, that’s how Sangoma and Digium decided to implement those cards and I don’t think it’s possible not to have them appear that way.

Do I think the fact it looks like a network card to the system might be the reason I lost Internet connectivity at one point, it sure is plausible…

For the firewall disabling the iptables and firewalld service was what was needed to be able to access the FreePBX web interface. Only disabling iptables was not enough…

(There is surely a better fix than that but I wanted to do something else of my weekend beside doing this…)

I don’t remember having to mess with iptables with the preceding distro (I have read that firewalld is new with Centos 7 so that I am sure I could not have with it) to be able to access the FreePBX web interface.

Is a system that does not use the FreePBX firewall (like mine) supposed to be firewalled?

Now assuming it should, should it be firewalled from traffic in private IP space (my PCs and my FreePBX system are in different subnets but both are in private IP space (172.16…) so assuming the difference in subnet is a problem it should be relaxed and allow traffic from addresses which are unroutable on the Internet.

As for the dashboard problem, I will see if I can make you a little video tonight (I have to get ready for work). Since my A200 is currently invisible to FreePBX right now I am quite sure it’s something else…

Do you have a “Please wait” or similar message (or maybe icon) which is supposed to appear when something takes a while to do? This is what what I see appears to be used for but instead I see “undefined” in it.

What “feeds” that popup text or icon? This seems to be missing on my box…

(And by the way, I can see that popup elsewhere as well, it’s just that it appeared to appear more frequently there…)

Thank you and have a nice day!

Nick


(Marbled) #62

Hi Dicko!

Right now it looks like there is nothing I can do configuration wise (setup-sangoma won’t even work) because it would appear that I still have DADHI drivers meant for the old kernel and no update is offered.

The network card aspect of the Sangoma card never proved to be a problem but I think it might have caused my system lost of network connectivity during the upgrade.

At least that is what seems to be the most plausible at the moment

Thank you and have a nice day!

Nick


(Marbled) #63

Hi Serggio!

Thank you!

So it looks for now that only people with DAHDI hardware might have lost network connectivity at one point…

I don’t remember, did you create your ticket and posted your log files?

My guess would be that you have broken dependencies in those logs…

Thank you, good luck and have a nice day!

Nick


(Serggio) #64

I dont know, is it right place, I posted:
https://issues.freepbx.org/browse/FREEPBX-15498


(Andrew Nagy) #65

You don’t need to do that. We are aware of the undefined issue


(Marbled) #66

Hi Andrew!

Thank you! I guess you were able to replicate it…

I know that, at least in the past, the Sangoma A200 “network interface” would appear in the network graphs and it would say something like “unknown network” or something similar, I guess you initially thought I was talking about this…

(That problem might still be there but since my A200 is not detected at all right now I can’t say…)

Thank you and have a nice day!

Nick