Indeed. Same situation just six weeks ago – down all weekend.
I’m running FreePBX 15 on SNG7 and have Edge mode enabled. One issue I have noticed since switching mirrors over the weekend using the command:
fwconsole setting MODULE_REPO https://mirror.freepbx.org
is that the latest version of Commercial EPM on the mirror.freepbx.org is showing as 15.0.5. I’m currently running 15.0.24.28 which was downloaded from mirror1/mirror2.
It is currently showing newer versions of modules such as Asterisk SIP Settings (15.0.6.20) are available (have 15.0.6.19 installed) so it looks like most modules are OK. It’s just Commercial EPM hasn’t been updated. Do I need to raise a ticket somehow or can someone from Sangoma check that mirror.freepbx.org has all of the current modules and is being updated correctly?
Having this same issue again. Can’t connect to Mirror Servers for Modules. @tm1000 why is this a routine occurrence on the weekends?
Did you try this:
@moussa854 I’m hesitant to do that as it’s been reported that they are not in sync with each other, meaning when a module update is pushed it not’s pushed to all the mirrors.
Not sure if you did not see the announcement but @tm1000 left Sangoma almost 6 months ago and is working over at Clearly IP now with a bunch of other guys including myself.
Nothing to be sorry about. We still try and help out where we can in FreePBX and our business does a lot around FreePBX still just not part of the inside FreePBX team anymore.
@tonyclewis I understand. On a side note, wish I knew about your company 3 months ago when I was looking for new SIP Trunking Providers. Maybe i won’t hurt looking more now, and have Clearly IP as a redundant provider for me
@joshzone90 … @tm1000, as well as myself and quite a few others, are no longer with Sangoma, we’ve left to start something new, but of course, we are still passionate about FreePBX and Opensource, so you can find many of my ‘new’ coworkers around here still helping out, but we wouldn’t be able to provide insight into why this happens every weekend. Remaining employees at Sangoma such as @mattf and @jsmith would be a better fit for that question, I’m sure they are digging into the issue with the FreePBX mirrors going down every weekend and can provide feedback. Unfortunately, I don’t see that they are monitoring the opensource mirrors on their status page https://status.sangoma.com/ so I guess there is no way to know if they are down for everyone or just you. At our new venture, we don’t have a need to monitor them.
Oh wait… I was wrong, they are monitoring it… “It’s Operational” with no Incidents reports so… still will need to have @jsmith dig in and see what’s up with their mirrors.
mirror servers were up and running for me earlier this morning and i just performed some tests with 13 and 15 systems - no issues
ill open an internal ticket regardless and ask they review historical stats
ty
@tm1000 and @tonyclewis designed the mirror servers, right? These high-maintenance servers seem to have some design flaw… 
(Brotherly love to all FreePBX team past and present!)
No clue whats wrong. I do not have insight into what the issue is that they have been going down. Of course it could be lots of different reasons. I have not been on the inside of Sangoma stuff since April but it seems to be happening more often lately but I am sure someone with inside knowledge will reply on whats going on.
Let’s put some of this into perspective. The repos where down in Jan and April this year, which put 2019 on the track as the last few years. Which is the repos going doing at least 2-4 times a year. The repos have been reported down almost every December for the last three years. They were down in Nov of last year and one or two times earlier in 2018.
Historically speaking that means the repos where down on an average of every 3-6 months per year. Some years less some years more. It would also seem the frequency of the repos being down jumping to almost every 30-60 days now started about July 2019, so oddly about 3 or so months after the last fix in April 2019.
Taking a step back and looking at this in whole, this isn’t a new problem. This is a problem that has existed for a very long time and instead of being resolved properly it was being managed to limit the frequencies of down time. It’s sounding like the only real fix is to give the repo network/systems an overhaul but until that can be achieved the only thing that can be done is to continue to manage the issue.
mirror1 and mirror2 were originally written by Philippe and rob over 15 years ago. It’s very old. It was slightly updated by the schmooze crew back in 2014 but still supports freepbx systems all the way back to 2.3. Yes. Even today it supports freepbx 2.3! Infact anything older than freepbx 12 switches to the old code. Imagine hundreds of thousands of elastix machines using 2.8 connecting daily. That is what mirror1 deals with daily along with many attempts at DDOS attacks. Sometimes successful.
The overhaul is https://mirror.freepbx.org. It’s a different server (dockerized for one, not that docker is a complete solution but it has some fault tolerance) than http://mirror1.freepbx.org. It was a complete rewrite by @jfinstrom and @xrobau so its code base is only about a year and a half old.
That’s why switching to or using https://mirror.freepbx.org is the solution
New installs of 14 and 15 already use the new https server. It’s just a matter of switching older installs to use the new url. You have to slowly phase something like this in which is why it hasn’t been forced (I suspect)…
Additionally the new server doesn’t support 12 or lower (might not even support 13? I can’t recall). So you can’t just turn off mirror1.
2 Likes