For whatever reason, this thread has become a hot issue for many, But I believe the underlying analysis is blinkered. and causing all sorts of “hot under the collar” emotive responses.
If you have a couple of dozen instances on a single host, then looking at the individual host’s problems is NOT the place to look, they will do what they can, and it would not be a problem, the one minute scheduled update takes a few hundred milliseconds.
Here is wher the problem becomes apparent, the individual VM can only perform to the level of resources provided by the host.
Here is the rub, the instantaneous load imposed on the host by the n VM’s all requesting the same resources at the same time.
It was this that identified my problem, not a problem on the VM’s , and only a second differential problem with FreePBX, it is a an instantaneous overload on the host, and this will invariably happen if the load is not spread as I proposed.
It would be the same if the host is XEN ,VMWARE or PROXMOX or many other likely local deployments, but would NOT be apparent on Amazon, Vultr, digital Ocean or likely whatever for obvious reasons.
Personally I prefer to no longer painfully use my own server farms, cloud instances are way cheaper in the long run and have proved themselves to be better than I to maintain uptime
FreePBX shouldn’t do that at all - For example, the dashboard stats that run every minute explicitly sleep for up to 30 seconds to make sure there ISN’T a massive load at 00 seconds of every minute.
The timeconditions toggle (which has been mentioned several times to TURN OFF) can not do that, because then it won’t be accurate. And the timeconditions event could be triggering this bug (which is NOT an Asterisk or FreePBX bug, it’s an issue in the Linux Kernel), and causing audio glitches.
FWIW, I’m now deploying FreePBX in LXD containers, which allows them to Activate and purchase Commercial Modules, and I’m not having any unusual issues – but, obviously, is not supported by Sangoma, and if I ever had a problem where I needed to call them about an issue, the first thing I would do is migrate it OUT of LXD and into a VMware instance so it IS supported!
There’s plenty of ways to do VMs - providing information, logs, and stats, on how you’re doing it is the first step to getting help with it.
Then although I had that problem in the past, it apparently doesn’t happen anymore so the problem is back to those that seem to still be experiencing it.
Exposure of the “cron job” would clarify the 30 second randomness though,
Apparently from your post it IS now in 14,
. . . // Sleep to fix crazy issues with large VM hosting providers
sleep(mt_rand(1,30));
. . .
Include yourself if necessary in the “crazy issues with large VM hosting providers”
I don’t see it in my FreePBI’s which are mostly 13. Perhaps the answer should have been from the beginning, if you are using < FreePBX 14Update to FreePBX 14 (if you can )
I will leave it here, and good luck to the OP and others in the ultimate resolution .
I must assume then that the on-minute overload experienced on multiple hosted VM on the same machine being experienced by some here must then be caused by something else.’
But I still posit that finding that vector and spreading it out would likely ameliorate the problem. just as FreePBX has so demonstrated how it is done for one particular instance, and pretty well exactly ( as both I and FreePBX) have suggested previously,
@fgunno / @4allbusiness So what is going on here? You both made a huge deal last week about having this issue, how it was a bug and all you wanted was help from the community. Well now you’ve both gone silent for a week after multiple communities members asked for details to help you. I’m just curious now what has happened?
Did you go the professional support route and open tickets with Sangoma?
Did your issues automagically go away?
You haven’t replied because you’re not able to facilitate getting the information that was requested by people but users aren’t really complaining right now so it’s ok, you’ll just bring this up again when users complain again but with the added twist that you’ve been waiting for some kind of solution for this “bug”?
So status update?
(I’ll let you figure out which option my money is on)
Before the thread got crazy with some user comment I just disable the CronJob that run every minutes and I have no issue since. So I will not reactivate all that just to send you some more proof of the problem.
The Cronjob is for module I dont used, so its most like putting a tape over a button that change the color of the clock when you dont need it…
If your so bright and want to test it I proposed to sell you a dedicated server and let you install 20-25 FreePBX VM and let you made your test, you will probably find no problem… or maybe not…
On my side I will probably just switch to a Multi-tenant commercial PBX system !
And this right here is part of your problem with this. Of course I can do this and end up with no issues. Because I did it. Not knowing your specs and what you are doing when you set these systems and hosts up will not help me make a test to prove or validate your issue. It, in fact, can do the opposite. Like as @tm1000 just pointed out. Sangoma runs double the amount of instances in a host and doesn’t see a problem.
You understand that part of this process is trying to replicate your issue. In order to do that we must be able to replicate your setup. Without that information any testing is skewed and invalid.
If you’re setting up VM instances as part of your “hosted voice” solution and your users aren’t paying to have their PBX just hosted. Then using FreePBX how you are using it was the wrong way to being with.
CAN YOU EXPLAIN YOURS?! You have been asked this MULTIPLE TIMES. Stop asking us what we are doing and tell us what you are doing. You’re now getting to the “do my homework for me” stage of this.
At some point we are giving away for free all of what we’ve learned. I think I’ve given you a lot to go on here. Furthermore we just released a blog a few days ago detailing more performance enhancements in freepbx. If you think I should write up a document detailing what we do to make money that’s just rediculous.
Everything you quoted above doesn’t prove what you are accusing us of. We don’t do anything magic. But what you want me to do is fully document the setup Sangoma uses so that you can copy it and duplicate it and compete against Sangoma. Think about this logically please. There’s nothing magic in it. It’s just hardware. Software. And smart people. It’s freepbx without any modifications for performance. Galera as the backend mysql cluster and g6 and g7 blades. Monitoring is done using a mix of pager duty and zabbix. We don’t disable any crons. Commercial modules are included and enabled. But we focus strictly on voip cloud. We don’t doncloud Minecraft or cloud websites. We only focus on voip. Because voip is hard.
Please remember that not only I asked you for information but three other people did and you refused about ten different times. If anyone uses your cloud system I feel sorry for them because of your lack of follow through on their behalf. Good luck.
