FreePBX crashing every 15-30 minutes on AWS EC2 instance

FreePBX crashing every 15-30 minutes on AWS EC2 instance.

Here’s what I have in /var/log/messages (the sigint is where I rebooted it):
Sep 16 09:21:11 freepbx dnsmasq[599]: reading /etc/resolv.conf
Sep 16 09:21:11 freepbx dnsmasq[599]: using nameserver 172.31.0.2#53
Sep 16 09:21:11 freepbx avahi-daemon[600]: Withdrawing address record for 172.31.20.124 on eth0.
Sep 16 09:21:11 freepbx avahi-daemon[600]: Leaving mDNS multicast group on interface eth0.IPv4 with address 172.31.20.124.
Sep 16 09:21:11 freepbx avahi-daemon[600]: Interface eth0.IPv4 no longer relevant for mDNS.
Sep 16 09:30:18 freepbx auditd[577]: Audit daemon rotating log files
Sep 16 09:39:18 freepbx systemd-logind: Removed session 57.
Sep 16 11:24:44 freepbx systemd: Received SIGINT.

It seems like its shutting down eth0, I don’t have any web or ssh access when it does that.

Help!!

100% AWS issue. You need to contact them. This is your server having issues not FreePBX.

Thanks! I’ve contacted them, I’ll let you know what they say.

AWS wrote back saying the problem is not on their end, and they don’t troubleshoot 3rd party linux distros…

It seems they server is loving network connectivity, from the freepbx avahi-daemon[600]: Withdrawing address record for 172.31.20.124 on eth0. Because if I take a screenshot of the instance it shows its at the login prompt, I just can’t ping it or SSH in.

Give up and look for a new hosting site.
AWS is not for us.

I have two instances of distro 1707-1 running on ec2 t2.micro instances without issue.

You didn’t mention how you built this or anything; impossible to offer advice.

Thanks guys!

I installed the FreePBX ISO on VMware, and then exported that to an AMI image.

I also have another t2.micro running FreePBX without this issue, so its very strange.

AWS has an engineer looking at the problem now, who seems very knowledgeable, he said:

As the issue was with the private IP being released (at-least appears in messages), we checked the DHCP server of the instance to see if the DHCP client of the instance is sending the DHCPRequest or not. Based on the investigation and logs, I can confirm that the DHCP client of the instance ##### is requesting the private IP 172.xxx.xxx.xxx of the instance via DHCPRequest and get the renewal successfully via DHCPAck. Having said that, it appears that the IP address is not being removed, it just that avahi detected that the IP address has been removed, and is changing its internal state in response and we need to look further.

This is fixed, the problem was I changed the time zone from UTC to my local time zone, and the DHCP lease would time out before it renewed, because it thought it wasn’t time to renew due to the time zone change. Changing it back to UTC fixed the issue.

I highly recommend AWS EC2 for FreePBX as there have been no other issues.

2 Likes

I’ll second the recommending EC2, it’s worked swimmingly so far.

I also have what I’m quite sure was the same issue as you had, but did get the short straw on support techs from EC2, mostly just recommending using a more powerful instance type. But I’m pretty sure this is what’s going to fix my issue, didn’t ever occur to me that changing that time would’ve caused an issue and quite hate that it does.

Not sure if this matters too much for you but it will be inconvenient for sure to have all of our call logs and recordings having incorrect timestamps.

Did you end up just leaving it with UTC or did you find a way to get it all correct?

Following this topic if any solutions have come up… I have a production system running on EC2, I tried changing the time zone when I first set it up and ran into this issue, so I just left it on UTC. I do have a possible change coming up regarding Scheduled Pages and it would be much simpler if the system time was on local time…

Setting RTC clock to read UTC time solves the problem

timedatectl set-local-rtc 0
reboot

Enjoy FreePBX logs on AWS with your local time zone

1 Like

People using aws need to be aware of this report:
https://issues.freepbx.org/browse/FREEPBX-17274

2 Likes

Wow, the solution by @alexhfch saved me. I changed my time-zone on a new FreePBX v15 server running on AWS EC2 and every hour or so I’d lose connectivity. Setting it back to 0 did the trick.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.