Mysql Server has Gone Away

[dicko]

I don’t use a partition I use a swapfile

swapon -s
Filename Type Size Used Priority
/.swapfile file 511996 196548 -1

free
total used free shared buffers cached
Mem: 1026900 896364 130536 0 42656 513844
-/+ buffers/cache: 339864 687036
Swap: 511996 196548 315448

uptime
18:06:53 up 317 days, 2:44, 1 user, load average: 0.21, 0.22, 0.18

rasterisk -x “sip show peers”|grep OK|wc -l
136

[hardocp]

so i just checked and my swap was only 256mb – i wonder if that is why i had an issue

I increased it to 3gb (since i have plenty of empty space on the ssd) – so now i have 2gb of memory and 3gb of swap

lets see if that helps or if there is still some other issue at fault here

[dicko]

grep -i oom[-_]kill /var/log/messages*

[hardocp]

/var/log/messages:Apr 6 01:10:39 715pbx kernel: mysqld invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0
/var/log/messages:Apr 6 01:10:39 715pbx kernel: oom_kill_process+0x212/0x430
/var/log/messages-20190331:Mar 27 13:39:32 715pbx kernel: asterisk invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0
/var/log/messages-20190331:Mar 27 13:39:32 715pbx kernel: oom_kill_process+0x212/0x430

[dicko]

2g of memory is more than enough for even quite a large asterisk PBX, but as you guessed. 512mb is too small for a swap file.Do you have a scheduled process running at 01:10 am like mysqldump perhaps?

[hardocp]

is it this one?

*/1 * * * * /usr/sbin/fwconsole calendar --sync 2>&1 > /dev/null

[dicko]

Possibly, while running top in one console, see if

time /usr/sbin/fwconsole calendar --sync

takes longer than 10 minutes

[hardocp]

pretty much finished instantaneously

time /usr/sbin/fwconsole calendar --sync
Starting Sync…
Finished

real 0m0.331s
user 0m0.237s
sys 0m0.047s

[dicko]

That was with 3g of swap or 512m?

[hardocp]

3gb of swap

[dicko]

From sysstat

sar -S for today

sar -S  -f /var/log/sysstat/sa03

For 03 of this month

[hardocp]

12:00:26 AM kbswpfree kbswpused %swpused kbswpcad %swpcad
12:10:26 AM 157692 104448 39.84 11868 11.36
12:20:26 AM 155644 106496 40.63 11520 10.82
12:30:27 AM 136700 125440 47.85 14700 11.72
12:40:26 AM 90108 172032 65.63 14420 8.38
12:50:26 AM 28924 233216 88.97 16172 6.93
01:00:26 AM 24 262116 99.99 14724 5.62
01:10:26 AM 20 262120 99.99 14780 5.64
01:20:26 AM 2836 259304 98.92 15576 6.01
Average: 71494 190646 72.73 14220 7.46

08:41:42 PM LINUX RESTART

08:50:01 PM kbswpfree kbswpused %swpused kbswpcad %swpcad
09:00:01 PM 3276796 0 0.00 0 0.00
09:10:02 PM 3276796 0 0.00 0 0.00
09:20:01 PM 3276796 0 0.00 0 0.00
09:30:01 PM 3276796 0 0.00 0 0.00
09:40:02 PM 3276796 0 0.00 0 0.00
09:50:01 PM 3276796 0 0.00 0 0.00
10:00:01 PM 3276796 0 0.00 0 0.00
Average: 3276796 0 0.00 0 0.00

[dicko]

Hopefully Q.E.D.

[hardocp]

Agreed – Will keep you posted

Thank you!