Queue_Log file sometimes empty

Edmond · March 13, 2021, 8:52pm

hi there, i am having issue with queue_log file sometimes is empty, i need to restart asterisk in order to work, can you help me where do i need to look at, its like 1 month that the issue started. I opened a ticket in asterisk forum but they closed it. I am using Asterisk version 16.15.1
and freepbx 15.0.17.24. Thanks

PitzKey · March 13, 2021, 11:03pm

Does it work right now? If not, please post the output of

ll /var/log/asterisk | grep queue

Edmond · March 14, 2021, 12:45am

yes it works now because i restart asterisk. I just figure it out that it stopped working right after the Asterisk Queue Logger restarted which happens at 3:35 utc time. After this time i have no data at all in the Queue_log. But now it works if it stopes working i will post the output of grep queue. Thanks

dicko · March 14, 2021, 1:06am

Please post the output of

crontab -l -u asterisk
crontab -l
ls -l /etc/logrotate.d/*
cat /etc/logrotate.d/asterisk

Edmond · March 14, 2021, 1:18am

crontab -l -u asterisk
*/15 * * * * [ -e /etc/asterisk/firewall.enabled ] && touch /var/spool/asterisk/ incron/firewall.firewall
*/5 * * * * /usr/sbin/fwconsole firewall sync > /dev/null 2>&1
3 1 * * * /usr/sbin/fwconsole certificates --updateall -q 2>&1 >/dev/null

5 * * * /usr/sbin/fwconsole util cleanplaybackcache -q
47 16 * * 1 [ -e /usr/sbin/fwconsole ] && /usr/sbin/fwconsole ma listonline --se ndemail -q > /dev/null 2>&1
- - - - [ -e /usr/sbin/fwconsole ] && sleep $((RANDOM%30)) && /usr/sbin/fwcon sole job --run --quiet 2>&1 > /dev/null

ls -l /etc/logrotate.d/*
-rw-r–r-- 1 root root 86 Jun 8 2020 /etc/logrotate.d/apcupsd
-rw-r–r-- 1 root root 700 Dec 23 15:41 /etc/logrotate.d/asterisk
-rw-r–r-- 1 root root 91 Nov 27 2019 /etc/logrotate.d/bootlog
-rw-r–r-- 1 root root 160 Sep 19 2018 /etc/logrotate.d/chrony
-rw-r–r--. 1 root root 252 Aug 5 2018 /etc/logrotate.d/fail2ban
-rw-r–r-- 1 root root 581 Mar 4 14:33 /etc/logrotate.d/freepbx-core
-rw-r–r-- 1 root root 194 Nov 27 2019 /etc/logrotate.d/httpd
-rw-r–r-- 1 root root 172 Sep 29 2016 /etc/logrotate.d/iscsiuiolog
-rw-r–r-- 1 root root 893 Apr 2 2020 /etc/logrotate.d/mariadb
-rw-r–r--. 1 root root 477 Jan 29 2018 /etc/logrotate.d/mongodb
-rw-r–r-- 1 root root 181 Jun 4 2020 /etc/logrotate.d/pms
-rw-r–r--. 1 root root 408 Aug 3 2017 /etc/logrotate.d/psacct
-rw-r–r--. 1 root root 1103 May 13 2019 /etc/logrotate.d/radiusd
-rw-r–r--. 1 root root 127 Oct 26 2018 /etc/logrotate.d/redis
-rw-r–r-- 1 root root 224 Nov 27 2019 /etc/logrotate.d/syslog
-rw-r–r-- 1 root root 188 Apr 1 2020 /etc/logrotate.d/vsftpd
-rw-r–r--. 1 root root 100 Oct 30 2018 /etc/logrotate.d/wpa_supplicant
-rw-r–r-- 1 root root 103 Apr 2 2020 /etc/logrotate.d/yum

cat /etc/logrotate.d/asterisk
/var/log/asterisk/queue_log {
daily
missingok
rotate 30
notifempty
sharedscripts
create 0640 asterisk asterisk
su asterisk asterisk
}

/var/spool/mail/asterisk {
daily
missingok
rotate 7
notifempty
sharedscripts
create 0660 asterisk mail
su asterisk mail
}

/var/log/asterisk/messages
/var/log/asterisk/event_log
/var/log/asterisk/full
/var/log/asterisk/dtmf
/var/log/asterisk/fail2ban {
daily
missingok
rotate 7
notifempty
sharedscripts
create 0640 asterisk asterisk
su asterisk asterisk
postrotate
/usr/sbin/asterisk -rx ‘logger reload’ > /dev/null 2> /dev/null
endscript
}
#This comment is to fix rpm file replacing
#Config file built on Wed Dec 23 15:41:49 UTC 2020

dicko · March 14, 2021, 1:29am

/var/log/asterisk/queue_log {
daily
missingok
rotate 30
notifempty
sharedscripts
create 0640 asterisk asterisk
su asterisk asterisk
}

looks good, but might need a ‘queue reload all’ , try removing sharedscripts and adding

  postrotate
    rasterisk -x "queue reload all" > /dev/null
  endscript

cat /etc/crontab ?

Edmond · March 14, 2021, 4:41am

now its not working, here is output:
-rw-r----- 1 asterisk asterisk 0 Mar 14 03:34 queue_log
-rw-rw-r-- 1 asterisk asterisk 770747 Feb 13 03:35 queue_log-20210213
-rw-rw-r-- 1 asterisk asterisk 856761 Feb 14 03:13 queue_log-20210214
-rw-rw-r-- 1 asterisk asterisk 889412 Feb 15 03:12 queue_log-20210215
-rw-rw-r-- 1 asterisk asterisk 706405 Feb 16 03:31 queue_log-20210216
-rw-rw-r-- 1 asterisk asterisk 524175 Feb 17 03:05 queue_log-20210217
-rw-rw-r-- 1 asterisk asterisk 591017 Feb 18 03:38 queue_log-20210218
-rw-rw-r-- 1 asterisk asterisk 550505 Feb 19 03:37 queue_log-20210219
-rw-rw-r-- 1 asterisk asterisk 758433 Feb 20 03:32 queue_log-20210220
-rw-rw-r-- 1 asterisk asterisk 804101 Feb 21 03:31 queue_log-20210221
-rw-rw-r-- 1 asterisk asterisk 789301 Feb 22 03:46 queue_log-20210222
-rw-rw-r-- 1 asterisk asterisk 585126 Feb 23 03:11 queue_log-20210223
-rw-rw-r-- 1 asterisk asterisk 531989 Feb 24 03:06 queue_log-20210224
-rw-rw-r-- 1 asterisk asterisk 595326 Feb 25 03:06 queue_log-20210225
-rw-rw-r-- 1 asterisk asterisk 625706 Feb 26 03:10 queue_log-20210226
-rw-rw-r-- 1 asterisk asterisk 832252 Feb 27 03:11 queue_log-20210227
-rw-rw-r-- 1 asterisk asterisk 940267 Feb 28 03:09 queue_log-20210228
-rw-rw-r-- 1 asterisk asterisk 841520 Mar 1 03:26 queue_log-20210301
-rw-rw-r-- 1 asterisk asterisk 553501 Mar 2 03:43 queue_log-20210302
-rw-rw-r-- 1 asterisk asterisk 739148 Mar 3 09:08 queue_log-20210303
-rw-rw-r-- 1 asterisk asterisk 719481 Mar 4 10:17 queue_log-20210304
-rw-rw-r-- 1 asterisk asterisk 374307 Mar 5 03:09 queue_log-20210305
-rw-r----- 1 asterisk asterisk 834165 Mar 6 03:47 queue_log-20210306
-rw-r----- 1 asterisk asterisk 821008 Mar 7 03:19 queue_log-20210307
-rw-r----- 1 asterisk asterisk 771685 Mar 8 03:48 queue_log-20210308
-rw-r----- 1 asterisk asterisk 524165 Mar 9 03:19 queue_log-20210309
-rw-r----- 1 asterisk asterisk 562693 Mar 10 03:17 queue_log-20210310
-rw-r----- 1 asterisk asterisk 618460 Mar 11 03:36 queue_log-20210311
-rw-r----- 1 asterisk asterisk 724928 Mar 12 03:08 queue_log-20210312
-rw-r----- 1 asterisk asterisk 1234616 Mar 13 08:00 queue_log-20210313
-rw-r----- 1 asterisk asterisk 812234 Mar 14 04:40 queue_log-20210314
-rw-rw-r-- 1 asterisk asterisk 0 Mar 13 07:40 queue_log_failed

Edmond · March 14, 2021, 4:47am

i did changed it from /usr/sbin/asterisk -rx ‘logger reload’ > /dev/null 2> /dev/null to rasterisk -x “queue reload all” > /dev/null , but do i need to restart asterisk? i did module reload logger and it started working again.

dicko · March 14, 2021, 4:56am

No, logrotate runs on the schedule defined in /etc/crontab. But leave the logger reload as it is, that stanza doesn’t cover the queue, does queue reload all also kick it back into gear?

Edmond · March 14, 2021, 5:06am

Before doing module reload logger i tried rasterisk -x “queue reload all” > /dev/null
but it did not work, here is what i have now:
[root@freepbx ~]# cat /etc/logrotate.d/asterisk
/var/log/asterisk/queue_log {
daily
missingok
rotate 30
notifempty
sharedscripts
create 0640 asterisk asterisk
su asterisk asterisk
}

/var/spool/mail/asterisk {
daily
missingok
rotate 7
notifempty
sharedscripts
create 0660 asterisk mail
su asterisk mail
}

/var/log/asterisk/messages
/var/log/asterisk/event_log
/var/log/asterisk/full
/var/log/asterisk/dtmf
/var/log/asterisk/fail2ban {
daily
missingok
rotate 7
notifempty
sharedscripts
create 0640 asterisk asterisk
su asterisk asterisk
postrotate
rasterisk -x “queue reload all” > /dev/null
endscript
}
#This comment is to fix rpm file replacing
#Config file built on Wed Dec 23 15:41:49 UTC 2020

[root@freepbx ~]# cat /etc/crontab
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root

For details see man 4 crontabs

Example of job definition:

.---------------- minute (0 - 59)

| .------------- hour (0 - 23)

| | .---------- day of month (1 - 31)

| | | .------- month (1 - 12) OR jan,feb,mar,apr …

| | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat

| | | | |

* * * * * user-name command to be executed

dicko · March 14, 2021, 5:11am

If you need

postrotate
rasterisk -x “queue reload all” > /dev/null
endscript

It belongs in the stanza for logrotate, don’t stop reloading logger it might well cause other problems

Edmond · March 14, 2021, 5:19am

Should i change it like it was before /usr/sbin/asterisk -rx ‘logger reload’ > /dev/null 2> /dev/null , because i dont know what i am doing

dicko · March 14, 2021, 5:21am

pastebin -i wazzup.doc after

logrotate -f -v /etc/logrotate.d/asterisk|tee wazzup.doc

Edmond · March 14, 2021, 11:49am

It loges to the old file but doing module reload logger fixes that.

Here is your command output:
[root@freepbx ~]# logrotate -f -v /etc/logrotate.d/asterisk|tee wazzup.doc
reading config file /etc/logrotate.d/asterisk
Allocating hash table for state file, size 15360 B

Handling 3 logs

rotating pattern: /var/log/asterisk/queue_log forced from command line (30 rotations)
empty log files are not rotated, old logs are removed
switching euid to 995 and egid to 995
considering log /var/log/asterisk/queue_log
log needs rotating
rotating log /var/log/asterisk/queue_log, log->rotateCount is 30
dateext suffix ‘-20210314’
glob pattern ‘-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]’
renaming /var/log/asterisk/queue_log.30 to /var/log/asterisk/queue_log.31 (rotatecount 30, logstart 1, i 30),
old log /var/log/asterisk/queue_log.30 does not exist
renaming /var/log/asterisk/queue_log.29 to /var/log/asterisk/queue_log.30 (rotatecount 30, logstart 1, i 29),
old log /var/log/asterisk/queue_log.29 does not exist
renaming /var/log/asterisk/queue_log.28 to /var/log/asterisk/queue_log.29 (rotatecount 30, logstart 1, i 28),
old log /var/log/asterisk/queue_log.28 does not exist
renaming /var/log/asterisk/queue_log.27 to /var/log/asterisk/queue_log.28 (rotatecount 30, logstart 1, i 27),
old log /var/log/asterisk/queue_log.27 does not exist
renaming /var/log/asterisk/queue_log.26 to /var/log/asterisk/queue_log.27 (rotatecount 30, logstart 1, i 26),
old log /var/log/asterisk/queue_log.26 does not exist
renaming /var/log/asterisk/queue_log.25 to /var/log/asterisk/queue_log.26 (rotatecount 30, logstart 1, i 25),
old log /var/log/asterisk/queue_log.25 does not exist
renaming /var/log/asterisk/queue_log.24 to /var/log/asterisk/queue_log.25 (rotatecount 30, logstart 1, i 24),
old log /var/log/asterisk/queue_log.24 does not exist
renaming /var/log/asterisk/queue_log.23 to /var/log/asterisk/queue_log.24 (rotatecount 30, logstart 1, i 23),
old log /var/log/asterisk/queue_log.23 does not exist
renaming /var/log/asterisk/queue_log.22 to /var/log/asterisk/queue_log.23 (rotatecount 30, logstart 1, i 22),
old log /var/log/asterisk/queue_log.22 does not exist
renaming /var/log/asterisk/queue_log.21 to /var/log/asterisk/queue_log.22 (rotatecount 30, logstart 1, i 21),
old log /var/log/asterisk/queue_log.21 does not exist
renaming /var/log/asterisk/queue_log.20 to /var/log/asterisk/queue_log.21 (rotatecount 30, logstart 1, i 20),
old log /var/log/asterisk/queue_log.20 does not exist
renaming /var/log/asterisk/queue_log.19 to /var/log/asterisk/queue_log.20 (rotatecount 30, logstart 1, i 19),
old log /var/log/asterisk/queue_log.19 does not exist
renaming /var/log/asterisk/queue_log.18 to /var/log/asterisk/queue_log.19 (rotatecount 30, logstart 1, i 18),
old log /var/log/asterisk/queue_log.18 does not exist
renaming /var/log/asterisk/queue_log.17 to /var/log/asterisk/queue_log.18 (rotatecount 30, logstart 1, i 17),
old log /var/log/asterisk/queue_log.17 does not exist
renaming /var/log/asterisk/queue_log.16 to /var/log/asterisk/queue_log.17 (rotatecount 30, logstart 1, i 16),
old log /var/log/asterisk/queue_log.16 does not exist
renaming /var/log/asterisk/queue_log.15 to /var/log/asterisk/queue_log.16 (rotatecount 30, logstart 1, i 15),
old log /var/log/asterisk/queue_log.15 does not exist
renaming /var/log/asterisk/queue_log.14 to /var/log/asterisk/queue_log.15 (rotatecount 30, logstart 1, i 14),
old log /var/log/asterisk/queue_log.14 does not exist
renaming /var/log/asterisk/queue_log.13 to /var/log/asterisk/queue_log.14 (rotatecount 30, logstart 1, i 13),
old log /var/log/asterisk/queue_log.13 does not exist
renaming /var/log/asterisk/queue_log.12 to /var/log/asterisk/queue_log.13 (rotatecount 30, logstart 1, i 12),
old log /var/log/asterisk/queue_log.12 does not exist
renaming /var/log/asterisk/queue_log.11 to /var/log/asterisk/queue_log.12 (rotatecount 30, logstart 1, i 11),
old log /var/log/asterisk/queue_log.11 does not exist
renaming /var/log/asterisk/queue_log.10 to /var/log/asterisk/queue_log.11 (rotatecount 30, logstart 1, i 10),
old log /var/log/asterisk/queue_log.10 does not exist
renaming /var/log/asterisk/queue_log.9 to /var/log/asterisk/queue_log.10 (rotatecount 30, logstart 1, i 9),
old log /var/log/asterisk/queue_log.9 does not exist
renaming /var/log/asterisk/queue_log.8 to /var/log/asterisk/queue_log.9 (rotatecount 30, logstart 1, i 8),
old log /var/log/asterisk/queue_log.8 does not exist
renaming /var/log/asterisk/queue_log.7 to /var/log/asterisk/queue_log.8 (rotatecount 30, logstart 1, i 7),
old log /var/log/asterisk/queue_log.7 does not exist
renaming /var/log/asterisk/queue_log.6 to /var/log/asterisk/queue_log.7 (rotatecount 30, logstart 1, i 6),
old log /var/log/asterisk/queue_log.6 does not exist
renaming /var/log/asterisk/queue_log.5 to /var/log/asterisk/queue_log.6 (rotatecount 30, logstart 1, i 5),
old log /var/log/asterisk/queue_log.5 does not exist
renaming /var/log/asterisk/queue_log.4 to /var/log/asterisk/queue_log.5 (rotatecount 30, logstart 1, i 4),
old log /var/log/asterisk/queue_log.4 does not exist
renaming /var/log/asterisk/queue_log.3 to /var/log/asterisk/queue_log.4 (rotatecount 30, logstart 1, i 3),
old log /var/log/asterisk/queue_log.3 does not exist
renaming /var/log/asterisk/queue_log.2 to /var/log/asterisk/queue_log.3 (rotatecount 30, logstart 1, i 2),
old log /var/log/asterisk/queue_log.2 does not exist
renaming /var/log/asterisk/queue_log.1 to /var/log/asterisk/queue_log.2 (rotatecount 30, logstart 1, i 1),
old log /var/log/asterisk/queue_log.1 does not exist
renaming /var/log/asterisk/queue_log.0 to /var/log/asterisk/queue_log.1 (rotatecount 30, logstart 1, i 0),
old log /var/log/asterisk/queue_log.0 does not exist
log /var/log/asterisk/queue_log.31 doesn’t exist – won’t try to dispose of it
renaming /var/log/asterisk/queue_log to /var/log/asterisk/queue_log.1
creating new /var/log/asterisk/queue_log mode = 0640 uid = 995 gid = 995
switching euid to 0 and egid to 0

rotating pattern: /var/spool/mail/asterisk forced from command line (7 rotations)
empty log files are not rotated, old logs are removed
switching euid to 995 and egid to 12
considering log /var/spool/mail/asterisk
log does not need rotating (log is empty)switching euid to 0 and egid to 0

rotating pattern: /var/log/asterisk/messages
/var/log/asterisk/event_log
/var/log/asterisk/full
/var/log/asterisk/dtmf
/var/log/asterisk/fail2ban forced from command line (7 rotations)
empty log files are not rotated, old logs are removed
switching euid to 995 and egid to 995
considering log /var/log/asterisk/messages
log /var/log/asterisk/messages does not exist – skipping
considering log /var/log/asterisk/event_log
log /var/log/asterisk/event_log does not exist – skipping
considering log /var/log/asterisk/full
log needs rotating
considering log /var/log/asterisk/dtmf
log /var/log/asterisk/dtmf does not exist – skipping
considering log /var/log/asterisk/fail2ban
log needs rotating
rotating log /var/log/asterisk/full, log->rotateCount is 7
dateext suffix ‘-20210314’
glob pattern ‘-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]’
renaming /var/log/asterisk/full.7 to /var/log/asterisk/full.8 (rotatecount 7, logstart 1, i 7),
old log /var/log/asterisk/full.7 does not exist
renaming /var/log/asterisk/full.6 to /var/log/asterisk/full.7 (rotatecount 7, logstart 1, i 6),
old log /var/log/asterisk/full.6 does not exist
renaming /var/log/asterisk/full.5 to /var/log/asterisk/full.6 (rotatecount 7, logstart 1, i 5),
old log /var/log/asterisk/full.5 does not exist
renaming /var/log/asterisk/full.4 to /var/log/asterisk/full.5 (rotatecount 7, logstart 1, i 4),
old log /var/log/asterisk/full.4 does not exist
renaming /var/log/asterisk/full.3 to /var/log/asterisk/full.4 (rotatecount 7, logstart 1, i 3),
old log /var/log/asterisk/full.3 does not exist
renaming /var/log/asterisk/full.2 to /var/log/asterisk/full.3 (rotatecount 7, logstart 1, i 2),
old log /var/log/asterisk/full.2 does not exist
renaming /var/log/asterisk/full.1 to /var/log/asterisk/full.2 (rotatecount 7, logstart 1, i 1),
old log /var/log/asterisk/full.1 does not exist
renaming /var/log/asterisk/full.0 to /var/log/asterisk/full.1 (rotatecount 7, logstart 1, i 0),
log /var/log/asterisk/full.8 doesn’t exist – won’t try to dispose of it
rotating log /var/log/asterisk/fail2ban, log->rotateCount is 7
dateext suffix ‘-20210314’
glob pattern ‘-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]’
renaming /var/log/asterisk/fail2ban.7 to /var/log/asterisk/fail2ban.8 (rotatecount 7, logstart 1, i 7),
old log /var/log/asterisk/fail2ban.7 does not exist
renaming /var/log/asterisk/fail2ban.6 to /var/log/asterisk/fail2ban.7 (rotatecount 7, logstart 1, i 6),
old log /var/log/asterisk/fail2ban.6 does not exist
renaming /var/log/asterisk/fail2ban.5 to /var/log/asterisk/fail2ban.6 (rotatecount 7, logstart 1, i 5),
old log /var/log/asterisk/fail2ban.5 does not exist
renaming /var/log/asterisk/fail2ban.4 to /var/log/asterisk/fail2ban.5 (rotatecount 7, logstart 1, i 4),
old log /var/log/asterisk/fail2ban.4 does not exist
renaming /var/log/asterisk/fail2ban.3 to /var/log/asterisk/fail2ban.4 (rotatecount 7, logstart 1, i 3),
old log /var/log/asterisk/fail2ban.3 does not exist
renaming /var/log/asterisk/fail2ban.2 to /var/log/asterisk/fail2ban.3 (rotatecount 7, logstart 1, i 2),
old log /var/log/asterisk/fail2ban.2 does not exist
renaming /var/log/asterisk/fail2ban.1 to /var/log/asterisk/fail2ban.2 (rotatecount 7, logstart 1, i 1),
old log /var/log/asterisk/fail2ban.1 does not exist
renaming /var/log/asterisk/fail2ban.0 to /var/log/asterisk/fail2ban.1 (rotatecount 7, logstart 1, i 0),
old log /var/log/asterisk/fail2ban.0 does not exist
log /var/log/asterisk/fail2ban.8 doesn’t exist – won’t try to dispose of it
renaming /var/log/asterisk/full to /var/log/asterisk/full.1
creating new /var/log/asterisk/full mode = 0640 uid = 995 gid = 995
renaming /var/log/asterisk/fail2ban to /var/log/asterisk/fail2ban.1
creating new /var/log/asterisk/fail2ban mode = 0640 uid = 995 gid = 995
running postrotate script
switching euid to 0 and egid to 0
switching euid to 0 and egid to 0

dicko · March 14, 2021, 1:36pm

Then add that to the postrotate script.

Edmond · March 14, 2021, 1:40pm

how and where exactly

dicko · March 14, 2021, 1:52pm

Between postrotate and endscript in the problematic stanza

Edmond · March 14, 2021, 4:35pm

is this correct:

cat /etc/logrotate.d/asterisk
/var/log/asterisk/queue_log {
daily
missingok
rotate 30
notifempty
sharedscripts
create 0640 asterisk asterisk
su asterisk asterisk
postrotate
rasterisk -x “module reload logger” > /dev/null
endscript
}

dicko · March 14, 2021, 5:25pm

Does it work?

Edmond · March 14, 2021, 5:53pm

we need to wait for the time when asterisk restart logger which happens everyday at 3:35 utc time