I have searched and searched but i cannot find anyone that has said anything that helps.
Where and why is my disk always full!
Try logging on to server via ssh as root, to system shell / console (not asterisk console!) and tell it to
cd /
to go to root directory, then
du -h | sort -h
(may/will run for a very long time and spew warnings in the process)
but should return the sizes and names of the biggest files shown last (just above the prompt).
1 Like
login as: root
[email protected]ās password:
Last login: Tue Jul 28 09:33:36 2015 from 204.101.240.36
======================================
Welcome to The FreePBX Distro
[root@sip ~]#
Does this look like the right place to be? I am not aware of logging in anywhere else.
yes, thatās the place.
(the warning 'bout not using the asterisk console was because someone on this forum did confuse the two before).
you may also want to run
df -h
see which system devices are getting low (which partitions)
1 Like
Here are the top of that list. what can i delete?! Does anything here look too high for what it is?
100M ./var/lib/asterisk/sounds
104M ./usr/src
109M ./usr/java/jre1.7.0_05/lib
110M ./usr/java
110M ./usr/java/jre1.7.0_05
113M ./lib/modules
113M ./lib/modules/2.6.32-220.13.1.el6.i686
121M ./var/lib/mysql/asteriskcdrdb
130M ./root/installer-tmp
130M ./root/installer-tmp/tmp
131M ./root
134M ./usr/libexec/webmin
144M ./var/lib/mysql
152M ./lib
168M ./var/lib/asterisk
169M ./usr/libexec
259M ./usr/lib
290M ./var/log/asterisk
297M ./usr/share
361M ./var/lib
441M ./var/log
893M ./var
1.1G ./usr
2.4G .
Now add the output of:-
cat /proc/partitions
df -h
mount
then go to /var/log
and do the du -h |sort -h againā¦
1 Like
and, from the above output, are you using it on a 4GB card on Raspberry PI ? Or other PIAF / small flash disk solution ?
[root@sip log]# du -h | sort -n
1.6M ./pbx
1.6M ./pbx/install
4.0K ./asterisk/cdr-csv
4.0K ./asterisk/cdr-custom
4.0K ./cluster
4.0K ./ntpstats
4.0K ./pbx/upgrade
4.5M ./httpd
8.0K ./ConsoleKit
30M ./audit
294M ./asterisk
459M .
hmm, in /var/log, do
ls -lShr
then again rinse, repeat:
cd /var/log/asterisk
du -h | sort -h
ls -lShr
1 Like
total 131M
-rw------- 1 root root 0 Jul 19 03:20 spooler-20150726
-rw------- 1 root root 0 Jul 12 03:47 spooler-20150719
-rw------- 1 root root 0 Jul 5 03:10 spooler-20150712
-rw------- 1 root root 0 Jun 28 03:36 spooler-20150705
-rw------- 1 root root 0 Jul 26 03:24 spooler
-rw------- 1 root root 0 Nov 25 2014 sangoma_mgd.log
-rw------- 1 root root 0 Nov 25 2014 sangoma_bri.log
-rw------- 1 root root 49 Dec 15 2014 yum.log-20150101
-rw------- 1 root root 58 Apr 9 13:48 yum.log
---------- 1 root root 292 Jul 29 08:54 lastlog
-rw------- 1 root root 339 Jul 17 14:44 secure-20150719
-rw------- 1 root root 339 Jul 7 14:16 secure-20150712
-rw------- 1 root root 834 Jul 20 08:35 secure-20150726
-rw------- 1 root root 1006 Jul 4 10:26 secure-20150705
-rw-r--r-- 1 root root 3.6K Jun 8 09:39 boot.log
drwxr-xr-x. 4 asterisk asterisk 4.0K Apr 29 2013 pbx
drwxr-xr-x. 2 ntp ntp 4.0K Nov 28 2011 ntpstats
drwx------. 2 root root 4.0K Jul 26 03:24 httpd
drwxr-xr-x. 2 root root 4.0K Apr 29 2013 ConsoleKit
drwx------. 2 root root 4.0K Apr 30 2012 cluster
drwxr-x---. 2 root root 4.0K May 23 15:00 audit
drwxrwxr-x. 4 asterisk asterisk 4.0K Jul 29 08:48 asterisk
-rw------- 1 root root 6.1K Jul 29 08:54 secure
-rw-r----- 1 mysql mysql 11K Jun 8 09:38 mysqld.log
-rw------- 1 root root 17K Jul 29 08:36 fail2ban.log
-rw-r--r-- 1 root root 19K Jun 8 14:31 dmesg.old
-rw-r--r-- 1 root root 19K Jun 8 09:38 dmesg
-rw------- 1 root root 23K Jul 29 08:01 cron
-rw------- 1 root root 31K Jan 23 2015 fail2ban.log-20150123
-rw------- 1 root root 32K Mar 19 03:30 fail2ban.log-20150319
-rw------- 1 root root 35K Jun 9 03:16 fail2ban.log-20150609
-rw------- 1 root root 41K Dec 16 2014 fail2ban.log-20141216
-rw------- 1 root root 48K Jul 5 03:10 cron-20150705
-rw------- 1 root root 48K Jul 19 03:20 cron-20150719
-rw------- 1 root root 49K Jul 12 03:47 cron-20150712
-rw------- 1 root root 49K Jul 26 03:24 cron-20150726
-rw------- 1 root root 61K Jul 26 03:24 lfd.log-20150726.gz
-rw-rw-r-- 1 root utmp 112K Jul 29 08:54 wtmp
-rw------- 1 root root 409K Jul 29 08:54 lfd.log
-rw-r----- 1 asterisk asterisk 2.4M Jul 29 08:54 maillog
-rw-r----- 1 asterisk asterisk 4.7M Jul 12 03:45 maillog-20150712
-rw-r----- 1 asterisk asterisk 5.1M Jul 19 03:05 maillog-20150719
-rw-r----- 1 asterisk asterisk 5.7M Jul 26 03:05 maillog-20150726
-rw-r----- 1 asterisk asterisk 6.4M Jul 5 03:05 maillog-20150705
-rw------- 1 root root 35M Jul 29 08:55 messages
-rw------- 1 root root 71M Jul 26 03:24 messages-20150726
[root@sip asterisk]# du -h | sort -h
4.0K ./cdr-csv
4.0K ./cdr-custom
294M
[root@sip asterisk]# ls -lShr
total 294M
-rw-r----- 1 asterisk asterisk 40 Jul 26 03:24 queue_log-20150727
-rw-r----- 1 asterisk asterisk 40 Jul 25 03:28 queue_log-20150726
-rw-r----- 1 asterisk asterisk 40 Jul 19 03:20 queue_log-20150720
-rw-r----- 1 asterisk asterisk 40 Jul 18 03:37 queue_log-20150719
-rw-r----- 1 asterisk asterisk 40 Jul 12 03:47 queue_log-20150713
-rw-r----- 1 asterisk asterisk 40 Jul 11 03:25 queue_log-20150712
-rw-r----- 1 asterisk asterisk 40 Jul 5 03:10 queue_log-20150706
-rw-r----- 1 asterisk asterisk 40 Jul 4 03:12 queue_log-20150705
-rw-r----- 1 asterisk asterisk 40 Jul 1 03:26 queue_log-20150702
-rw-r----- 1 asterisk asterisk 93 Jul 29 08:36 fail2ban
-rw-r----- 1 asterisk asterisk 384 Jul 26 15:51 fail2ban-20150727
-rw-r----- 1 asterisk asterisk 1001 Jul 29 08:55 queue_log
-rw-r----- 1 asterisk asterisk 1.1K Jul 21 16:27 fail2ban-20150722
-rw-r----- 1 asterisk asterisk 1.4K Jul 22 21:19 fail2ban-20150723
-rw-r----- 1 asterisk asterisk 1.5K Jul 28 18:39 fail2ban-20150729
-rw-r----- 1 asterisk asterisk 1.6K Jul 24 21:51 fail2ban-20150725
-rw-r----- 1 asterisk asterisk 1.6K Jul 23 23:15 fail2ban-20150724
drwxrwxr-x. 2 asterisk asterisk 4.0K Nov 21 2012 cdr-custom
drwxrwxr-x. 2 asterisk asterisk 4.0K Apr 29 2013 cdr-csv
-rw-r----- 1 asterisk asterisk 5.7K Jul 27 22:27 fail2ban-20150728
-rw-r----- 1 asterisk asterisk 14K Jul 21 17:03 queue_log-20150722
-rw-r----- 1 asterisk asterisk 16K Jul 16 15:54 queue_log-20150717
-rw-r----- 1 asterisk asterisk 17K Jul 17 16:56 queue_log-20150718
-rw-r----- 1 asterisk asterisk 20K Jul 28 16:57 queue_log-20150729
-rw-r----- 1 asterisk asterisk 25K Jul 22 16:58 queue_log-20150723
-rw-r----- 1 asterisk asterisk 28K Jul 10 16:37 queue_log-20150711
-rw-r----- 1 asterisk asterisk 29K Jul 20 16:58 queue_log-20150721
-rw-r----- 1 asterisk asterisk 30K Jul 23 16:36 queue_log-20150724
-rw-r----- 1 asterisk asterisk 30K Jul 9 16:42 queue_log-20150710
-rw-r----- 1 asterisk asterisk 31K Jul 15 16:45 queue_log-20150716
-rw-r----- 1 asterisk asterisk 36K Jul 8 16:52 queue_log-20150709
-rw-r----- 1 asterisk asterisk 40K Jun 29 16:12 queue_log-20150630
-rw-r----- 1 asterisk asterisk 44K Jul 7 16:56 queue_log-20150708
-rw-r----- 1 asterisk asterisk 53K Jul 26 03:24 full-20150726
-rw-r----- 1 asterisk asterisk 54K Jul 3 16:56 queue_log-20150704
-rw-r----- 1 asterisk asterisk 58K Jun 30 16:50 queue_log-20150701
-rw-r----- 1 asterisk asterisk 64K Jul 24 16:51 queue_log-20150725
-rw-r----- 1 asterisk asterisk 65K Jul 27 16:37 queue_log-20150728
-rw-r----- 1 asterisk asterisk 69K Jul 6 16:56 queue_log-20150707
-rw-r----- 1 asterisk asterisk 79K Jul 14 16:22 queue_log-20150715
-rw-r----- 1 asterisk asterisk 93K Jul 13 17:01 queue_log-20150714
-rw-r----- 1 asterisk asterisk 130K Jul 2 17:13 queue_log-20150703
-rw-r----- 1 asterisk asterisk 199K Jul 27 03:44 full-20150727
-rw-r----- 1 asterisk asterisk 469K Jul 29 08:55 full
-rw-rw-r-- 1 asterisk asterisk 636K Apr 29 2013 freepbx_debug
-rw-r----- 1 asterisk asterisk 6.9M Jul 29 03:06 full-20150729
-rw-r----- 1 asterisk asterisk 8.8M Jul 24 03:27 full-20150724
-rw-r----- 1 asterisk asterisk 8.9M Jul 23 03:07 full-20150723
-rw-rw-r-- 1 asterisk asterisk 8.9M Jul 23 09:24 freepbx.log
-rw-r--r-- 1 asterisk asterisk 9.1M Jul 29 08:56 freepbx_dbug
-rw-r----- 1 asterisk asterisk 19M Jul 25 03:28 full-20150725
-rw-r----- 1 asterisk asterisk 24M Jul 28 03:34 full-20150728
-rw-rw-r-- 1 asterisk asterisk 207M Jul 29 08:55 cdr.db
OK so ā¦
seems you have a lot in /var/log/messages, and that your logrotate may be configured to keep too much back history. Maybe delve into the messages (and fail2ban) logs, also see the /var/log/asterisk/full.log and its previous copies - see what fills them ? Also your maillog seems excessively large - basically, try looking into any of these large log files and try to understand what fills them (I AM NOT asking to post content(s) here, try understanding them first yourselfā¦)
(useful commands : less [filename], cat filename | more, mc ā¦ 
)
So I think the general concensus is that his disk is so close to full because he doesnāt have much disk space.
Seems like a reasonable result to me 
Note, @Laveniuk, it would be REALLY HANDY if you could actually show us how much disk space you have. Run the command df -h and paste the output.
My guess is that the OP actually has a big disk but it is inappropriately partitioned/mounted (note the /cluster directory? ), hence my request for information, Pretty well everything fits nicely in 5G if a big partition is mounted on /var
cat /proc/partitions
df -h
mount
do that and it is fixable.
1 Like
@dicko - I have one (other, not FreePBX) server partitioned by Debian Installer (duh) and yeah that installer gives not a lot to /var out of the 75GB disk(s) used:
[server] ~$ df -h
Filesystem Size Used Avail Use% Mounted on
[...]
rootfs 314M 193M 101M 66% /
/dev/md5 61G 16G 43G 27% /home
/dev/md4 360M 2.1M 335M 1% /tmp
/dev/md1 8.2G 620M 7.1G 8% /usr
/dev/md2 2.7G 2.3G 309M 89% /var
(thatās btw guided partitioning of Debian Wheezy for youā¦ )
But I suspect more that OP is running on a small 4GB flash somewhere.
(or, re-looking at his second post, on a VM in the cloud somewhere and heās given it too little of disk).
A āguided partitioningā of Debian will indeed offer that schema. but in FreePBX, the only content of /home would be /home/asterisk with presumably a content only of .gpg, this is, also presumably, to have the asterisk user have access to GPG when and if your āHAā schema fails.
So in your case where you have the majority of the disk appropriated by home , and I guess you have 43G worth of NON asterisk stuff there
.
I would suggest you are yourself dangerously close to running out of space on /var my guess /var/log but you know how to figure that out with your bash-fu and fix it before your system breaks 
@dicko this is
: - )
My proper freepbx server uses ādefault FreePBX distro partitioningā /boot and / scheme 
and has about 29% usage on /
On the machine with 309M (89%) left you ARE close to critical. Journaled filesystems get very needy as you get past 80% and growing, check your logrotate scripts.
29% of what? it is all relative, /var will in any Asterisk/FreePBX/(Hylafax)/apache implementation amount to a vast majority of your disk needs, just look at itās content, mysql, recordings, voicemail, faxes, etc etc etc. THAT / partition is going to b happy with 5-10 G the rest needs to be for var or just use a BIG_ENOUGH disk for / and donāt worry (until it is full then the whole thing dies not just asterisk) , this guy just didnāt RTFM . . .
Just look at his third post, do you have a /cluster directory, if so what does it contain ?
hint:-
du -h --max-depth=1 /
compare with
du -hx --max-depth=1 /
if you want to separate disk from partition usage
That machine with 21% space on /var left, is an (to be retired for hw reasons) auxilary backup file server running Debian Wheezy.
FreePBX machine is a Completely Different Machine and not even in the same room running Freepbx distro.
The FreePBX machine has
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md2 72G 20G 49G 29% /
tmpfs 499M 0 499M 0% /dev/shm
/dev/md0 291M 32M 245M 12% /boot
(this is default as FreePBX distro partitioned it while installing back in Novemberā¦ )
[root@sip ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 59G 48G 7.9G 86% /
tmpfs 1.5G 0 1.5G 0% /dev/shm
/dev/sda1 97M 26M 67M 28% /boot
here is my disk space. 7.9 GB left out of a 60GB