HDD is running to the max

Topics (not sure which fora)
when not sure where to post, post here and mods will move it to right forum.

Moderators: leecollings, remb0

Post Reply
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

HDD is running to the max

Post by Derik »

Please is there someone that can help me to clean my HDD from my Cubieboard 2..
ScreenShot056.jpg
ScreenShot056.jpg (41.54 KiB) Viewed 3015 times
I see my drive is running to the max..

I try the clean option, that is give some space, only not for long.
My image is:
http://www.armbian.com/cubieboard-2/
ScreenShot059.jpg
ScreenShot059.jpg (33.43 KiB) Viewed 3009 times
ScreenShot058.jpg
ScreenShot058.jpg (34.25 KiB) Viewed 3015 times
Are there some maps that are using a lot of space that i can remove without having problem,,
Or a better solution..
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
Egregius
Posts: 2592
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: HDD is running to the max

Post by Egregius »

ssh to it
cd /
du -h -d 1
"du -h -d 1" will show you the actual size of the folders.
Go in to the biggest folders and rerun the command until you find out what's eating your drive.
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: HDD is running to the max

Post by Derik »

mmm you think i am smart .....
Only this screendumps are almost my max ,, :-)

Code: Select all

linaro@cubie:/$ sudo du -h -d 1
5.1M    ./boot
4.0G    ./var
4.0K    ./media
13M     ./tmp
16K     ./lost+found
4.0K    ./opt
42M     ./home
192K    ./run
4.0M    ./etc
0       ./dev
5.1M    ./bin
4.0K    ./mnt
5.5M    ./sbin
du: cannot access './proc/12638/task/12638/fd/4': No such file or directory
du: cannot access './proc/12638/task/12638/fdinfo/4': No such file or directory
du: cannot access './proc/12638/fd/4': No such file or directory
du: cannot access './proc/12638/fdinfo/4': No such file or directory
0       ./proc
4.0K    ./srv
679M    ./usr
32K     ./root
396M    ./lib
0       ./sys
5.2G    .
linaro@cubie:/$
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
marcelr
Posts: 42
Joined: Friday 22 May 2015 21:10
Target OS: Linux
Domoticz version: svn 2470
Location: Ehv, NL
Contact:

Re: HDD is running to the max

Post by marcelr »

4.0G ./var

Why is /var so large? Maybe it's time to clean up some garbage there.

grtz,

marcelr
SweetPants

Re: HDD is running to the max

Post by SweetPants »

/var contains all logging. Check there what is using your disk

Verstuurd vanaf mijn HTC Desire 610 met Tapatalk
User avatar
Egregius
Posts: 2592
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: HDD is running to the max

Post by Egregius »

well,
cd /var
sudo du -h -d 1

/var contains so much more than logging.
Recently I had same issue on one of my RPI's wich had a 2GB card. Appeared that all old kernels where still on the card...
User avatar
gizmocuz
Posts: 2546
Joined: Thursday 11 July 2013 18:59
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Location: Top of the world
Contact:

Re: HDD is running to the max

Post by gizmocuz »

also the tool 'ncdu' is helpfull, it is a graphical 'du' application
Quality outlives Quantity!
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: HDD is running to the max

Post by Derik »

Can i just simple delete the whole map ./var
ScreenShot061.jpg
ScreenShot061.jpg (69.29 KiB) Viewed 2889 times
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
jvdz
Posts: 2328
Joined: Tuesday 30 December 2014 19:25
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.107
Location: Netherlands
Contact:

Re: HDD is running to the max

Post by jvdz »

Derik wrote:Can i just simple delete the whole map ./var
Not when you still want to use the system afterwards. ;)
So have you checked the subdirectories for large files. I would first check /log, /backups and /tmp

Jos
New Garbage collection scripts: https://github.com/jvanderzande/GarbageCalendar
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: HDD is running to the max

Post by Derik »

Just the first big ones?
ScreenShot062.jpg
ScreenShot062.jpg (126.31 KiB) Viewed 2880 times
And here to?
ScreenShot063.jpg
ScreenShot063.jpg (150.78 KiB) Viewed 2880 times
Can i remove the older versions???
The only backup i wil give a lot of money is my Domoticz DB...
When my OS stock, i set a new os on my sd...
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
jvdz
Posts: 2328
Joined: Tuesday 30 December 2014 19:25
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.107
Location: Netherlands
Contact:

Re: HDD is running to the max

Post by jvdz »

Seems you aren't running the logrotate utility that normally would clean and compress logfiles to ensure they don't grow to that size.
EDIT: Looks like it was working till 25 January last year as there are compressed versions of auth.log

Jos
New Garbage collection scripts: https://github.com/jvanderzande/GarbageCalendar
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: HDD is running to the max

Post by Derik »

jvdz wrote:Seems you aren't running the logrotate utility that normally would clean and compress logfiles to ensure they don't grow to that size.

Jos
Ok
I did change nothing. [ i think ]
Just flash the sd and run domoticz on it...[ runs 2-3 years ]

Can i remove this log files..? [ in all the years i did not use this stuff ]
And the DPKG.Status old versions?
And the alternative and apt.extend. xx old versions?
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
jvdz
Posts: 2328
Joined: Tuesday 30 December 2014 19:25
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.107
Location: Netherlands
Contact:

Re: HDD is running to the max

Post by jvdz »

Why worry about the few Kb's of the DPKG? The issue seems to be the log files not being cleared.
You can find quite some hits on Google on this question around deleting auth.log, but think you really should fix the logrotate so the system won't fill up again over time.

Jos
New Garbage collection scripts: https://github.com/jvanderzande/GarbageCalendar
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: HDD is running to the max

Post by Derik »

Hero's....
ScreenShot064.jpg
ScreenShot064.jpg (17.95 KiB) Viewed 2861 times
And proud @ myself...
Last edited by Derik on Monday 27 June 2016 21:07, edited 1 time in total.
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
Egregius
Posts: 2592
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: HDD is running to the max

Post by Egregius »

I would run these commands:

sudo truncate -s 0 *.log -> empty all logfiles without deleting them and thus preserving ownership and permissions.
sudo rm *.gz -> remove all old compressed logfiles. This can also be done in /var/backups

Don't know what those btmp files are occupying 200MB, should google that to see if it's safe to remove/empty them.

Also, try to find out why logrotate doesn't run anymore, then that handles it automatically for you.
In the config files of logrotate you can specify how many .gz files are kept.
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: HDD is running to the max

Post by Derik »

Code: Select all

linaro@cubie:~$ cd /var/log
linaro@cubie:/var/log$ sudo truncate -s0 auth.log
linaro@cubie:/var/log$ sudo truncate -s0 syslog
linaro@cubie:/var/log$ sudo truncate -s0 daemon.log
linaro@cubie:/var/log$ sudo truncate -s0 kern.log
And a few others:

Code: Select all

linaro@cubie:/var/log$ sudo truncate -s0 btmp.1					
linaro@cubie:/var/log$ sudo truncate -s0 btmp					
linaro@cubie:/var/log$ sudo truncate -s0 message					
linaro@cubie:/var/log$ sudo truncate -s0 messages					
linaro@cubie:/var/log$ sudo truncate -s0 auth.log.1					
linaro@cubie:/var/log$ sudo truncate -s0 debug					
linaro@cubie:/var/log$ sudo truncate -s0 messages.1					
linaro@cubie:/var/log$ sudo truncate -s0 kern.log.1					
linaro@cubie:/var/log$ sudo truncate -s0 syslog.1					
linaro@cubie:/var/log$ sudo truncate -s0 lastlog					
linaro@cubie:/var/log$ sudo truncate -s0 dpkg.log.1					
linaro@cubie:/var/log$ sudo truncate -s0 auth.log					
And look:
ScreenShot065.jpg
ScreenShot065.jpg (8.83 KiB) Viewed 2853 times
And:
ScreenShot066.jpg
ScreenShot066.jpg (12.62 KiB) Viewed 2853 times
And:
ScreenShot067.jpg
ScreenShot067.jpg (70.53 KiB) Viewed 2851 times
This give my CB more space then ever...:-)


THANKS ALL!!

xxx
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
marcelr
Posts: 42
Joined: Friday 22 May 2015 21:10
Target OS: Linux
Domoticz version: svn 2470
Location: Ehv, NL
Contact:

Re: HDD is running to the max

Post by marcelr »

Drops as steeply as the exchange rate of the British Pound :-)

grtz,

marcelr
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: HDD is running to the max

Post by Derik »

marcelr wrote:Drops as steeply as the exchange rate of the British Pound :-)

grtz,

marcelr

:D :D :lol: :lol: :lol:
My drop is better/;
Image
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
Egregius
Posts: 2592
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: HDD is running to the max

Post by Egregius »

gizmocuz wrote:also the tool 'ncdu' is helpfull, it is a graphical 'du' application
Seems a nice tool, thanks.

But not available on OS X and Synology so I searched some more and came up with these commands:

find biggest folders

Code: Select all

perl -e'%h=map{/.\s/;99**(ord$&&7)-$`,$_}`sudo du -h -d 1`;die@h{sort%h}'
find biggest folders and subfolders

Code: Select all

perl -e'%h=map{/.\s/;99**(ord$&&7)-$`,$_}`sudo du -h`;die@h{sort%h}'
Almost the same can be done with piping the du command to sort. This command will display all sizes in MB so not good for small files:
1 level:

Code: Select all

sudo du -BM -d 1| sort -n
With subfolders:

Code: Select all

sudo du -BM | sort -n
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest