38 thoughts on - Suddenly Increased My Hard Disk

• Start with ‘du -hs /’. If one of the directories is bigger than expected, re-run in that directory. Repeat until you find the unexpectedly large directory.

• Hi Digimer,

  Should i need to run above command in home directory or root. Could you help me.

• The command that he gave you will work from any directory. The / means start from the root directory so where you are when you run it is irrelevant.

• howevver if he doesn’t run it as the root user, it won’t be able to calculate space in directories the current user can’t access.

  if you are running X-Windows, there’s a nifty utility KDirStat that scans the disk tallying space, then gives you an interactive graphical view of usage. I believe you can install it from epel as package k4dirstat

• Hi Chandran,

  Basically, df reads the superblock only and trusts it completely. du reads each object and sums them up.

  Any running process can keep a deleted file open. This means the space will still be reserved and seen by df, but since du will no longer see a reference to that file in the directory tree, it cannot see those reserved blocks.

  Try to see the number of open file which could be causing the problem.

  –Regards Ashishkumar S. Yadav

• Hi John, Am currently running dos mode not graphical mode. Could you have any other method.

  Hi Ashish, You are correct. qmailtoaster backup file which was around 184 GB in backup.gz type and i have removed .bz2 type file with the same backup/mailbkup directory.

  After removed .bz2 file it’s gone backup.gz also which was 184 GB file. I have run this command locate .gz but couldn’t find out it.

  how do i see the open files.

  Could you help me to find out this file or any soln.

• for unix shell (its *not* ‘dos mode’, MSDOS is a legacy system of Microsoft), du is the command to use.

  du -hs /*

  will give you a summary of the space used by every top level directory. ignore /dev /proc and /sys as they aren’t real directories, theya re pseudo file systems. if it turns out, for example, that /var is the biggest pig, then

  du -hs /var/*

  to see the totals of whats in each folder in /var

  rinse, repeat…

• Hi Chandran,

  Follow this link for more information,
  <http://www.cyberciti.biz/faq/howto-linux-get-list-of-open-files/>

  –Regards Ashishkumar S. Yadav

• Hi John, Ashish,

  Still no luck . I have tried your commands in root folder. It’s showing max size 384 only in home directory.

  But if i try df -h shown 579.

  Is there any way to find out recycle bin folder

• Linux shell has no such thing as a recycle bin, thats a windows thing
  (some linux graphical desktops might create one in for a user’s files, it would be in his /home/username somewhere).

  df -h shows a lot of info, for instance on one of my servers, I see…

  # df -h Filesystem Size Used Avail Use% Mounted on
  /dev/mapper/vg_free-lv_root
  50G 7.7G 39G 17% /
  tmpfs 12G 0 12G 0% /dev/shm
  /dev/sda1 477M 146M 306M 33% /boot
  /dev/mapper/vg_free-lv_home
  30G 7.1G 23G 25% /home
  /dev/mapper/vg_free-lvpgsql
  30G 671M 30G 3% /var/lib/pgsql
  /dev/mapper/vg_free-lvimages
  150G 61G 90G 41% /var/lib/libvirt/images
  /dev/mapper/vgdata-lvhome2
  1.8T 470G 1.4T 26% /home2

  # du -hs /home/*
  398M /home/downloads
  16K /home/ipsloth
  16K /home/junk
  4.1G /home/observers
  32M /home/pierce
  2.5G /home/scac

  2.5GB + 4.1 GB + the odds and ends there adds up to pretty close to the
  7.1GB ‘used’ in the /home file system.

• Hi John, Thank you.

  For my system shows like below. df -h

  Filesystem Size Used Avail Use% Mounted on
  /dev/sda2 909G 576G 287G 67% /
  tmpfs 3.9G 0 3.9G 0% /dev/shm
  /dev/sda1 3.9G 160M 3.5G 5% /boot
  /dev/sdb1 916G 382G 488G 44% /bkhdd

  First hard disk /dev/sda Second hard disk /dev/sdb

  The problem is in first hard disk show above size but my email box folders having below result.

  du -hs /*
  8.0K /backup
  7.5M /bin
  *382G /bkhdd*
  89M /boot
  4.0K /cgroup
  4.0K /command
  208K /dev
  79M /etc
  *386G /home*
  4.0K /isoqlog.domains
  322M /lib
  20K /lost+found
  4.0K /media
  0 /misc
  4.0K /mnt
  0 /net
  64M /root
  14M /sbin
  4.0K /selinux
  4.0K /service
  4.0K /srv
  680K /tmp
  4.3G /usr
  1.9G /var

  Here

  /bkhdd is mount in /dev/sdb

  /home is mount in /dev/sda

  But the size is not match in /dev/sda

• I do not see a mount for /home in the file systems listed on df -h, are you sure its mounted ? it appears to just be a folder on /

• And the equivalent of that at the cli is ncdu – also from EPEL.

• Hi John,
  /home is not mount. It’s under the / root directory. sorry i had mention wrongly.
  / is mount in /dev/sda

• In article , Chandran Manikandan wrote:

  You can look to see what files have been created or altered in the last day by using “find” (as root):

  # find / -xdev -type f -ctime -1 -ls

  For the last 2 days, change the -1 to -2

  You can read “man find” to understand the options.

  This will only show you files that still exist in the filesystem. If a large file has been created and is still held open although deleted (unlinked), it will not show up. But in that case, you can either search for it with
  “lsof”, or just reboot the system to reclaim the space.

  Cheers Tony

• You’re most of the way there. You’ve probably identified the culprit.
  The backup file consumed your space, and some process still has it open so that it’s not being freed on the filesystem.

  As root, look for deleted files:

  # ls -l /proc/*/fd/* | grep ‘(deleted)’

  For instance, among the output on my system, I see this:

  l-wx——. 1 gmessmer gmessmer 64 Apr 7 09:41 /proc/28087/fd/1 ->
  /home/gmessmer/tmp.1 (deleted)

  The file /proc/28087/fd/1 appears as a symlink to a deleted file. In that case, the process with PID 28087 has the file open. I can use “ps”
  to determine what the process is, and terminate it appropriately.

• Hi Tony and Gordon, Thanks for your help. I have tried above your commands I couldn’t see that size of the file.

  Usually the qmailbackup created like below format in backup location.

  201604151929-backup.tar.gz
  201604151929-qmailadminpasswd.tar.bz2
  201604151929-qmailcontrolusers.tar.bz2
  201604151929-spamassassin-files.tar.bz2
  201604151929-squirrelmail-plugins.tar.bz2
  201604151929-squirrelmail-prefs.tar.bz2
  201604151929-vpopmail.sql.gz
  201604151929-vpopmail.tar.bz2

  Then i have tried to delete rm -f *.bz2 files or *.gz files.

  The above same steps followed on 01 April files.

  After that it doesn’t show the 201604011929-backup.tar.gz file in backup location. but the extra 194 GB size occupied in my server which is above mentioned backup file size.

  I had checked mailbox folder size 386 GB and there is no other big size folder and files in my server. But i am wonder why it shows 580GB occupied in my hard disk.

  how to find out and solve this issue. I have run fail2ban and firewall also in my server. i have doubt anybody hacked or accessed my system from outside. Could anyone help me.

• ===>
  please post results.

  ===>
  why do you believe that increase is related to gmailtoaster?

  ===>

  ===>
  have you at any time run kde?

  ria, i use kde and some time back, found that depending on how a file or directory is deleted, deletion is saved in cache.

  is it possible that gnome is doing same?

• Hi Geleem, Please have a look below of my result.

  For my system shows like below. df -h

  Filesystem Size Used Avail Use% Mounted on
  /dev/sda2 909G 576G 287G 67% /
  tmpfs 3.9G 0 3.9G 0% /dev/shm
  /dev/sda1 3.9G 160M 3.5G 5% /boot
  /dev/sdb1 916G 382G 488G 44% /bkhdd

  First hard disk /dev/sda Second hard disk /dev/sdb

  The problem is in first hard disk show above size but my email box folders having below result.

  du -hs /*
  8.0K /backup
  7.5M /bin
  *382G /bkhdd*
  89M /boot
  4.0K /cgroup
  4.0K /command
  208K /dev
  79M /etc
  *386G /home*
  4.0K /isoqlog.domains
  322M /lib
  20K /lost+found
  4.0K /media
  0 /misc
  4.0K /mnt
  0 /net
  64M /root
  14M /sbin
  4.0K /selinux
  4.0K /service
  4.0K /srv
  680K /tmp
  4.3G /usr
  1.9G /var

  Here

  /bkhdd is mount in /dev/sdb

  / is mount in /dev/sda

  But the size is not match in /dev/sda

  I am running CentOS 6.5 32 bit with CLI Mode. How do i check the cache size in this mode.

• ===>
  ok. look at size from a diff perspective.

  in a virtual terminal, enter df -H, then df -h, followed by df -k.

  are you sizes closer with “-k” ?

  if in wonder, open second v-term, enter man df to get an understanding of how block size and rounding cause interesting results.

  same applies to du.

  neither have a -b to see actual bytes, which i believe is because disk usage is by blocks, not bytes.

• Do you have any dot (.XXX) directories or large files under / ?
  These would not be measured by “du -hs /*”.

• Unmount /bkhdd temporarily, and then run “du -hs /*” again. One possibility is that you ran a backup while /dev/sdb1 was not mounted, and now you are consuming a bunch of space that you can’t see because you’ve mounted another filesystem on top of it.

• What commands? I suggested that you look for deleted but open files using:
  # ls -l /proc/*/fd/* | grep ‘(deleted)’

  If you see any, you won’t see their size. All you’ll really see is the PID of the process that has the files open, and in that case, you must terminate that process.

  Do you see deleted files when you run the command above?

• lsof command may be of help either…

  Valeri

  ++++++++++++++++++++++++++++++++++++++++
  Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
  ++++++++++++++++++++++++++++++++++++++++

• ===>
  lsof | grep deleted would work better. ;-)

• Better how?

  lsof hasn’t always been installed by default, while /proc is pretty reliably available.

• ===>

  ‘lsof’, aka, list open files, will list every open file on system, and there are a lot.

  ‘grep deleted’ will list _only_ the deleted files that are still open.

  ===>

  install lsof if not installed.

• That’s exactly the same thing that “ls -l /proc/*/fd/* | grep
  ‘(deleted)'” will do. So how is lsof better, exactly?

  I know. I’m frustrated today. I spent the weekend dealing with people nit-picking. It’s hard to exaggerate how much time and energy can be wasted by people insisting on meaningless changes.

• ===>

  i did not say it was. i was only correcting Valerie’s post.

  ===>

  post was not to be nit-picking.

  adding ‘grep deleted’ is not meaningless.

  now, can we get back to op’s problem?

• Thanks, I was kind of indulging my laziness, mentioned the command name and assumed keen person will look up command usage him/herself. But on a receiving end being lazy person I would prefer to just copy and paste using _your_ post as opposed to my own ;-)

  Valeri

  ++++++++++++++++++++++++++++++++++++++++
  Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
  ++++++++++++++++++++++++++++++++++++++++

• <>

  ===>
  welcome and welcome to the ‘lzpc’, lazy people club’. charter members excuse it as terse. B-)

  being aware aware of your knowledge from your past post, i was a bit amazed at your reply.

  there is nothing wrong c&p. i am of habit when opportunity presents itself. it lessens typos and need for thinking about spelling.

  also good for adding a list member’s name to my email address book. ;-)

• lsof will show the sizes of the deleted files.

  lsof | grep deleted | sort -k7n

• That’s a reasonable G.D. answer.

  The only problem is that on my system, the default output includes a TID column that is present in only some of the lines, so the size column is 8 for many lines and 7 for the others, so sort doesn’t work as expected unless you change lsof’s output.

  And you probably should “grep ‘ (deleted)$’ unless you want to see files like /home/gordon/.thunderbird/xxxxxxx.default/calendar-data/deleted.sqlite.

  Still, I guess I learned one thing.

• Dear Friends,

  Finally fixed my issue. As you told i have unmount the external hard disk then i checked the
  /bkhdd/backup folder. I saw that 190GB backup tar.gz file then i deleted and again remount it.

  Thanks a lot for your kind supporting to me to fix this issue.

  Why it’s happened like this environment and how to avoid it.

• don’t write to mount points when they aren’t mounted, the files get written to the file system. and don’t create any directories in the mount point… like, if you were mounting /dev/sdb1 as /bkhdd then on the root file ssytem (without that mount) there should never have been any /bkhdd/backup directory. in fact /bkhdd should not be writable by your user processes.

  I remember older Unix systems would refuse to mount a file system to a non-empty directory, for exactly this reason, it hides stuff thats already there.

• John, thanks for reminding this to all of us, I for one keep forgetting about it (at least if I’m not dealing with it myself which usually acts as a federal offense on me ;-) I know one Linux admin who removes write bit from mount points.

  Valeri

  ++++++++++++++++++++++++++++++++++++++++
  Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
  ++++++++++++++++++++++++++++++++++++++++

• I usually put a file with a name like “mount point for /dev/mdxyz” into that mount point folder just in case I fail to mount it. Then when you do a ls or ll it shows me I have a problem.

• Generally the way to avoid it would be for your backup application to fail if the destination does not exist, and alert the admin.

  For instance, rsnapshot has the “no_create_root” option to do exactly that.

  If you’re making backups with scripts of your own, just start with something like:
  test -d $destination || alert_admin_and_exit

  …where $destination is a sub-directory under the mount point. Backing up directly to the mount point can make it more complicated to detect the presence of the mounted filesystem.