Troubles Expanding File System. – Solved

Home » CentOS » Troubles Expanding File System. – Solved
CentOS No Comments

To follow-up and provide a conclusion to my issue, in case anyone else runs into a similar situation.

TLDR – go to bottom and read item #7.

To recap the issue:
I have a Dell PowerEdge server with a CentOS KVM host (Earth) with one CentOS guest (Sequoia) that I am trying to expand the partition and filesystem on.  I have LVM logical volumes on the host system (Earth), which are used as devices/partitions on the guest system (Sequoia).  In this particular situation I have successfully extended the logical volume (lv_SeqEco) on Earth from 500GB to 700GB.

1.  Checking the disk information (lsblk) on Earth shows that the logical volume (lv_SeqEco) is now listed as 700GB.

2.  Checking disk information (lsblk) on Sequoia shows that the disk
/dev/vde is still listed as 500GB, and partition /dev/vde1 where the mount point /ecosystem is located is also listed as 500GB.

3.  I had tried using the resize2fs command to expand the filesystem on
/dev/vde1, but it returned with the result that there was nothing to do.  Which makes sense now after I checked the disk information, since
/dev/vde on Sequoia has not increased from 500GB to 700GB.

4.  On previous occasions when I have done this task, I would just start GParted on Sequoia and use the GUI to expand the partition and filesystem.  I am unable to do this now, as the VGA adapter on my server has died and I have no graphical output to the attached monitor.

5.  My goal was to not have to not have to reboot the system. Especially since it is a 10-year old server and there is no longer VGA output to view the boot process.

What I tried, and what worked to solve my issue:

1.  First, I tried rescannning the device on the guest:
echo 1 >
/sys/devices/pci0000:00/0000:00:01.1/host1/target1:0:0/1:0:0:0/rescan

This resulted in no output, and checking lsblk on the guest (Sequoia)
showed no change:
vde  500GB  disk vde1   500GB   part /ecosystem

2.  Second, I tried using “virsh blockresize” from the host (Earth) system. Get the block size information earth# virsh domblkinfo SequoiaVM vde output:  vde capacity 751619276800

Resize the block (express the size in Bytes)
earth# virsh blockresize SequioaVM vde 751619276800B
output:  Block device vde is resized.

Check the block device on the guest to confirm. sequoia# lsblk vde   700GB   disk vde1   500GB   part /ecosystem

Therefore, virsh blockresize was successful.

3.  I now have the larger disk recognized on the guest system, but still need to expand the partition and the filesystem.  Tried the following options.

resize2fs -p /dev/vde1
   output:  filesystem is already xx blocks long, nothing to do.

growpart /dev/vde 1
   output:  This showed output indicating the partition had changed
      changed:  partition=1 start=63
      old:  size=1048575937
               end=1048576000
      new:  size=1468003572
                 end=1468003635

However, checking lsblk on the guest still showed the same result
   vde   700GB   disk
   vde1   500GB   part /ecosystem

On the off chance that growpart was successful, I ran resize2fs again and it produced the same result as above.

partprobe
   output:  failed to re-read the partition table on all attached devices (including /dev/vde1); device or resource busy.

Tried using “parted” and its resize command.  First, using the print command to get the parameters of /dev/vde (output: end=751617861119B, size=751617828864B).  Then changing parted to select (use) /dev/vde1 and used print command to view parameters of /dev/vde1 (output: 
end=536870879743B, size=536870879744B).

(parted)# resize 1 0 751617861119
output:  The location 751617861119 is outside of the device /dev/vde1

Looked at using fdisk; however, the documentation stated that the only way to change the partition size using fdisk is by deleting it and recreating it.  I didn’t want to do this.

I finally decided that everything indicated that I would not be able to complete this while the system was online and/or mounted, despite all my research information showing that.

4.  Unmounted the filesystem of the guest and run partprobe.

sequoia# service smb stop sequoia# umount /ecosystem sequoia# partprobe
   output:  This produced the same warnings as before (failed to re-read the partition, device busy) for all partitions that were still mounted, but did not produce the warning for partition /dev/vde1 that had been unmounted.

sequoia# lsblk
  vde   700GB   disk
   vde1   700GB   part /ecosystem

Success.  The new partition is now recognized in the kernel, and I
should now be able to resize the filesystem.

5.  Run resize2fs

sequoia# resize2fs -p /dev/vde1
   output:  please run e2fsck -f /dev/vde1 first

sequoia# e2fsck -f -C 0 /dev/vde1  (-C 0 will display the progress)
   output:  system passed all checks.

sequoia# resize2fs -p /dev/vde1
   output:  resizing the filesystem on /dev/vde1 to 183500446 (4k)
blocks, begin Pass 1, extending inode table, filesystem is now 183500446
blocks long.

6.  Remount the filesystem and check the filesystem size.

sequoia# mount /dev/vde1 /ecosystem sequoia# service smb start sequoia# df -h
   filesystem=/dev/vde1
   size=689G
   used=459G
   available=196G
   used%=71%
   mount=/ecosystem

7.  So in conclusion I needed to do the following steps to resize and expand my disk without a reboot.  There may still be other ways that this can be accomplished, but this worked for my CentOS 6 system (both host and guest).

  a.  On the host system, lvextend to expand the logical volume used by the guest as it’s disk.
  b.  On the host system, virsh blockresize to get the guest system to recognize the expanded disk space.
  c.  On the guest system, growpart to grow the partition to the expanded space.
  d.  On the guest system, unmount the filesystem (stop services as required).
  e.  On the guest system, partprobe to get the kernel to recognize the expanded partition.
  f.  On the guest system, resize2fs (run filesystem check if required)
to expand the filesystem.

Thanks to all that provided suggestions and insight, it was very helpful in my research and methodical stepping through this process.

Jeff