Recover From An Fsck Failure

Home » CentOS » Recover From An Fsck Failure
CentOS 4 Comments

This is CentOS-6x.

I have cloned the HDD of a CentOS-6 system. I booted a host with that drive and received the following error:

checking filesystems

/dev/mapper/vg_voinet01-lv_root: clean, 128491/4096000 files, 1554114/16304000
blocks

/dev/sda1: clean, 47/120016 files, 80115/512000 blocks

/dev/mapper/vg_voinet01-lv_home: clean, 7429/204800 files, 90039/819200 blocks

/dev/mapper/vg_voinet01-LogVol04: clean, 770219/20480000 files,
34881086/8102000 blocks

fsck.ext4: Bad magic number in super-block while trying to open
/dev/mapper/vg_voinet01-lv_log
/dev/mapper/vg_voinet01-lv_log The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2 filesystem
(and not swap or ufs or something else), then the superblock is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsk -b 8193

/dev/mapper/vg_voinet-lv_spool: clean, 372/614400 files, 171186/2457600 blocks

*** An error occurred during the file system check.
*** Dropping you to a shell; the system will reboot
*** when you leave the shell. Give root password for maintenance
(or type Control-D to continue):

I ran mke2fs to locate the backup superblocks:

mke2fs -n /dev/mapper/vg_voinet01-lv_log
. . . Superblock backups stored on blocks:
32768, 90304, 163840, 229376, 294912, 819200, 884736, 1605632

and ran:

e2fsck -b 32768 /dev/mapper/vg_voinet01-lv_log The superblock could not be read or does not describe a correct ext2

The same thing happened for the next backup superblock addrees. And all the rest reported an invalid argument error from e2fsck.

Is this recoverable? How?

4 thoughts on - Recover From An Fsck Failure

  • That will only tell you what mke2fs would do on that machine. I don’t know if it will be the same on every machine. You should probably run

    dumpe2fs /dev/mapper/vg_voinet01-lv_log | grep superblock

    If that doesn’t work, then I suspect it’s not recoverable using fsck. If you are sure that it is an ext2/3/4 filesystem on there, then you can try using something like TestDisk to scan for partitions. It should be in epel.

    P.

  • What output do you get from:

    file -s /dev/mapper/vg_voinet01-lv_log
    lsblk -f /dev/mapper/vg_voinet01-lv_log

  • file -s /dev/mapper/vg_voinet01-lv_log
    /dev/mapper/vg_voinet01-lv_log: symbolic link TO ‘../DM-5’
    dm-f

    lsblk -f /dev/mapper/vg_voinet01-lv_log NAME FSTYPE LABEL UUID MOUNTPOINT
    vg_voinet01-lv_log (dm-5)

    The cloned source is a CentOS-6.6 based FreePBSX appliance using lvm. It has a single 500GB HDD. The source host is running and testing of its HDD shows no errors from the test. However, the number of recoverable read and write errors are abnormally high indicating that the HDD is approaching the end of its service life. I wish to replace it without having to rebuild the entire PBX
    system from scratch.

    All of the LVMs on the cloned HDD are ext4 file systems. The only one that failed is the log partition /var/log. If I cannot fix this then I wish to replace it with an empty fs and simply rsync the contents of /var/log from the running system to it. I need to do this anyway just prior to replacement as the PBX is running.

    However, it is about six years since I last had to do this and while I have my notes I would appreciate a short precis of the steps involved.

    This is the file system layout.

    /etc/fstab
    /dev/mapper/vg_voinet01-lv_root / ext4 defaults 1 1
    UUID02ab0fo-f985-4903-86e2-e218b1345e0 /boot ext4 defaults 1 2
    /dev/mapper/vg_voinet01-lv_home /home ext4 defaults 1 2
    /dev/mapper/vg_voinet01-LogVol04 /var ext4 defaults 1 2
    /dev/mapper/vg_voinet01-lv_log /var/log ext4 defaults 1 2
    /dev/mapper/vg_voinet01-lv_spool /var/spool ext4 defaults 1 2
    /dev/mapper/vg_voinet01-lv_swap swap swap defaults 0 0
    tmpfs /dev/shm tmpfs defaults,nodev,. . .

    parted -l Model: ATA WDC WD1002FAEX-0 (scsi)
    Disk /dev/sda: 1000GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 1094kB 525MB 524MB primary ext4 boot
    2 525MB 500GB 500GB primary lvm

    Model: Linux device-maper (linear) (dm)
    Disk: /dev/mapper/vg_voinet01-lv_spool: 68.7GB
    Sector size (logical/physical): 512B/512B
    Partition Table: loop

    Number Start End Size Type File system Flags
    1 0.00B 68.7GB 68.7GB primary ext4

    Error: /dev/mapper/vg_voinet01-lv_log: unrecognized disk label

    Model: Linux device-maper (linear) (dm)
    Disk: /dev/mapper/vg_voinet01-lv_home: 3355MB
    Sector size (logical/physical): 512B/512B
    Partition Table: loop

    Number Start End Size Type File system Flags
    1 0.00B 3355MB 3355MB primary ext4

    Model: Linux device-maper (linear) (dm)
    Disk: /dev/mapper/vg_voinet01-LogVol04: 336GB
    Sector size (logical/physical): 512B/512B
    Partition Table: loop

    Number Start End Size Type File system Flags
    1 0.00B 336GB 336GB primary ext4

    Error: /dev/mapper/vg_voinet01-lv_swap: unrecognized disk label

    Model: Linux device-maper (linear) (dm)
    Disk: /dev/mapper/vg_voinet01-lv_spool: 67.1GB
    Sector size (logical/physical): 512B/512B
    Partition Table: loop

    Number Start End Size Type File system Flags
    1 0.00B 67.1GB 67.1GB primary ext4

    I will have to do this from a live cd I presume as the shell that the failed boot process dropped me into give a “File-based locking initialisation failed.”
    when I try to run vgdisplay or vgs.

    Any guidance appreciated.

  • Hi

    How exactly did you create the cloned disk?

    If the source disk still works and is in operation without system errors, the cloned disk with its filesystems should really be without error as well.

    If the filesystems have errors, maybe the cloning didn’t go well?

    If you cloned the disk while the system was running, it should at least be done in single user mode to prevent filesystem errors.

    Regards, Simon