Software RAID Complete Drives Or Individual Partitions

Home » CentOS » Software RAID Complete Drives Or Individual Partitions
CentOS 21 Comments

I have been reading about software raid. I configured my first software raid system about a month ago. I have 4 500 Gig drives configured in RAID 5 configuration with a total of 1.5TB.
Currently I configured the complete individual drivers as software raid, then created a /dev/md0 with the drives I then created a /file_storage partition on /dev/md0.

I created my /boot / and swap partitions on a non raid drive in my system. Is the the proper way to configure software raid?

21 thoughts on - Software RAID Complete Drives Or Individual Partitions

  • I’ve read (and would agree) that using the entire drive can hide the fact that a softraid is there. If you set up a partition on the disk and mark the file system as type “fd” then no matter what you know that disk is part of a softraid array.

    What you configured works. Is it wrong? No.

    I generally use LVM on my systems, so my layouts has /boot carved out as its own partition and then another partition for the LVM PV. And if you use standard partitions instead of LVM, then create individual softraid arrays for each partition.
    * Remember, /boot can only be on a raid1 software array.

    Veering slightly from your original question… I recently set up softraid arrays by hand before invoking the Anaconda installer (on a 6.3 install). Recent mdadm packages that ship with CentOS
    support metadata 1.1 and 1.2 (… actually defaulting to 1.2 I believe), but GRUB 0.97 only supports metadata 1.0 and not the metadata version that mdadm defaulted to. On my CentOS 5 installs in the past I’ve specifically set –metadata=0.90 to avert any catastrophes like this.

    Fun problem to troubleshoot … I knew it once that system wouldn’t boot though. Kind of odd that the installer didn’t pick up on the metadata version and flag it or modify it. In the end I ended up rescuing the system by backing up and recreating the /boot softraid with metadata 1.0 :)

  • Hey Chris,

    What you have done is a totally acceptable way of building a raid array.

    Software raid on Linux is amazingly flexible. It is able to build arrays on individual matching drives as you have done, drives of different physical sizes, a combination physical drives and partitions on other drives, or a combination of partitions on different drives. It can even build a raid array on several partitions on one physical drive, not that you would ever want to do that.

    In other words, if you can dream it up, software raid can probably build it. The question is why are you using raid at all? If you are trying to increase access speed or data security then raid makes sense. The appropriate configuration depends on your available resources and the nature of your intent.


    _
    °v°
    /(_)\
    ^ ^ Mark LaPierre Registered Linux user No #267004
    https://linuxcounter.net/
    ****

  • indeed. the primary justification for the “R” in RAID, Redundant, is high availability. having the OS on a non-raid volume completely violates this. RAID is most definitely NOT a substitute for backups.

  • I essentially configured the software raid to create a home server and not have to worry about individual drives. I’ve built a collection of several several gigs of music and pictures. Enough to look into a raid system. Plus it is alot cheaper than a hardware raid setup. Plus I want to do testing with it to possible use it for some of the clients that the business I work for do work for for basic file server services

    —–Original Message—–
    From: Mark LaPierre Sent: 3/5/2013 6:27 PM
    To: CentOS@CentOS.org Subject: Re: [CentOS] Software RAID complete drives or individual partitions

    Hey Chris,

    What you have done is a totally acceptable way of building a raid array.

    Software raid on Linux is amazingly flexible. It is able to build arrays on individual matching drives as you have done, drives of different physical sizes, a combination physical drives and partitions on other drives, or a combination of partitions on different drives. It can even build a raid array on several partitions on one physical drive, not that you would ever want to do that.

    In other words, if you can dream it up, software raid can probably build it. The question is why are you using raid at all? If you are trying to increase access speed or data security then raid makes sense. The appropriate configuration depends on your available resources and the nature of your intent.


    _
    °v°
    /(_)\
    ^ ^ Mark LaPierre Registered Linux user No #267004
    https://linuxcounter.net/
    ****

  • I am using FreeNAS for my home fileserver, and really liking it. its a turnkey NAS oriented build of FreeBSD. the OS itself boots from a 4GB
    USB stick, so ALL your disks can be in the data raid configuration (I’m using 4x3TB RAIDZ, which is equivalent to raid5). its entirely web managed.

  • Im not so much concerned about the os not being on a raid system. I am really concerned about my data, music, pictures,docs, etc. I run a minimum os CentOS 5.9 install anyway so it would take long to reload the os if i had to.

    —–Original Message—

  • Im in the process of configuring my data to be backed up to external devices.

    —–Original Message—

  • True, the data is generally more important than the OS (or hardware – you can buy more).

    BUT do you _really_ want to reload the OS and have to tweak config files again?
    Especially if you do not have the OS on a raid volume, back at least /etc/
    up nightly or weekly (whatever fits your scenario) so you have at least some config files to go off of.

    There’s a fellow on the Gentoo Forums that has a signature that says:
    “Computer users fall into two groups:-
    those that do backups those that have never had a hard drive fail.”

    You might be lucky, but you won’t want to get caught if your luck to drys up! :)

  • Do you (or anyone…) have any rules of thumb regarding drives over
    2TB or using raid sets that were created on CentOS5 under CentOS6? I
    once booted a CentOS5 box with the initial CentOS6 ‘live’ CD and it permanently broke all of the auto-detected mirrors so I have been a little reluctant to it again.

  • If you configure an entire drive as a raid device, you’d have a device name like /dev/mdp0, which you’d then partition. I think what you’ve done is created only one partition per disk, and made those partitions into a RAID set. That’s not wrong, but it’s not the same thing.

    Non-partitionable RAID sets such as the one you’ve created are the most common configuration for software RAID. Hardware RAID volumes are almost always the partionable type.

    “Proper” is relative to its fitness for a specific purpose. As you haven’t indicated a specific purpose, “proper” doesn’t have any real meaning.

    The array you’ve created will work, and it will protect your data from loss due to the failure of a single disk. You need to make sure your
    “root” mail is delivered to someone who will read it in a timely manner, or else that protection is not useful. The array’s performance will be relatively lower than a single-drive configuration or a RAID10
    configuration, but that may be acceptable for bulk storage. The array will not protect you from filesystem corruption or from accidental deletion. Subject to those and other limitations, your array seems more or less proper.

  • As far as I know, GRUB 0.97 only supports metadata 0.90, as does LILO. Anaconda will create arrays with 0.90 metadata for this reason.

    The kernel wiki disagrees with me:
    https://raid.wiki.kernel.org/index.php/RAID_superblock_formats

    Debian’s documentation indicates that only grub 1.98+20100720-1 or later will boot from a RAID volume with a newer metadata format:
    http://www.debian.org/releases/stable/i386/release-notes/ch-information.en.html#mdadm-metadata

  • My understanding of Linux mdadm RAID5 is that a write will read the block being written and the parity block. The calculations can be done with only those blocks, and the two are written. That’s one extra read per write plus parity calculations.

    I’m quite certain that I’ve seem some hardware RAID arrays that will read the entire stripe to do a write.

    RAID5 will always write more slowly than RAID1 or RAID10, but that can sometimes be acceptable if capacity is more important than performance.

    If there’s anything to avoid, it’d be old 3ware hardware. Those cards are often less reliable than the disks they’re attached to, and that’s saying something.

    All hardware raid is “just software raid in the controller rather than the OS”. The advantages of hardware RAID are offloading parity calculations to dedicated hardware so that the CPU doesn’t need to do it, and a battery backed write cache.

    The write cache is critical to safely writing a RAID array in the event of a power loss, and can greatly improve performance provided that you don’t write enough data to fill the cache.

    The host CPU is very often faster with parity than the dedicated hardware, which is why Alan Cox has been quoted as saying that the best RAID controllers in the world are made by Intel and AMD. However, if you think you need the couple of percent of CPU cycles that would have been used by software RAID, you might prefer the hardware solution.

    If you have a disk on which a bad sector is found, it’s time to replace it no matter how your partitions are set up. Drives reserve a set of sectors for re-mapping sectors that are detected as bad. If your OS
    sees a bad sector, it’s because that reserve has been exhausted. More sectors will continue to go bad, and you will lose data. Always replace a drive as soon as your OS sees bad sectors, or before based on SMART data.

    Partitioning into many smaller chunks is probably a waste of time. Like most of the other participants in this thread, I create software RAID
    sets of one or two partitions per disk and use LVM on top of that.

    Hopefully BTRFS will simplify this even further in the near future. :)

  • I wouldn’t hold my breath. Someone on the BackupPC list reported that it had a tiny limit on hardlinks which would make it seem to me like they don’t quite understand how filesystems are supposed to work.

  • The limitation was fixed in 3.7:
    https://bugzilla.kernel.org/show_bug.cgi?id762

    I think the btfs developers understand quite well how filesystems are supposed to work. As best I understand it, the limitation was a result of back-references, which are an important feature to keep inodes from ending up as orphans. If you’re going to have an on-line fsck, that matters.

  • OK, but in my opinion it is worse if that was a design decision that is just now being changed. Bugs I can understand, but not choosing to design a new filesystem with unrealistic limitations.

  • you need GPT to partition devices larger than 2TB, regardless of the parittion size.

    me, I use parted….

    # parted /dev/sdc “label gpt”
    # parted /dev/sdc -a none “mkpart primary 512s -1s”

    to make a single full disk partition that starts at sector 512, which is
    256K bytes, which is usually a decent boundary for SSDs, large raids, and other such devices.

  • It’s not being changed, per se. A reasonable means of increasing the number of back-references has been implemented. (again, AFAICT)

  • Large drives often have 4k sectors – don’t you want a 1M offset to make sure the partition start is aligned? Gparted seems to do that by default. Also, is it possible to make the raid auto-assemble at boot like smaller ones do? I had to put an ARRAY entry into
    /etc/mdadm.conf with the devices but I think someone advised doing
    “set raid on” in parted.

  • I can tell you from my recent experience that whatever concoction of GRUB
    0.97 (shipped with CentOS 6.3) supports booting off of metadata 1.0

    I often encounter metadata 0.90 … on many [all?] of the aging CentOS 5
    installs I see.

    Grub 1.99 or thereabouts is pretty awesome. Grub2 (as Debian packages it) supports booting off LVM which is slick. Not overly useful, but convenient if you would rather /boot be part of the rootfs … especially with kernels getting larger.

    A /boot partition that could be 100MB with CentOS 5 now needs to be around
    512MB with CentOS 6 (found this out the hard way with a development system). Disk space is cheap … but I still don’t want to waste space!
    :)

LEAVE A COMMENT