Best Configuration For Encrypted Software RAID 1?

Home » CentOS » Best Configuration For Encrypted Software RAID 1?
CentOS 12 Comments

I’m setting up a computer with CentOS 6.4 and a mirrored software RAID. I would like it to be encrypted so I was wondering what the best configuration is. The only info I could find is http://lists.CentOS.org/pipermail/CentOS-docs/2008-October/001912.html but it appears to be a bit old and the info on the wiki (
http://wiki.CentOS.org/HowTos/EncryptTmpSwapHome ) doesn’t seem to address RAIDs.

My main question is will it be better to encrypt the RAID itself or the two partitions used by the RAID? Any other things I should be aware of?

Thanks, Dave

12 thoughts on - Best Configuration For Encrypted Software RAID 1?

  • This depends on your use-case. Personally, I want my servers to be able to boot headless, so I leave /boot, and / unencrypted, RAID or not. Then I encrypt the LV (or partition) I am going to put data I care about on. I don’t think there is any benefit to encrypting the partitions behind the MD device as it won’t be able to form until you decrypt the devices. I’d keep crypt on the resulting /dev/mdX, at the lowest.

    Again, it depends on your use-case.

  • /boot absolutely can’t be encrypted

    I use LUKS in conjunction with Serial over LAN … otherwise I’d have to manually mount (or script it) so my encrypted volume is mounted. In my case as well, I only have the volume where my backup data goes … so
    /boot, /, and others are not encrypted (no need).

    Create a software raid array and then create your LUKS encrypted volume on top of that md device. It is *highly recommended* to write random data to the underlying disk device prior to creating the LUKS volume. I believe I
    referenced [0] on the Arch Linux wiki a bunch way back when, but you’ll find other great references on the Gentoo wiki as well.

    cryptsetup is the utility you’re looking for. (As I’m sure you already know…since your mailing list link mentions it.) :)

    [0] https://wiki.archlinux.org/index.php/Dm-crypt_with_LUKS

  • Thanks for the feedback. I’m actually doing this through an Anaconda config file and I’m using the following settings:

    part raid.boota –sizeP0 –ondisk sda part raid.bootb –sizeP0 –ondisk sdb raid /boot –fstype=ext4 –level=1 –device=md0 raid.boota raid.bootb part raid.slasha –grow –sizeP0 –ondisk sda part raid.slashb –grow –sizeP0 –ondisk sdb raid / –fstype=ext4 –level=1 –encrypted –passphrase= –device=md1 raid.slasha raid.slashb

    Is that the “right” way to do it? Or is there a configuration that would work better?

    Thanks, Dave

  • That looks right. Maybe somebody else will have a tip.

    The key to your puzzle is –encrypted and you’ve got that.

    I don’t have a kickstart config that has both software raid and luks encryption, so that’s a scenario I have not tested. Sounds like a perfect lab to spin up a VM on a Friday! :)

  • That example is helpful for the RAID stuff, but it doesn’t show an example of encryption and so that’s what my question is really about. Should the raid line have the encryption like I have in my example? Or should it be on each of the partitions? I’m just curious if there’s a pro/con to each of those options. My very basic/quick testing seemed to indicate that doing it on the raid line was the better solution, but I was just wondering if anyone had experience with this?

    Thanks, Dave

  • It is impossible to encrypt /boot… (you only have one other partition =>
    root)
    Single LUKS volume = ease of management? So I’m thinking LUKS with LVM
    inside.

    If you encrypt root, but don’t encrypt swap you could be exposing data. As far as I’ve seen if you encrypt root, then you encrypt swap and most/all other partitions. It really depends on what you want encrypted (remember there will be some CPU cycles and additional I/O wait).

    ** Where else would you place the encryption? (Please list of scenarios.)
    **

    You’ll notice I choose not to put the LUKS passphrase in plain text in the kickstart config (Anaconda asks for one before partitioning the disks).

    I have an example below with and without LVM. I’ve done non-LVM encrypted setups in the past, so I haven’t tested that recently. But the raid+luks+lvm I kickstarted on a test VM (notice the virtio disks).

    # clears ALL partitions on ALL drives clearpart –all –initlabel

    # set up software raid arrays part raid.01 –sizeQ2 –asprimary –ondisk=vda part raid.02 –sizeQ2 –asprimary –ondisk=vdb part raid.03 –size24 –asprimary –ondisk=vda part raid.04 –size24 –asprimary –ondisk=vdb part raid.05 –sizeQ20 –asprimary –grow –ondisk=vda part raid.06 –sizeQ20 –asprimary –grow –ondisk=vdb

    # set up partitions on the software raid array (no LVM)
    #raid /boot –fstype=ext4 –level=RAID1 raid.01 raid.02
    #raid swap –fstype=swap –level=RAID0 raid.03 raid.04 –encrypted
    #raid / –fstype=ext4 –level=RAID1 raid.05 raid.06 –encrypted
    # #
    # OR #
    # #
    # set up partitions on the software raid array (with LVM)
    raid /boot –fstype=ext4 –device=md0 –level=RAID1 raid.01 raid.02
    raid swap –fstype=swap –device=md1 –level=RAID0 raid.03 raid.04
    –encrypted raid pv.01 –fstype=ext4 –device=md2 –level=RAID1 raid.05 raid.06
    –encrypted

    # actual LVM volume group and logical volumes volgroup vg00 pv.01
    logvol / –vgname=vg00 –size@96 –maxsizea44 –grow
    –fstype=ext4 –name=rootfs logvol /home –vgname=vg00 –sizeQ2 –grow
    –fstype=ext4 –name=home

    I’ve noticed the same passphrase is used for both LUKS volumes (Anaconda only asked for a passphrase once) — swap and the LVM PV are separate softraid MDs.

    Hopefully this is helpful.

  • Yes, just ease of management. We’re not going to be adding/changing disks or anything and only having to type a single passphrase is best.

    Thanks for the reminder, but we’re actually not using swap on this system.

    The encryption could be on the individual partitions and then the raid would use the two encrypted partitions. I tried this and it seemed that then there were two encryption processes running so my very simple testing seemed to indicate that this sort of configuration would double the CPU usage.

    Thanks the examples are good for comparison.

    We want the kickstart to be fully automated/unattended. We use a dummy passphrase during the kickstart and then change it after the setup is complete. Also, like I mentioned before, only having to type a single passphrase is what we’re going for.

    Thanks, Dave

  • encrypt data once and let md mirror the encrypted stuff or let md mirror and encrypt data twice, once per raid member.

    Encryption is CPU hungry. Performance wise the winner seems clear.

  • Certainly the simplest.
    +1 for LVM inside the LUKS volume ;)

    In my example, my swap was striped, so it made sense (but with the price of RAM there’s hardly an excuse for swapping to disk!).

    I’ll second this. I’ve noticed the iowait is fairly high on my offsite encrypted backup server (backups are on software raid with LUKS on top). And the kcryptd process consumes a fair bit of cpu time.

    And kcrypd isn’t SMP aware [0] (unless that has changed) so there’s another bottleneck.

    [0] http://www.redhat.com/archives/dm-devel/2009-April/msg00151.html

  • Thanks for all the feedback. I just wanted to make sure that there wasn’t some gotcha that I was missing or any slight tweak that would improve performance, but it sounds like there’s not. Thanks, Dave

  • For the sake of anyone who reads this later. –fsprofile=ext4 needs to be specified on the partition definition lines so that it will be use the proper parameters. When it’s not specified, only the defaults are used are performance is significantly degraded.

    So the above definition would look like this:
    part raid.boota –fsprofile=ext4 –sizeP0 –ondisk sda part raid.bootb –fsprofile=ext4 –sizeP0 –ondisk sdb raid /boot –fstype=ext4 –level=1 –device=md0 raid.boota raid.bootb part raid.slasha –fsprofile=ext4 –grow –sizeP0 –ondisk sda part raid.slashb –fsprofile=ext4 –grow –sizeP0 –ondisk sdb raid / –fstype=ext4 –level=1 –encrypted –passphrase= –device=md1 raid.slasha raid.slashb

    You can see info about it here:
    https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/s1-kickstart2-options.html

LEAVE A COMMENT