Avago (LSI) SAS-3 Controller, Poor Performance On CentOS 7

Home » CentOS » Avago (LSI) SAS-3 Controller, Poor Performance On CentOS 7
CentOS 4 Comments

I have a handful of new systems where I’ve seen unexpectedly low disk performance on an Avago SAS controller, when using CentOS 7. It looked like a regression, so I installed CentOS 6 on one of them and reloaded CentOS 7 on the other. Immediately after install, a difference is apparent in the RAID rebuild speed. The CentOS 6 system is initializing its software RAID5 array at somewhere around 120MB/s, while the CentOS 7
system is rebuilding at around 55MB/s.

It took a while to get an older system installed, and I wasn’t able to boot CentOS 6 under UEFI, to that’s one difference between the systems.
I also set the elevator on CentOS 7 to cfq, to match the CentOS 6
system. That didn’t have any apparent effect.

Other than that, the kernel reports “MSI-X vectors supported: 96” on C6, but “MSI-X vectors supported: 8” on C7. I don’t know if that’s a difference that would affect performance.

Does anyone have experience with these controllers? Any pointers while I continue tracking this down?

Output from lspci follows, as well as dmesg output regarding the MPT
controller and the RAID array status on C6 and C7.

01:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 (rev 02)
Subsystem: LSI Logic / Symbios Logic Device 30e0
Physical Slot: 1
Flags: bus master, fast devsel, latency 0, IRQ 24
I/O ports at e000 [size%6]
Memory at fb240000 (64-bit, non-prefetchable) [sizedK]
Memory at fb200000 (64-bit, non-prefetchable) [size%6K]
Expansion ROM at fb100000 [disabled] [size=1M]
Capabilities: [50] Power Management version 3
Capabilities: [68] Express Endpoint, MSI 00
Capabilities: [d0] Vital Product Data
Capabilities: [a8] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [c0] MSI-X: Enable+ Count

4 thoughts on - Avago (LSI) SAS-3 Controller, Poor Performance On CentOS 7

  • What firmware is/are on the cards?
    The driver version difference between C6 and C7 could interact badly with old firmware. What firmware does $vendor suggest?

    Does C7 in BIOS mode work any better?

    Run `lsiget` and compare lsi tech support bundles.

  • In the dmesg output I included:
    mpt3sas0: LSISAS3008: FWVersion(10.00.00.00), ChipRevision(0x02), BiosVersion(08.25.00.00)

    I suspected that was the case, and tried running elrepo’s “kmod-mpt3sas”
    from their testing repo. C7 has version 04.100.00.00, while elrepo’s appears to be 13.100.00.00. That didn’t change anything.

    Haven’t asked yet, since the same firmware produces different behavior on the two OSs.

    That’ll be one of my next tests.

    I’ll look around for that, too. No vendor tools are installed, currently.

  • I know with the SAS2 chips (SAS2008, SAS2308, etc), FreeBSD strongly recommends the firmware revision matches the driver revision, so if the driver is P20, the firmware also should be P20. I don’t have any SAS3
    stuff.