14 thoughts on - BUG: Soft Lockup – CPU#0 Stuck For 36s! [swapper/0:0]

• FWIW, I’ve had no problems with CentOS 7.x VMs running in ESXi 5.5.0 GA
  (build 1331820)

• 2016-08-18 12:39 GMT-04:00 correomm :

  I’m also seeing those errors in several servers, running under 5.5. Currently investigating if this
  <https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId09996>
  has anything to do (the resource overcommit bit).

  HTH, Carlos.

• Does this happen (only) while taking or consolidating snapshots? The VM
  is suspended during these operations and the OS isn’t too crazy about it, especially if you have slow storage.

  Jack

• Yes, I tried it, but does not exists:

  vmguest # cat /proc/sys/kernel/softlockup_thresh cat: /proc/sys/kernel/softlockup_thresh: No such file or directory

• 2016-08-18 13:32 GMT-04:00 JJB :

  Nope, no snapshots. Just plain running. In fact, many times the guests are under light usage (internal instrumentation, no external VMware stats). We’re investigating because we do have reasons to believe that our provider is probably overcommitting or overselling (not out of malice, AFAIK).

  HTH, Carlos.

• No, I don’t use snapshots.

  It is a Dell 2 TB Enterprise 3.5″ SATA Hard Drive.

  The disk activity of the host is normal to low. Few VM’s.

• Not sure if this was the last email on this.  If not ignore me. However I found a post for new operating systems that says to set the watchdog_thresh value instead of softlockup_thresh. 
  http://askubuntu.com/questions/592412/why-is-there-no-proc-sys-kernel-softlockup-thresh this is an Ubuntu post, but on my CentOS 7 system this parameter exists, and softlockup_thresh does not.  I have set it but I will need to see if I still get the CPU lock up messages on my VM. I hope this helps.KM

  From: correomm
  To: CentOS mailing list
  Sent: Thursday, August 18, 2016 1:50 PM
  Subject: Re: [CentOS] BUG: soft lockup – CPU#0 stuck for 36s! [swapper/0:0]

  Yes, I tried it, but does not exists:

  vmguest # cat /proc/sys/kernel/softlockup_thresh cat: /proc/sys/kernel/softlockup_thresh: No such file or directory

  CentOS mailing list CentOS@CentOS.org https://lists.CentOS.org/mailman/listinfo/CentOS

• All,This happens on all of our CentOS 7 VMs.  but as stated in the email trail, the file softlockup_thresh does not exist.  Should it be added?  What is the best way to get rid of this behavior. Thanks in advance and sorry if I missed something along the way.KM

  From: correomm
  To: CentOS mailing list
  Sent: Thursday, August 18, 2016 1:55 PM
  Subject: Re: [CentOS] BUG: soft lockup – CPU#0 stuck for 36s! [swapper/0:0]

  Yes, I tried it, but does not exists:

  vmguest # cat /proc/sys/kernel/softlockup_thresh cat: /proc/sys/kernel/softlockup_thresh: No such file or directory

  CentOS mailing list CentOS@CentOS.org https://lists.CentOS.org/mailman/listinfo/CentOS

  | | Virus-free. http://www.avg.com |

• Never saw this email….Did anyone get it?  anyone know how to fix this?thanks again.

  From: KM
  To: CentOS mailing list
  Sent: Monday, August 7, 2017 11:26 AM
  Subject: Re: [CentOS] BUG: soft lockup – CPU#0 stuck for 36s! [swapper/0:0]

  All,This happens on all of our CentOS 7 VMs.  but as stated in the email trail, the file softlockup_thresh does not exist.  Should it be added?  What is the best way to get rid of this behavior. Thanks in advance and sorry if I missed something along the way.KM

  From: correomm
  To: CentOS mailing list
  Sent: Thursday, August 18, 2016 1:55 PM
  Subject: Re: [CentOS] BUG: soft lockup – CPU#0 stuck for 36s! [swapper/0:0]

  Yes, I tried it, but does not exists:

  vmguest # cat /proc/sys/kernel/softlockup_thresh cat: /proc/sys/kernel/softlockup_thresh: No such file or directory

  CentOS mailing list CentOS@CentOS.org https://lists.CentOS.org/mailman/listinfo/CentOS

  | | Virus-free. http://www.avg.com |

• Never saw this email….Did anyone get it?  anyone know how to fix this?thanks again.

  From: KM
  To: CentOS mailing list
  Sent: Monday, August 7, 2017 11:26 AM
  Subject: Re: [CentOS] BUG: soft lockup – CPU#0 stuck for 36s! [swapper/0:0]

  All,This happens on all of our CentOS 7 VMs.  but as stated in the email trail, the file softlockup_thresh does not exist.  Should it be added?  What is the best way to get rid of this behavior. Thanks in advance and sorry if I missed something along the way.KM

  From: correomm
  To: CentOS mailing list
  Sent: Thursday, August 18, 2016 1:55 PM
  Subject: Re: [CentOS] BUG: soft lockup – CPU#0 stuck for 36s! [swapper/0:0]

  Yes, I tried it, but does not exists:

  vmguest # cat /proc/sys/kernel/softlockup_thresh cat: /proc/sys/kernel/softlockup_thresh: No such file or directory

  CentOS mailing list CentOS@CentOS.org https://lists.CentOS.org/mailman/listinfo/CentOS

  | | Virus-free. http://www.avg.com |

• Yes, I see this behavior as well. Never have found a solution – other than increasing the threshold and pretending it doesn’t happen.

• Adam Tauno Williams wrote:

  mark

• On bare metal is usually means some hardware has gone into an uninteruptable IRQ and the CPU is waiting for it to go away. I saw this with systems with Green disk drives a while ago. Something going to talk to the drive would just sit for long times while the drive spun up, the cache was validated etc. Other things would be drives on USB disks too when some other USB item started needing input.. since it is a hub environment they can spew for a while and the CPU would report a soft-lockup.

• Not hardly. We discovered green drives were nothing we wanted right after they came out. And I’m talking at work, with servers, all drives are either enterprise, as we bought them, or NAS-rated (e.g. WD Red).

  mark