Kernel Errors After Updating

Home » CentOS » Kernel Errors After Updating
CentOS 1 Comment

Hi all,

I have updated my CentOS 6.5 KVM host to kernel 2.6.32-431.23.3 this morning … After 2 hours working, the following kernel error appears and all vm guests goes slowly ….

device monif4 entered promiscuous mode monif: port 5(monif4) entering forwarding state INFO: task cgroup:83 blocked for more than 120 seconds.
Not tainted 2.6.32-431.23.3.el6.x86_64 #1
“echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message. cgroup D 0000000000000009 0 83 2 0x00000000
ffff880411cb9d60 0000000000000046 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 ffff880411cb2aa0
ffff880411cb3058 ffff880411cb9fd8 000000000000fbc8 ffff880411cb3058
Call Trace:
[] __mutex_lock_slowpath+0x13e/0x180
[] ? do_rebuild_sched_domains+0x0/0x50
[] mutex_lock+0x2b/0x50
[] cgroup_lock+0x15/0x20
[] do_rebuild_sched_domains+0x18/0x50
[] worker_thread+0x170/0x2a0
[] ? autoremove_wake_function+0x0/0x40
[] ? worker_thread+0x0/0x2a0
[] kthread+0x96/0xa0
[] child_rip+0xa/0x20
[] ? kthread+0x0/0xa0
[] ? child_rip+0x0/0x20
INFO: task cgroup:83 blocked for more than 120 seconds.
Not tainted 2.6.32-431.23.3.el6.x86_64 #1
“echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message. cgroup D 0000000000000009 0 83 2 0x00000000
ffff880411cb9d60 0000000000000046 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 ffff880411cb2aa0
ffff880411cb3058 ffff880411cb9fd8 000000000000fbc8 ffff880411cb3058
Call Trace:
[] __mutex_lock_slowpath+0x13e/0x180
[] ? do_rebuild_sched_domains+0x0/0x50
[] mutex_lock+0x2b/0x50
[] cgroup_lock+0x15/0x20
[] do_rebuild_sched_domains+0x18/0x50
[] worker_thread+0x170/0x2a0
[] ? autoremove_wake_function+0x0/0x40
[] ? worker_thread+0x0/0x2a0
[] kthread+0x96/0xa0
[] child_rip+0xa/0x20
[] ? kthread+0x0/0xa0
[] ? child_rip+0x0/0x20
INFO: task cgroup:83 blocked for more than 120 seconds.
Not tainted 2.6.32-431.23.3.el6.x86_64 #1
“echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message. cgroup D 0000000000000009 0 83 2 0x00000000
ffff880411cb9d60 0000000000000046 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 ffff880411cb2aa0
ffff880411cb3058 ffff880411cb9fd8 000000000000fbc8 ffff880411cb3058
Call Trace:
[] __mutex_lock_slowpath+0x13e/0x180
[] ? do_rebuild_sched_domains+0x0/0x50
[] mutex_lock+0x2b/0x50
[] cgroup_lock+0x15/0x20
[] do_rebuild_sched_domains+0x18/0x50
[] worker_thread+0x170/0x2a0
[] ? autoremove_wake_function+0x0/0x40
[] ? worker_thread+0x0/0x2a0
[] kthread+0x96/0xa0
[] child_rip+0xa/0x20
[] ? kthread+0x0/0xa0
[] ? child_rip+0x0/0x20
INFO: task cgroup:83 blocked for more than 120 seconds.
Not tainted 2.6.32-431.23.3.el6.x86_64 #1
“echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message. cgroup D 0000000000000009 0 83 2 0x00000000
ffff880411cb9d60 0000000000000046 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 ffff880411cb2aa0
ffff880411cb3058 ffff880411cb9fd8 000000000000fbc8 ffff880411cb3058
Call Trace:
[] __mutex_lock_slowpath+0x13e/0x180
[] ? do_rebuild_sched_domains+0x0/0x50
[] mutex_lock+0x2b/0x50
[] cgroup_lock+0x15/0x20
[] do_rebuild_sched_domains+0x18/0x50
[] worker_thread+0x170/0x2a0
[] ? autoremove_wake_function+0x0/0x40
[] ? worker_thread+0x0/0x2a0
[] kthread+0x96/0xa0
[] child_rip+0xa/0x20
[] ? kthread+0x0/0xa0
[] ? child_rip+0x0/0x20
INFO: task cgroup:83 blocked for more than 120 seconds.
Not tainted 2.6.32-431.23.3.el6.x86_64 #1
“echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message. cgroup D 0000000000000009 0 83 2 0x00000000
ffff880411cb9d60 0000000000000046 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 ffff880411cb2aa0
ffff880411cb3058 ffff880411cb9fd8 000000000000fbc8 ffff880411cb3058
Call Trace:
[] __mutex_lock_slowpath+0x13e/0x180
[] ? do_rebuild_sched_domains+0x0/0x50
[] mutex_lock+0x2b/0x50
[] cgroup_lock+0x15/0x20
[] do_rebuild_sched_domains+0x18/0x50
[] worker_thread+0x170/0x2a0
[] ? autoremove_wake_function+0x0/0x40
[] ? worker_thread+0x0/0x2a0
[] kthread+0x96/0xa0
[] child_rip+0xa/0x20
[] ? kthread+0x0/0xa0
[] ? child_rip+0x0/0x20
INFO: task cgroup:83 blocked for more than 120 seconds.
Not tainted 2.6.32-431.23.3.el6.x86_64 #1
“echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message. cgroup D 0000000000000009 0 83 2 0x00000000
ffff880411cb9d60 0000000000000046 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 ffff880411cb2aa0
ffff880411cb3058 ffff880411cb9fd8 000000000000fbc8 ffff880411cb3058
Call Trace:
[] __mutex_lock_slowpath+0x13e/0x180
[] ? do_rebuild_sched_domains+0x0/0x50
[] mutex_lock+0x2b/0x50
[] cgroup_lock+0x15/0x20
[] do_rebuild_sched_domains+0x18/0x50
[] worker_thread+0x170/0x2a0
[] ? autoremove_wake_function+0x0/0x40
[] ? worker_thread+0x0/0x2a0
[] kthread+0x96/0xa0
[] child_rip+0xa/0x20
[] ? kthread+0x0/0xa0
[] ? child_rip+0x0/0x20
INFO: task cgroup:83 blocked for more than 120 seconds.
Not tainted 2.6.32-431.23.3.el6.x86_64 #1
“echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message. cgroup D 0000000000000009 0 83 2 0x00000000
ffff880411cb9d60 0000000000000046 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 ffff880411cb2aa0
ffff880411cb3058 ffff880411cb9fd8 000000000000fbc8 ffff880411cb3058
Call Trace:
[] __mutex_lock_slowpath+0x13e/0x180
[] ? do_rebuild_sched_domains+0x0/0x50
[] mutex_lock+0x2b/0x50
[] cgroup_lock+0x15/0x20
[] do_rebuild_sched_domains+0x18/0x50
[] worker_thread+0x170/0x2a0
[] ? autoremove_wake_function+0x0/0x40
[] ? worker_thread+0x0/0x2a0
[] kthread+0x96/0xa0
[] child_rip+0xa/0x20
[] ? kthread+0x0/0xa0
[] ? child_rip+0x0/0x20
INFO: task cgroup:83 blocked for more than 120 seconds.
Not tainted 2.6.32-431.23.3.el6.x86_64 #1
“echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message. cgroup D 0000000000000009 0 83 2 0x00000000
ffff880411cb9d60 0000000000000046 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 ffff880411cb2aa0
ffff880411cb3058 ffff880411cb9fd8 000000000000fbc8 ffff880411cb3058
Call Trace:
[] __mutex_lock_slowpath+0x13e/0x180
[] ? do_rebuild_sched_domains+0x0/0x50
[] mutex_lock+0x2b/0x50
[] cgroup_lock+0x15/0x20
[] do_rebuild_sched_domains+0x18/0x50
[] worker_thread+0x170/0x2a0
[] ? autoremove_wake_function+0x0/0x40
[] ? worker_thread+0x0/0x2a0
[] kthread+0x96/0xa0
[] child_rip+0xa/0x20
[] ? kthread+0x0/0xa0
[] ? child_rip+0x0/0x20

Uhmm , how can I debug this?? Any idea??

One thought on - Kernel Errors After Updating

  • Here I have the same error, but we’re using VMware. I tried to change de type os SCSI controller on VMware and the CentOS loaded other module too, but the VMs continues to displays this timeout erros on dmesg. The machine just do not respond to anything, and 3~5min after, it turns back to life.

    I tried the MPT (mptscsih) and de VMware (vmw_pvscsi) module, in many kernel and CentOS version; all with the same error. I’ve also found this same error on old kernels like 2.6.18-308.4.1.el5.

    Did you got this erros too?
    mptscsih: ioc0: task abort: SUCCESS (rv=2002) (sc=ffff8801378bdb80)
    mptscsih: ioc0: attempting task abort! (sc=ffff88013782b2c0)

    Or:
    sd 2:0:1:0: [sdb] task abort on host 2, ffff880432fdb8c0
    sd 2:0:5:0: [sdf] task abort on host 2, ffff880432fdb1c0

    —
    Rodrigo Bezerra Brasil Belém, PA, BR

    Intelligence is the ability to avoid doing work, yet getting the work done.
    -Linus Torvalds

    OpenPGP hex keyID: 0xB05DAFA91AA38CA7

    CentOS mailing list CentOS@CentOS.org http://lists.CentOS.org/mailman/listinfo/CentOS