4 thoughts on - Figure Out Why Kdump Is Not Producing Dmesg In CentOS Stream 8

• As far as I remember, kdump won’t write anything if the produced file will be bigger than the FS.Go through RHEL8 documentation and setup kdump .

  Best Regards, Strahil Nikolov

  On Sat, Apr 13, 2024 at 13:16, public1020 via Discuss wrote: I created a CentOS 8 stream machine on aws, and configured crashkernel in grub

  “`
  $ cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 root=UUID=e52ef623-609b-4202-9b2c-ac7aba5c3bee ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 crashkernel=128M
  “`

  Then I installed and verified kdump is functioning

  “`
  # systemctl status kdump
  ● kdump.service – Crash recovery kernel arming Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: enabled)
  Active: active (exited) since Sat 2024-04-13 08:42:55 UTC; 40s ago Process: 808 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCESS)
  Main PID: 808 (code=exited, status=0/SUCCESS)

  Apr 13 08:42:43 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Install squash loader ***
  Apr 13 08:42:43 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Stripping files ***
  Apr 13 08:42:44 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Stripping files done ***
  Apr 13 08:42:44 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Squashing the files inside the initramfs ***
  Apr 13 08:42:54 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Squashing the files inside the initramfs done ***
  Apr 13 08:42:54 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Creating image file ‘/boot/initramfs-4.18.0-552.el8.x86_64kdump.img’ ***
  Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Creating initramfs image file ‘/boot/initramfs-4.18.0-552.el8.x86_64kdump.img’ done ***
  Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal kdumpctl[814]: kdump: kexec: loaded kdump kernel Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal kdumpctl[814]: kdump: Starting kdump: [OK]
  Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal systemd[1]: Started Crash recovery kernel arming.
  “`

  Now I need to mimic kernel crash, so I used sysrq to do that

  “`
  echo c > /proc/sysrq-trigger
  “`

  However, after reboot, I didn’t see a dmesg file in /var/crash as in CentOS 7.X

  “`
  # ls /var/crash
  – nothing, empty folder –
  “`

  The only thing I can see is /var/log/kdump.log

  “`
  + 2024-04-13 08:35:56 /usr/bin/kdumpctl@679: ret=0
  + 2024-04-13 08:35:56 /usr/bin/kdumpctl@680: set +x
  + 2024-04-13 08:42:55 /usr/bin/kdumpctl@675: /sbin/kexec -s -d -p ‘–command-line=BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never nokaslr novmcoredd hest_disable disable_cpu_apicid=0’ –initrd=/boot/initramfs-4.18.0-552.el8.x86_64kdump.img /boot/vmlinuz-4.18.0-552.el8.x86_64
  Try gzip decompression. Try LZMA decompression.
  + 2024-04-13 08:42:55 /usr/bin/kdumpctl@679: ret=0
  + 2024-04-13 08:42:55 /usr/bin/kdumpctl@680: set +x
  + 2024-04-13 08:44:31 /usr/bin/kdumpctl@675: /sbin/kexec -s -d -p ‘–command-line=BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never nokaslr novmcoredd hest_disable disable_cpu_apicid=0’ –initrd=/boot/initramfs-4.18.0-552.el8.x86_64kdump.img /boot/vmlinuz-4.18.0-552.el8.x86_64
  Try gzip decompression. Try LZMA decompression.
  + 2024-04-13 08:44:31 /usr/bin/kdumpctl@679: ret=0
  + 2024-04-13 08:44:31 /usr/bin/kdumpctl@680: set +x
  “`

  I need the dmesg to understand what’s causing the crash. How can I do that?
  P.S I have a kernel module that triggers a crash but I don’t know why, and to demonstrated the problem with kdump, I use sysrq so anyone want to help can easily reproduce it.

• I have a disk large enough to hold the whole memory dump. And the same configuration works fine on CentOS 7.

• CentOS 8 is with newer stack.I remember there were subtle differences.

  This article is a good start: https://access.redhat.com/articles/5332081#ssh

  Best Regards, Strahil Nikolov

• I checked the article you mentioned, I don’t need SSH or nfs, I just want the same behaivor on CentOS 7. It dumps everything to /var/crash/IP-date/{vmlinux,dmesg.txt}

  I checked the kdump.conf and I didn’t see anything changed, I suspect it might be a problem from makedumpfile

  Is there any way to create a debug output from this program or kdump? Maybe it failed somehow. https://www.linux.org/docs/man5/makedumpfile.html