kdump
is a service which provides a crash dumping mechanism. The service enables you to save the contents of the system memory for analysis. kdump
uses the kexec
system call to boot into the second kernel (a capture kernel) without rebooting; and then captures the contents of the crashed kernel’s memory (a crash dump or a vmcore) and saves it into a file. The second kernel resides in a reserved part of the system memory.
You can enable kdump
for all installed kernels on a machine or only for specified kernels. This is useful when there are multiple kernels used on a machine, some of which are stable enough that there is no concern that they could crash.
When kdump
is installed, a default /etc/kdump.conf
file is created. The file includes the default minimum kdump
configuration. You can edit this file to customize the kdump
configuration, but it is not required.
The kdump
service is installed and activated by default on the new Red Hat Enterprise Linux installations. Learn about kdump
and how to install kdump
when it is not enabled by default.
Some installation options, such as custom Kickstart installations, in some cases do not install or enable kdump
by default. If this is your case, follow the procedure below.
Prerequisites
An active RHEL subscription
The kexec-tools package
Fulfilled requirements for
kdump
configurations and targets. For details, see Supported kdump configurations and targets.
Procedure
Check whether
kdump
is installed on your system:
# rpm -q kexec-tools
Output if the package is installed:
kexec-tools-2.0.17-11.el8.x86_64
Output if the package is not installed:
package kexec-tools is not installed
1. Install
kdump
and other necessary packages by:
# dnf install kexec-tools
Important
Starting with kernel-3.10.0-693.el7 the Intel IOMMU
driver is supported with kdump
. For prior versions, kernel-3.10.0-514[.XYZ].el7 and earlier, it is advised that Intel IOMMU
support is disabled, otherwise the capture kernel is likely to become unresponsive.
Plan and build your kdump
environment.
When planning and building your kdump
environment, it is important to know how much space the crash dump file requires.
The makedumpfile --mem-usage
command estimates how much space the crash dump file requires. It generates a memory usage report. The report helps you determine the dump level and which pages are safe to be excluded.
Procedure
# makedumpfile --mem-usage /proc/kcore
TYPE PAGES EXCLUDABLE DESCRIPTION
-------------------------------------------------------------
ZERO 501635 yes Pages filled with zero
CACHE 51657 yes Cache pages
CACHE_PRIVATE 5442 yes Cache pages + private
USER 16301 yes User process pages
FREE 77738211 yes Free pages
KERN_DATA 1333192 no Dumpable kernel data
Important
The makedumpfile --mem-usage
command reports required memory in pages. This means that you must calculate the size of memory in use against the kernel page size.
The memory reservation for kdump
occurs during the system boot. The memory size is set in the system’s Grand Unified Bootloader (GRUB) configuration. The memory size depends on the value of the crashkernel=
option specified in the configuration file and the size of the system physical memory.
You can define the crashkernel=
option in many ways. You can specify the crashkernel=
value or configure the auto
option. The crashkernel=auto
parameter reserves memory automatically, based on the total amount of physical memory in the system. When configured, the kernel automatically reserves an appropriate amount of required memory for the capture kernel. This helps to prevent Out-of-Memory (OOM) errors.
Procedure
1. Prepare the crashkernel=
option.
Prepare the
crashkernel=
option.
For example, to reserve 128 MB of memory, use the following:
crashkernel=128M
Alternatively, you can set the amount of reserved memory to a variable depending on the total amount of installed memory. The syntax for memory reservation into a variable is
crashkernel=
:
,
:
. For example:
crashkernel=512M-2G:64M,2G-:128M
The command reserves 64 MB of memory if the total amount of system memory is in the range of 512 MB and 2 GB. If the total amount of memory is more than 2 GB, the memory reserve is 128 MB.
2. Apply the crashkernel=
option to your boot loader configuration:
# grubby --update-kernel=ALL --args="crashkernel=
# grubby --update-kernel=ALL --args="crashkernel=auto"
Replace
with the value of the crashkernel=
option that you prepared in the previous step.
The crash dump is usually stored as a file in a local file system, written directly to a device. Alternatively, you can set up for the crash dump to be sent over a network using the NFS
or SSH
protocols. Only one of these options to preserve a crash dump file can be set at a time. The default behavior is to store it in the /var/crash/
directory of the local file system.
Prerequisites
Root
permissions.kdump
configurations and targets. For details, see Supported kdump configurations and targets.Procedure
To store the crash dump file in
/var/crash/
directory of the local file system, edit the /etc/kdump.conf
file and specify the path:
path /var/crash
The option path /var/crash
represents the path to the file system in which kdump
saves the crash dump file.
You can test that the crash dump process works and is valid before the machine enters production.
Warning
The commands below cause the kernel to crash. Use caution when following these steps, and never carelessly use them on active production system.
Procedure
kdump
enabled.kdump
is running:# systemctl start kdump
# systemctl enable --now kdump
# systemctl is-active kdump
active
echo 1 > /proc/sys/kernel/sysrq
echo c > /proc/sysrq-trigger