VSCode 调试 Linux内核

• 内核和gdb在虚拟机上，为了可以图形化方面查看调试，Windows上使用VSCode 
• 目前环境配置：Host Ubuntu 20.04，Target aarch64，Kernel 4.12，Qemu 4.2.1

1、虚拟机安装工具

sudo apt-get install gcc-aarch64-linux-gnu build-essential libncurses5-dev gdb-multiarch qemu-system-arm

2、编译 busybox

• 下载 busybox 源码进行编译文件系统，把编译好的busybox拷贝到 Kernel 4.12源码中的 _install 目录下

3、编译内核

• 1. 下载 Kernel 4.12 源码到虚拟机上，文章编译目标是 aarch64
• 2. 内核交叉编译需要环境变量准备

export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-gnu-

• 3. 进入内核目录下 使用 默认的 aarch64配置
  • make defconfig 生成一个默认的 .config文件

make defconfig

• 4. 修改内核配置 make menuconfig

# initramfs目录修改，即要打包文件系统目录名，这里是_install
# General setup --->
#     [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support
#         (_install) Initramfs source file(s)
# 删除默认的启动参数
# Boot options  --->
#     ()  Default kernel command string
# 调整地址总线长度为48位
# Kernel Features  --->
#     Page size (4KB)  ---> 
#     Virtual address space size (48-bit)  --->
# 内核编译加入调试信息，一定要选上
# Kernel hacking  --->
#     Compile-time checks and compiler options  ---> 
#         [*] Compile the kernel with debug info

• 5.  修改后编译，make -j8 ，速度取决你的配置 不多说，一般需要个20分钟左右
• 编译后两个文件注意：
  • linux-4.12/arch/arm64/boot/Image  没有调试信息内核文件
  • linux-4.12/vmlinux 有调试信息内核文件（较大）

4、调试前测试

• 根据自己及其状态，修改内存、cpu等参数
• 下面代码可以自己写一个脚本 如 run_qemu.sh

#!/usr/bin/bash
qemu-system-aarch64 -machine virt -cpu cortex-a57 \
    -nographic -smp 4 -m 2048 \
    -kernel Image -append "rdinit=/linuxrc console=ttyAMA0"

• 这里给出运行后进入文件系统shell下 示例

yexiang@think-pc:$ ./run_qemu.sh 
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.12.14 (ssji@lenovo) (gcc version 8.4.0 (Ubuntu/Linaro 8.4.0-3ubuntu1) ) #1 SMP PREEMPT Sun May 9 14:30:08 CST 2021
[    0.000000] Boot CPU: AArch64 Processor [411fd070]
[    0.000000] Machine model: linux,dummy-virt
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: UEFI not found.
[    0.000000] cma: Reserved 16 MiB at 0x00000000bf000000
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000bfffffff]
[    0.000000] NUMA: Adding memblock [0x40000000 - 0xbfffffff] on node 0
[    0.000000] NUMA: Initmem setup node 0 [mem 0x40000000-0xbfffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xbefe8b00-0xbefea5ff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000040000000-0x00000000bfffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000bfffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000bfffffff]
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv0.2 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: Trusted OS migration not required
[    0.000000] percpu: Embedded 24 pages/cpu @ffff80007efb5000 s59800 r8192 d30312 u98304
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] CPU features: enabling workaround for ARM erratum 834220
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 516096
[    0.000000] Policy zone: DMA
[    0.000000] Kernel command line: rdinit=/linuxrc console=ttyAMA0
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Memory: 2027968K/2097152K available (8892K kernel code, 1102K rwdata, 5992K rodata, 2240K init, 401K bss, 52800K reserved, 16384K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffff000000000000 - 0xffff000008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000   (129022 GB)
[    0.000000]       .text : 0xffff000008080000 - 0xffff000008930000   (  8896 KB)
[    0.000000]     .rodata : 0xffff000008930000 - 0xffff000008f10000   (  6016 KB)
[    0.000000]       .init : 0xffff000008f10000 - 0xffff000009140000   (  2240 KB)
[    0.000000]       .data : 0xffff000009140000 - 0xffff000009253a00   (  1103 KB)
[    0.000000]        .bss : 0xffff000009253a00 - 0xffff0000092b80b4   (   402 KB)
[    0.000000]     fixed   : 0xffff7dfffe7fd000 - 0xffff7dfffec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffff7e0000000000 - 0xffff800000000000   (  2048 GB maximum)
[    0.000000]               0xffff7e0000000000 - 0xffff7e0002000000   (    32 MB actual)
[    0.000000]     memory  : 0xffff800000000000 - 0xffff800080000000   (  2048 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000]  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GICv2m: range[mem 0x08020000-0x08020fff], SPI[80:143]
[    0.000000] arch_timer: cp15 timer(s) running at 62.50MHz (virt).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns
[    0.000209] sched_clock: 56 bits at 62MHz, resolution 16ns, wraps every 4398046511096ns
[    0.009126] Console: colour dummy device 80x25
[    0.011416] Calibrating delay loop (skipped), value calculated using timer frequency.. 125.00 BogoMIPS (lpj=250000)
[    0.011594] pid_max: default: 32768 minimum: 301
[    0.013171] Security Framework initialized
[    0.015443] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.021669] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.024714] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes)
[    0.024868] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes)
[    0.153895] ASID allocator initialised with 65536 entries
[    0.249592] EFI services will not be available.
[    0.411219] smp: Bringing up secondary CPUs ...
[    0.820675] Detected PIPT I-cache on CPU1
[    0.821393] CPU1: Booted secondary processor [411fd070]
[    0.824500] smp: Brought up 1 node, 2 CPUs
[    0.824581] SMP: Total of 2 processors activated.
[    0.825076] CPU features: detected feature: 32-bit EL0 Support
[    0.828533] CPU: All CPU(s) started at EL1
[    0.829238] alternatives: patching kernel code
[    0.889558] devtmpfs: initialized
[    0.899109] DMI not present or invalid.
[    0.900367] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.900637] futex hash table entries: 512 (order: 4, 65536 bytes)
[    0.903859] pinctrl core: initialized pinctrl subsystem
[    0.965044] NET: Registered protocol family 16
[    1.033952] cpuidle: using governor menu
[    1.039365] vdso: 2 pages (1 code @ ffff000008937000, 1 data @ ffff000009145000)
[    1.039684] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    1.098071] DMA: preallocated 256 KiB pool for atomic allocations
[    1.119816] Serial: AMBA PL011 UART driver
[    1.178089] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 39, base_baud = 0) is a PL011 rev1
[    1.192223] console [ttyAMA0] enabled
[    1.354242] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    1.363437] ACPI: Interpreter disabled.
[    1.384287] vgaarb: loaded
[    1.386800] SCSI subsystem initialized
[    1.399828] usbcore: registered new interface driver usbfs
[    1.400767] usbcore: registered new interface driver hub
[    1.401663] usbcore: registered new device driver usb
[    1.404212] pps_core: LinuxPPS API ver. 1 registered
[    1.404736] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti 
[    1.405590] PTP clock support registered
[    1.406812] dmi: Firmware registration failed.
[    1.408783] Advanced Linux Sound Architecture Driver Initialized.
[    1.433257] clocksource: Switched to clocksource arch_sys_counter
[    1.437196] VFS: Disk quotas dquot_6.6.0
[    1.438664] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.441856] pnp: PnP ACPI: disabled
[    1.507206] NET: Registered protocol family 2
[    1.518610] TCP established hash table entries: 16384 (order: 5, 131072 bytes)
[    1.519608] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
[    1.520332] TCP: Hash tables configured (established 16384 bind 16384)
[    1.522443] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[    1.527777] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[    1.546594] NET: Registered protocol family 1
[    1.559255] RPC: Registered named UNIX socket transport module.
[    1.559919] RPC: Registered udp transport module.
[    1.560111] RPC: Registered tcp transport module.
[    1.560228] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.776013] hw perfevents: enabled with armv8_pmuv3 PMU driver, 5 counters available
[    1.780724] kvm [1]: HYP mode not available
[    1.817828] audit: initializing netlink subsys (disabled)
[    1.848104] audit: type=2000 audit(0.914:1): state=initialized audit_enabled=0 res=1
[    1.850627] workingset: timestamp_bits=44 max_order=19 bucket_order=0
[    1.963242] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    2.012842] NFS: Registering the id_resolver key type
[    2.018538] Key type id_resolver registered
[    2.019158] Key type id_legacy registered
[    2.020421] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    2.029638] 9p: Installing v9fs 9p2000 file system support
[    2.076946] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[    2.077410] io scheduler noop registered
[    2.078275] io scheduler cfq registered (default)
[    2.078434] io scheduler mq-deadline registered
[    2.078587] io scheduler kyber registered
[    2.125385] pl061_gpio 9030000.pl061: PL061 GPIO chip @0x0000000009030000 registered
[    2.141348] OF: PCI: host bridge /pcie@10000000 ranges:
[    2.144845] OF: PCI:    IO 0x3eff0000..0x3effffff -> 0x00000000
[    2.149004] OF: PCI:   MEM 0x10000000..0x3efeffff -> 0x10000000
[    2.152275] OF: PCI:   MEM 0x8000000000..0xffffffffff -> 0x8000000000
[    2.155441] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]
[    2.169474] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00
[    2.170552] pci_bus 0000:00: root bus resource [bus 00-ff]
[    2.171042] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    2.171340] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]
[    2.171693] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]
[    2.183905] pci 0000:00:01.0: BAR 6: assigned [mem 0x10000000-0x1007ffff pref]
[    2.187256] pci 0000:00:01.0: BAR 4: assigned [mem 0x8000000000-0x8000003fff 64bit pref]
[    2.191994] pci 0000:00:01.0: BAR 1: assigned [mem 0x10080000-0x10080fff]
[    2.192440] pci 0000:00:01.0: BAR 0: assigned [io  0x1000-0x101f]
[    2.243214] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)
[    2.246888] xenfs: not registering filesystem on non-xen platform
[    2.258244] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    2.265352] SuperH (H)SCI(F) driver initialized
[    2.267566] msm_serial: driver initialized
[    2.291855] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    2.360923] loop: module loaded
[    2.376207] hisi_sas: driver version v1.6
[    2.408898] libphy: Fixed MDIO Bus: probed
[    2.409945] tun: Universal TUN/TAP device driver, 1.6
[    2.468145] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    2.483904] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    2.492937] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    2.493821] igb: Copyright (c) 2007-2014 Intel Corporation.
[    2.495130] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
[    2.502320] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    2.512974] sky2: driver version 1.30
[    2.520379] VFIO - User Level meta-driver version: 0.3
[    2.538213] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    2.539512] ehci-pci: EHCI PCI platform driver
[    2.541416] ehci-platform: EHCI generic platform driver
[    2.542212] ehci-orion: EHCI orion driver
[    2.542953] ehci-exynos: EHCI EXYNOS driver
[    2.543439] ehci-msm: Qualcomm On-Chip EHCI Host Controller
[    2.543986] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    2.544343] ohci-pci: OHCI PCI platform driver
[    2.544911] ohci-platform: OHCI generic platform driver
[    2.545436] ohci-exynos: OHCI EXYNOS driver
[    2.547269] usbcore: registered new interface driver usb-storage
[    2.571334] rtc-pl031 9010000.pl031: rtc core: registered pl031 as rtc0
[    2.577226] i2c /dev entries driver
[    2.605723] sdhci: Secure Digital Host Controller Interface driver
[    2.606011] sdhci: Copyright(c) Pierre Ossman
[    2.607088] Synopsys Designware Multimedia Card Interface Driver
[    2.609256] sdhci-pltfm: SDHCI platform and OF driver helper
[    2.614616] ledtrig-cpu: registered to indicate activity on CPUs
[    2.620391] usbcore: registered new interface driver usbhid
[    2.620820] usbhid: USB HID core driver
[    2.630729] NET: Registered protocol family 17
[    2.632364] 9pnet: Installing 9P2000 support
[    2.633092] Key type dns_resolver registered
[    2.635953] registered taskstats version 1
[    2.644272] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[    2.649743] rtc-pl031 9010000.pl031: setting system clock to 2021-06-25 07:23:15 UTC (1624605795)
[    2.654719] ALSA device list:
[    2.654912]   No soundcards found.
[    2.658676] uart-pl011 9000000.pl011: no DMA platform data
[    3.005047] Freeing unused kernel memory: 2240K

Please press Enter to activate this console. 
/ # ls
bin      etc      mnt      sbin     tmp
dev      linuxrc  proc     sys      usr
/ # 

• 提示：进入qemu中退出 按 ctrl+a 然后再按 x 才能退出去

5、虚拟机中使用 gdb调试

• 为了调试内核需要让 gdb 与 qemu 关联
• 调试环境启动Qemu ，下面可以自己写个脚本 如 dbg_qemu.sh 
  • -s: 在1234端口接受GDB调试
  • -S：冻结CPU直到远程GDB输入相应命令

#!/usr/bin/bash
qemu-system-aarch64 -machine virt -cpu cortex-a57 \
    -nographic -smp 4 -m 2048 \
    -kernel Image -append "rdinit=/linuxrc console=ttyAMA0" -S -s

• 运行后就会停住

• 新开一个窗口 启动 gdb 运行步骤如下

$ cd linux-4.12
$ gdb-multiarch vmlinux
(gdb) target remote localhost:1234
(gdb) b start_kernel
(gdb) c
(gdb) layout src

# 可以看到GDB停留在start_kernel处

• 示例流程如下：

yexiang@think-pc:$ gdb-multiarch vmlinux
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
    .

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from vmlinux...
(gdb) 
(gdb) 
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
0x0000000040000000 in ?? ()
(gdb) 
(gdb) b start_kernel
Breakpoint 1 at 0xffff000008f10790: file init/main.c, line 493.
(gdb) c
Continuing.

Thread 1 hit Breakpoint 1, start_kernel () at init/main.c:493
493             set_task_stack_end_magic(&init_task);
(gdb) 
(gdb) layout src

• layout中调试界面等都不友好，所以下面准备 vscode调试

6、VSCode 图形化调试 kenrel

• 1. 调试前先看 VSCode 配置远程登入 Remote-SSH 把 VSCode 远程连接虚拟机配置好
• 2. VSCode 中打开 linux4.2 文件夹 找到 init/main.c ，在 _start_kernel 中打断点

• 3. 按F5进入调试，VSCode 自动生成 .vscode 并且会提示 修改 launch.json 文件

• 4. launch.json 文件 修改如下：

{
    // 使用 IntelliSense 了解相关属性。 
    // 悬停以查看现有属性的描述。
    // 欲了解更多信息，请访问: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "(gdb) 启动",
            "type": "cppdbg",
            "request": "launch",
            // workspaceFolder 是你当前打开文件夹的路径
            "program": "${workspaceFolder}/vmlinux",
            "args": [],
            "stopAtEntry": false,
            "cwd": "${workspaceFolder}",
            "environment": [],
            "externalConsole": false,
            "MIMode": "gdb",
            // 如果远程登入到linux 服务器上面，路径不用写 /user/bin/gdb-multiarch
            "miDebuggerPath": "gdb-multiarch",
            "miDebuggerServerAddress": "localhost:1234",
            "setupCommands": [
                {
                    "description": "为 gdb 启用整齐打印",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ]
        }
    ]
}

• 5.  虚拟机运行 dbg_qemu.sh （上面给出的命令）
• 6.  VSCode 再次 F5 运行

7、可能出现的问题

• 调试过程中有可能会出现乱跳的现象，主要是内核编译优化导致，默认是 -O2，可查资料修改重编内核，如修改为 -O0  （但是 -O0 编译一般会出现问题，具体原因等有时间再查）
• 修改的位置 linux4.2/Makefile

ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE
KBUILD_CFLAGS   += $(call cc-option,-Oz,-Os)
KBUILD_CFLAGS   += $(call cc-disable-warning,maybe-uninitialized,)
else
ifdef CONFIG_PROFILE_ALL_BRANCHES
KBUILD_CFLAGS   += -O2 $(call cc-disable-warning,maybe-uninitialized,)
#KBUILD_CFLAGS  += -O0
else
KBUILD_CFLAGS   += -O2
#KBUILD_CFLAGS   += -O0
endif
endif