用QEMU模拟ARM开发板，搭建Linux kernel运行环境

前言

有的时候我们想调试linux kernel或者linux应用程序，但是又没有硬件环境，这个时候可以选择用模拟器的方法，模拟出一个硬件环境。

Android emulator

Android系统的模拟器叫Emulator，使用这个模拟器需要下载Android源代码，编译成功之后才能运行。由于编译Android源码还是需要一定的硬件配置，我尝试了没编译成功。

QEMU

QEMU是一个可以模拟ARM架构的开源软件，我们可以使用他来模拟vexpress-a9开发板，在上面搭建linux kernel运行环境。参考QEMU模拟vexpress-a9 搭建Linux kernel运行环境，成功的在ubuntu 16.04上模拟运行了linux kernel最小系统，这样就方便代码的调试学习了。

以下是kernel启动过程的打印：

neo@neo-virtual-machine:~/work/linux-4.14.7/arch/arm/boot/dts$ qemu-system-arm -M vexpress-a9 -m 512M -kernel ../zImage -dtb ./vexpress-v2p-ca9.dtb -nographic -append "root=/dev/mmcblk0 console=ttyAMA0 rw init=/linuxrc" -sd /home/neo/work/a9rootfs.ext3 
WARNING: Image format was not specified for '/home/neo/work/a9rootfs.ext3' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument
Booting Linux on physical CPU 0x0
Linux version 4.14.7 (neo@neo-virtual-machine) (gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)) #1 SMP Fri Jul 8 21:48:11 CST 2022
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: V2P-CA9
Memory policy: Data cache writeback
CPU: All CPU(s) started in SVC mode.
percpu: Embedded 16 pages/cpu @9fbae000 s36428 r8192 d20916 u65536
Built 1 zonelists, mobility grouping on.  Total pages: 130048
Kernel command line: root=/dev/mmcblk0 console=ttyAMA0 rw init=/linuxrc
log_buf_len individual max cpu contribution: 4096 bytes
log_buf_len total cpu_extra contributions: 12288 bytes
log_buf_len min size: 16384 bytes
log_buf_len: 32768 bytes
early log buf free: 14976(91%)
PID hash table entries: 2048 (order: 1, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 509556K/524288K available (6144K kernel code, 402K rwdata, 1364K rodata, 1024K init, 169K bss, 14732K reserved, 0K cma-reserved)
Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
    vmalloc : 0xa0800000 - 0xff800000   (1520 MB)
    lowmem  : 0x80000000 - 0xa0000000   ( 512 MB)
    modules : 0x7f000000 - 0x80000000   (  16 MB)
      .text : 0x80008000 - 0x80700000   (7136 kB)
      .init : 0x80900000 - 0x80a00000   (1024 kB)
      .data : 0x80a00000 - 0x80a649d8   ( 403 kB)
       .bss : 0x80a6bcb8 - 0x80a96434   ( 170 kB)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
Hierarchical RCU implementation.
	RCU event tracing is enabled.
	RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
GIC CPU mask not found - kernel will fail to boot.
GIC CPU mask not found - kernel will fail to boot.
smp_twd: clock not found -2
sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
clocksource: arm,sp804: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns
Failed to initialize '/smb@4000000/motherboard/iofpga@7,00000000/timer@12000': -22
Console: colour dummy device 80x30
Calibrating local timer... 98.69MHz.
Calibrating delay loop... 973.20 BogoMIPS (lpj=4866048)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
CPU: Testing write buffer coherency: ok
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x60100000 - 0x60100060
Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
SMP: Total of 1 processors activated (973.20 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
random: get_random_u32 called from bucket_table_alloc+0xf0/0x240 with crng_init=0
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 0
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 1024 (order: 4, 65536 bytes)
random: fast init done
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
cpuidle: using governor ladder
hw-breakpoint: debug architecture 0x4 unsupported.
Serial: AMBA PL011 UART driver
OF: amba_device_add() failed (-19) for /memory-controller@100e0000
OF: amba_device_add() failed (-19) for /memory-controller@100e1000
OF: amba_device_add() failed (-19) for /watchdog@100e5000
irq: type mismatch, failed to map hwirq-75 for /interrupt-controller@1e001000!
10009000.uart: ttyAMA0 at MMIO 0x10009000 (irq = 38, base_baud = 0) is a PL011 rev1
console [ttyAMA0] enabled
1000a000.uart: ttyAMA1 at MMIO 0x1000a000 (irq = 39, base_baud = 0) is a PL011 rev1
1000b000.uart: ttyAMA2 at MMIO 0x1000b000 (irq = 40, base_baud = 0) is a PL011 rev1
1000c000.uart: ttyAMA3 at MMIO 0x1000c000 (irq = 41, base_baud = 0) is a PL011 rev1
OF: amba_device_add() failed (-19) for /smb@4000000/motherboard/iofpga@7,00000000/wdt@f000
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Advanced Linux Sound Architecture Driver Initialized.
clocksource: Switched to clocksource arm,sp804
NET: Registered protocol family 2
TCP established hash table entries: 4096 (order: 2, 16384 bytes)
TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
hw perfevents: enabled with armv7_cortex_a9 PMU driver, 1 counters available
workingset: timestamp_bits=30 max_order=17 bucket_order=0
squashfs: version 4.0 (2009/01/31) Phillip Lougher
jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
9p: Installing v9fs 9p2000 file system support
io scheduler noop registered (default)
io scheduler mq-deadline registered
io scheduler kyber registered
clcd-pl11x 10020000.clcd: PL111 designer 41 rev2 at 0x10020000
clcd-pl11x 10020000.clcd: /clcd@10020000 hardware, 1024x768@59 display
Console: switching to colour frame buffer device 128x48
clcd-pl11x 1001f000.clcd: PL111 designer 41 rev2 at 0x1001f000
clcd-pl11x 1001f000.clcd: /smb@4000000/motherboard/iofpga@7,00000000/clcd@1f000 hardware, 640x480@59 display
40000000.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
40000000.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
Concatenating MTD devices:
(0): "40000000.flash"
(1): "40000000.flash"
into device "40000000.flash"
libphy: Fixed MDIO Bus: probed
libphy: smsc911x-mdio: probed
smsc911x 4e000000.ethernet eth0: MAC Address: 52:54:00:12:34:56
isp1760 4f000000.usb: bus width: 32, oc: digital
isp1760 4f000000.usb: NXP ISP1760 USB Host Controller
isp1760 4f000000.usb: new USB bus registered, assigned bus number 1
isp1760 4f000000.usb: Scratch test failed.
isp1760 4f000000.usb: can't setup: -19
isp1760 4f000000.usb: USB bus 1 deregistered
usbcore: registered new interface driver usb-storage
rtc-pl031 10017000.rtc: rtc core: registered pl031 as rtc0
IR NEC protocol handler initialized
IR RC5(x/sz) protocol handler initialized
IR RC6 protocol handler initialized
IR JVC protocol handler initialized
IR Sony protocol handler initialized
IR SANYO protocol handler initialized
IR Sharp protocol handler initialized
IR MCE Keyboard/mouse protocol handler initialized
IR XMP protocol handler initialized
mmci-pl18x 10005000.mmci: Got CD GPIO
mmci-pl18x 10005000.mmci: Got WP GPIO
mmci-pl18x 10005000.mmci: mmc0: PL181 manf 41 rev0 at 0x10005000 irq 34,35 (pio)
ledtrig-cpu: registered to indicate activity on CPUs
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
mmc0: new SD card at address 4567
mmcblk0: mmc0:4567 QEMU! 32.0 MiB 
input: AT Raw Set 2 keyboard as /devices/platform/smb@4000000/smb@4000000:motherboard/smb@4000000:motherboard:iofpga@7,00000000/10006000.kmi/serio0/input/input0
aaci-pl041 10004000.aaci: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 33
aaci-pl041 10004000.aaci: FIFO 512 entries
oprofile: using arm/armv7-ca9
NET: Registered protocol family 17
9pnet: Installing 9P2000 support
Registering SWP/SWPB emulation handler
rtc-pl031 10017000.rtc: setting system clock to 2022-07-09 02:59:40 UTC (1657335580)
ALSA device list:
  #0: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 33
input: ImExPS/2 Generic Explorer Mouse as /devices/platform/smb@4000000/smb@4000000:motherboard/smb@4000000:motherboard:iofpga@7,00000000/10007000.kmi/serio1/input/input2
EXT4-fs (mmcblk0): mounting ext3 file system using the ext4 subsystem
EXT4-fs (mmcblk0): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext3 filesystem) on device 179:0.
Freeing unused kernel memory: 1024K
random: crng init done
can't run '/etc/init.d/rcS': No such file or directory

Please press Enter to activate this console. 
/ # 

在启动的最后有这样一句打印，说明rootfs仍然不够完善：

can't run '/etc/init.d/rcS': No such file or directory

在进入控制台后发现只有根文件系统里的目录结构，而没有常用的/proc、/sys/等目录，/dev目录下也没有生成设备文件。
proc和sysfs虚拟文件系统需要手动挂载才会生成。分别mkdir创建/proc和、/sys/目录，然后执行以下指令：

mount -t proc proc /proc  
mount sysfs /sys -t sysfs
mdev -s

就可以在目录下看到相应的文件。

/proc # ls
1              28             768            execdomains    partitions
10             312            769            fb             self
11             313            770            filesystems    slabinfo
12             314            771            fs             softirqs
13             316            8              interrupts     stat
14             318            838            iomem          swaps
15             328            9              ioports        sys
16             4              asound         irq            sysrq-trigger
17             444            buddyinfo      kallsyms       sysvipc
18             445            bus            kmsg           thread-self
19             446            cgroups        kpagecount     timer_list
2              465            cmdline        kpageflags     tty
20             5              config.gz      loadavg        uptime
21             524            consoles       locks          version
22             6              cpu            meminfo        vmallocinfo
23             678            cpuinfo        misc           vmstat
24             7              crypto         modules        zoneinfo
242            727            device-tree    mounts
25             764            devices        mtd
26             765            diskstats      net
27             766            driver         pagetypeinfo

/sys # ls
block     class     devices   fs        module
bus       dev       firmware  kernel    power

/dev # ls
console             tty10               tty55
cpu_dma_latency     tty11               tty56
fb0                 tty12               tty57
fb1                 tty13               tty58
full                tty14               tty59
gpiochip0           tty15               tty6
gpiochip1           tty16               tty60
gpiochip2           tty17               tty61
gpiochip3           tty18               tty62
hwrng               tty19               tty63
input               tty2                tty7
kmsg                tty20               tty8

创建好了设备文件之后，应用程序才可以对设备驱动节点进行访问。

使用QEMU还可以用图形界面的方式启动linux kernel：

qemu-system-arm -M vexpress-a9 -m 512M -kernel ../zImage -dtb ./vexpress-v2p-ca9.dtb -append "root=/dev/mmcblk0 rw init=/linuxrc" -sd /home/neo/work/a9rootfs.ext3

效果就是模拟在显示器上显示控制台画面，因此可以调试写framebuffer刷屏的程序效果，下图执行程序向/dev/fb0写入，将屏幕刷成蓝色：