树莓派编译uboot及内核
为了实验ebpf，需要自己编译内核开启相关选项，正好手头有树莓派的板子，所以正好用上。
更换内核
首先我自己用官方工具在sd卡上烧录了64位无桌面的系统，然后按照官方的文档，进行了内核的交叉编译，更换后使用uname -a发现确实更换成功了。编译内核没花时间，但wsl2挂载sd卡，去安装新编译的文件折腾了好久，详见WSL2简单探索
内核版本如下:
$ head Makefile
# SPDX-License-Identifier: GPL-2.0
VERSION = 5
PATCHLEVEL = 10
SUBLEVEL = 110
EXTRAVERSION =
NAME = Dare mighty things
挂载sd卡:
#!/bin/bash
sudo mkdir -p /mnt
sudo mkdir -p /mnt/fat32
sudo mkdir -p /mnt/ext4
sudo mount /dev/sdd1 /mnt/fat32
sudo mount /dev/sdd2 /mnt/ext4
卸载sd卡
#!/bin/bash
sudo umount /mnt/fat32
sudo umount /mnt/ext4
编译:
#!/bin/bash
KERNEL=kernel8
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcm2711_defconfig
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- Image modules dtbs -j12
安装:
#!/bin/bash
root=/mnt
bootfs=${root}/fat32
rootfs=${root}/ext4
sudo env PATH=$PATH make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- INSTALL_MOD_PATH=${rootfs} modules_install
#sudo mkdir -p ${bootfs}
#sudo mkdir -p ${bootfs}/overlays/
sudo cp arch/arm64/boot/Image ${bootfs}/kernel-myconfig.img
sudo cp arch/arm64/boot/dts/broadcom/*.dtb ${bootfs}/
sudo cp arch/arm64/boot/dts/overlays/*.dtb* ${bootfs}/overlays/
sudo cp arch/arm64/boot/dts/overlays/README ${bootfs}/overlays/
#tar -acvf output.tar.gz output/
#sudo scp output.tar.gz [email protected]:/home/pi
更换内核成功后开始搞uboot
安装uboot
使用以下脚本编译一次成功
#!/bin/bash
ARCH=arm64
CROSS_COMPILE=aarch64-linux-gnu-
make rpi_3_defconfig
make -j 12
config.h的配置，最主要的可能是这一句。
kernel=u-boot.bin
然而编好改完启动不起来，怀疑是配置有问题，但是看了官方文档，确认64位 3b的板子就是这个defconfig.
重新编译另一个defconfig，成功加载。
#!/bin/bash
export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-gnu-
#make rpi_3_defconfig
make rpi_arm64_defconfig
make -j 12
最终uboot能成功加载，并且从sd卡启动系统成功:
U-Boot> help
? - alias for 'help'
...
fatload - load binary file from a dos filesystem
fatls - list files in a directory (default /)
mmc - MMC sub system
U-Boot> mmc part
Partition Map for MMC device 0 -- Partition Type: DOS
Part Start Sector Num Sectors UUID Type
1 8192 524288 95d33193-01 0c
2 532480 61988864 95d33193-02 83
U-Boot> fatls mmc 0:1
overlays/
29707 bcm2710-rpi-2-b.dtb
18693 COPYING.linux
1594 LICENCE.broadcom
145 issue.txt
...
52476 bootcode.bin
131 cmdline.txt
2217 config.txt
...
8219600 kernel8.img
624640 u-boot.bin
...
21590528 kernel-myconfig.img
41 file(s), 4 dir(s)
U-Boot> fatload mmc 0:1 ${kernel_addr_r} kernel-myconfig.img
21590528 bytes read in 903 ms (22.8 MiB/s)
U-Boot> booti ${kernel_addr_r} - ${fdt_addr}
Moving Image from 0x80000 to 0x200000, end=17e0000
## Flattened Device Tree blob at 2eff7f00
Booting using the fdt blob at 0x2eff7f00
Working FDT set to 2eff7f00
Using Device Tree in place at 000000002eff7f00, end 000000002f002ffe
Working FDT set to 2eff7f00
搞一个环境变量:
mmcboot=fatload mmc 0:1 ${kernel_addr_r} kernel-myconfig.img;booti ${kernel_addr_r} - ${fdt_addr}
之后可以直接run mmcboot来从sd卡启动了。
这里没设置bootargs，看下启动参数
pi@link:~$ cat /proc/cmdline
coherent_pool=1M 8250.nr_uarts=1 snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1 video=Composite-1:720x480@60i vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000 console=ttyAMA0,115200 console=tty1 root=PARTUUID=95d33193-02 rootfstype=ext4 fsck.repair=yes rootwait cfg80211.ieee80211_regdom=CN
pi@link:~$ cat /boot/cmdline.txt
console=serial0,115200 console=tty1 root=PARTUUID=95d33193-02 rootfstype=ext4 fsck.repair=yes rootwait cfg80211.ieee80211_regdom=CN
对比启动后的参数和cmdline.txt里的参数，发现两者不同。也不知道这参数哪来的。
把cmdline.txt里的内容设置为bootargs，Ok,起不来了。。。仔细对比发现，上面的参数比下面多，并且最关键的，console的参数不一样，把下面的改成console=ttyAMA0,115200 ，然后启动，有打印了，但是停住了
[ 4.594079] smsc95xx v2.0.0
[ 4.711357] SMSC LAN8700 usb-001:003:01: attached PHY driver [SMSC LAN8700] (mii_bus:phy_addr=usb-001:003:01, irq=POLL)
[ 4.723345] smsc95xx 1-1.1:1.0 eth0: register 'smsc95xx' at usb-3f980000.usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:43:86:0a
[ 4.941958] random: crng init done
[ 35.806937] cam-dummy-reg: disabling
我决定还是不乱改了。
uboot到内核流程简述
因为我是用了booti来启动的，因此调用的是do_booti，这里的uboot版本如下：
$ head Makefile
# SPDX-License-Identifier: GPL-2.0+
VERSION = 2024
PATCHLEVEL = 01
SUBLEVEL =
EXTRAVERSION = -rc5
NAME =
我这里最终调的可能是do_bootm_linux，这个函数的实现不多:
/* Main Entry point for arm bootm implementation
*
* Modeled after the powerpc implementation
* DIFFERENCE: Instead of calling prep and go at the end
* they are called if subcommand is equal 0.
*/
int do_bootm_linux(int flag, int argc, char *const argv[],
struct bootm_headers *images)
{
/* No need for those on ARM */
if (flag & BOOTM_STATE_OS_BD_T || flag & BOOTM_STATE_OS_CMDLINE)
return -1;
if (flag & BOOTM_STATE_OS_PREP) {
boot_prep_linux(images);
return 0;
}
if (flag & (BOOTM_STATE_OS_GO | BOOTM_STATE_OS_FAKE_GO)) {
boot_jump_linux(images, flag);
return 0;
}
boot_prep_linux(images);
boot_jump_linux(images, flag);
return 0;
}
看前面do_booi里面的flag，最终应该是直接调了boot_prep_linux，然后返回0.再往后，推断函数结构，应该是调了image_setup_linux，然后再调board_prep_linux。这里不继续分析了，可以参考《U-Boot 学习》。各种处理后，调用boot_selected_os，里面调用了boot_jump_linux，
/* Subcommand: GO */
static void boot_jump_linux(struct bootm_headers *images, int flag)
{
#ifdef CONFIG_ARM64
void (*kernel_entry)(void *fdt_addr, void *res0, void *res1,
void *res2);
int fake = (flag & BOOTM_STATE_OS_FAKE_GO);
kernel_entry = (void (*)(void *fdt_addr, void *res0, void *res1,
void *res2))images->ep;
debug("## Transferring control to Linux (at address %lx)...\n",
(ulong) kernel_entry);
bootstage_mark(BOOTSTAGE_ID_RUN_OS);
announce_and_cleanup(fake);
if (!fake) {
...
do_nonsec_virt_switch();
update_os_arch_secondary_cores(images->os.arch);
...
if ((IH_ARCH_DEFAULT == IH_ARCH_ARM64) &&
(images->os.arch == IH_ARCH_ARM))
armv8_switch_to_el2(0, (u64)gd->bd->bi_arch_number,
(u64)images->ft_addr, 0,
(u64)images->ep,
ES_TO_AARCH32);
else
armv8_switch_to_el2((u64)images->ft_addr, 0, 0, 0,
images->ep,
ES_TO_AARCH64);
}
...
}
其中，images->ep就是内核的入口点:
/*
* armv8_switch_to_el2() - switch from EL3 to EL2 for ARMv8
*
* @args: For loading 64-bit OS, fdt address.
* For loading 32-bit OS, zero.
* @mach_nr: For loading 64-bit OS, zero.
* For loading 32-bit OS, machine nr
* @fdt_addr: For loading 64-bit OS, zero.
* For loading 32-bit OS, fdt address.
* @arg4: Input argument.
* @entry_point: kernel entry point
* @es_flag: execution state flag, ES_TO_AARCH64 or ES_TO_AARCH32
*/
void __noreturn armv8_switch_to_el2(u64 args, u64 mach_nr, u64 fdt_addr,
u64 arg4, u64 entry_point, u64 es_flag);
实现在 arch/arm/cpu/armv8/transition.S，看汇编代码：
.pushsection .text.armv8_switch_to_el2, "ax"
ENTRY(armv8_switch_to_el2)
switch_el x6, 1f, 0f, 0f
0:
cmp x5, #ES_TO_AARCH64
b.eq 2f//如果arch type为arm64，则跳转到label 2处
/*
* When loading 32-bit kernel, it will jump
* to secure firmware again, and never return.
*/
bl armv8_el2_to_aarch32
2:
/*
* x4 is kernel entry point or switch_to_el1
* if CONFIG_ARMV8_SWITCH_TO_EL1 is defined.
* When running in EL2 now, jump to the
* address saved in x4.
*/
br x4//跳转到kernel entry处
1: armv8_switch_to_el2_m x4, x5, x6
ENDPROC(armv8_switch_to_el2)
.popsection
在这里，就进入了内核的入口点
内核入口
内核入口点:
在linux5.10的代码中，流程大概如下:primary_entry->__primary_switch->__primary_switched->start_kernel.
/*
* The following fragment of code is executed with the MMU enabled.
*
* x0 = __PHYS_OFFSET
*/
SYM_FUNC_START_LOCAL(__primary_switched)
...
b start_kernel
SYM_FUNC_END(__primary_switched)
关于start_kernel，参考《Linux内核4.14版本：ARM64的内核启动过程（二）——start_kernel》。
这个start_kernel在init/main.c里面。内核居然也有main.c，长知识了。
start_kernel最终调用了reset_init，这里相比之前的流程多了一层调用:
实现如下。
noinline void __ref rest_init(void)
{
struct task_struct *tsk;
int pid;
rcu_scheduler_starting();
/*
* We need to spawn init first so that it obtains pid 1, however
* the init task will end up wanting to create kthreads, which, if
* we schedule it before we create kthreadd, will OOPS.
*/
pid = kernel_thread(kernel_init, NULL, CLONE_FS);
/*
* Pin init on the boot CPU. Task migration is not properly working
* until sched_init_smp() has been run. It will set the allowed
* CPUs for init to the non isolated CPUs.
*/
rcu_read_lock();
tsk = find_task_by_pid_ns(pid, &init_pid_ns);
set_cpus_allowed_ptr(tsk, cpumask_of(smp_processor_id()));
rcu_read_unlock();
numa_default_policy();
pid = kernel_thread(kthreadd, NULL, CLONE_FS | CLONE_FILES);
rcu_read_lock();
kthreadd_task = find_task_by_pid_ns(pid, &init_pid_ns);
rcu_read_unlock();
/*
* Enable might_sleep() and smp_processor_id() checks.
* They cannot be enabled earlier because with CONFIG_PREEMPTION=y
* kernel_thread() would trigger might_sleep() splats. With
* CONFIG_PREEMPT_VOLUNTARY=y the init task might have scheduled
* already, but it's stuck on the kthreadd_done completion.
*/
system_state = SYSTEM_SCHEDULING;
complete(&kthreadd_done);
/*
* The boot idle thread must execute schedule()
* at least once to get things moving:
*/
schedule_preempt_disabled();
/* Call into cpu_idle with preempt disabled */
cpu_startup_entry(CPUHP_ONLINE);
}
引用下解释:
rcu_scheduler_starting();
通过调用函数rcu_scheduler_starting，来启动 RCU 锁调度器。pid = kernel_thread(kernel_init, NULL, CLONE_FS);pid = kernel_thread(kernel_init, NULL, CLONE_FS);
调用函数 kernel_thread 创建 kernel_init 线程，也就是大名鼎鼎的 init 内核进程。init 进程的 PID 为 1。 init 进程一开始是内核进程(也就是运行在内核态)，后面 init 进程会在根文件系统中查找名为“init”这个程序，这个“init”程序处于用户态，通过运行这个“init”程序， init 进程就会实现从内核态到用户态的转变。pid = kernel_thread(kthreadd, NULL, CLONE_FS | CLONE_FILES);
调用函数 kernel_thread 创建 kthreadd 内核进程，此内核进程的 PID 为 2。kthreadd进程负责所有内核进程的调度和管理。cpu_startup_entry(CPUHP_ONLINE);
调用函数cpu_startup_entry 来进入 idle 进程， cpu_startup_entry 会调用cpu_idle_loop， cpu_idle_loop 是个 while 循环，也就是 idle 进程代码。 idle 进程的 PID 为 0， idle进程叫做空闲进程，如果学过 FreeRTOS 或者 UCOS 的话应该听说过空闲任务。 idle 空闲进程就和空闲任务一样，当 CPU 没有事情做的时候就在 idle 空闲进程里面“瞎逛游”，反正就是给CPU 找点事做。当其他进程要工作的时候就会抢占 idle 进程，从而夺取 CPU 使用权。其实大家应该可以看到 idle 进程并没有使用 kernel_thread 或者 fork 函数来创建，因为它是有主进程演变而来的。
————————————————
版权声明：本文为CSDN博主「风雨兼程8023」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/yangguoyu8023/article/details/121452085
看最后调用的cpu_startup_entry:
void cpu_startup_entry(enum cpuhp_state state)
{
arch_cpu_idle_prepare();
cpuhp_online_idle(state);
while (1)
do_idle();
}
内核也有死循环whiel(1);泪目。
内核初始化
继续看kernel_init:
static int __ref kernel_init(void *unused)
{
int ret;
kernel_init_freeable();
...
if (ramdisk_execute_command) {//static char *ramdisk_execute_command = "/init";
ret = run_init_process(ramdisk_execute_command);
if (!ret)
return 0;
pr_err("Failed to execute %s (error %d)\n",
ramdisk_execute_command, ret);
}
/*
* We try each of these until one succeeds.
*
* The Bourne shell can be used instead of init if we are
* trying to recover a really broken machine.
*/
//execute_command 的值是通过uboot 传递，在 bootargs 中使用“init=xxxx”就可以了，比如“init=/linuxrc”表示根文件系统中的 linuxrc 就是要执行的用户空间 init 程序。
if (execute_command) {
ret = run_init_process(execute_command);
if (!ret)
return 0;
panic("Requested init %s failed (error %d).",
execute_command, ret);
}
if (CONFIG_DEFAULT_INIT[0] != '\0') {
ret = run_init_process(CONFIG_DEFAULT_INIT);
if (ret)
pr_err("Default init %s failed (error %d)\n",
CONFIG_DEFAULT_INIT, ret);
else
return 0;
}
//想尽办法去运行init
if (!try_to_run_init_process("/sbin/init") ||
!try_to_run_init_process("/etc/init") ||
!try_to_run_init_process("/bin/init") ||
!try_to_run_init_process("/bin/sh"))
return 0;
panic("No working init found. Try passing init= option to kernel. "
"See Linux Documentation/admin-guide/init.rst for guidance.");
}
这里开头调用的kernel_init_freeable会去调用do_initcalls初始化子系统。
do_initcalls()这里就是驱动程序员需要关心的步骤，其中按照各个内核模块初始化函数所自定义的启动级别（1～7），按顺序调用器初始化函数。对于同一级别的初始化函数，安装编译是链接的顺序调用，也就是和内核Makefile的编写有关。在编写内核模块的时候需要知道这方面的知识，比如你编写的模块使用的是I2C的API，那你的模块的初始化函数的级别必须低于I2C子系统初始化函数的级别（也就是级别数（1~7）要大于I2C子系统）。如果编写的模块必须和依赖的模块在同一级，那就必须注意内核Makefile的修改了。
init进程启动和身份转换
static int try_to_run_init_process(const char *init_filename)
{
int ret;
ret = run_init_process(init_filename);
if (ret && ret != -ENOENT) {
pr_err("Starting init: %s exists but couldn't execute it (error %d)\n",
init_filename, ret);
}
return ret;
}
...
static int run_init_process(const char *init_filename)
{
const char *const *p;
argv_init[0] = init_filename;
pr_info("Run %s as init process\n", init_filename);
pr_debug(" with arguments:\n");
for (p = argv_init; *p; p++)
pr_debug(" %s\n", *p);
pr_debug(" with environment:\n");
for (p = envp_init; *p; p++)
pr_debug(" %s\n", *p);
return kernel_execve(init_filename, argv_init, envp_init);
}
参考资料:
树莓派4B U-boot移植并加载裸机程序
在树莓派3b上运行uboot
树莓派4 嵌入式Linux开发过程详解
使用U-Boot让树莓派从U盘启动
RPI 3 booting from U-boot
uboot的烧写及使用
U-boot启动流程U-boot启动流程
Linux、树莓派启动过程
内核启动参数详解
U-Boot 学习
Linux内核4.14版本：ARM64的内核启动过程（二）——start_kernel