openwrt作为一个基于linux开发的比较完善的嵌入式系统，可以快速移植到各种平台上。初次下载开源代码后，简单浏览后很是诧异，居然没看到uboot和kernel部分的代码，甚至没看到任何模块的代码，最多只是些patch和配置文件。
按照文档编译后，发现多了些工程目录，进而发现了很多源码，猜测到大概是Makefile或feed脚本在编译时在线下载的代码。为了后来者，初次入门openwrt少踩坑，完成此章，内容多为我入门的困惑与解答。
1. uboot和kernel在哪里？
事实上，拿到marvell release代码后会看到：
$ tree -L 1 marvell/
marvell/
├── fastpath
├── fota
├── linux
├── lte-telephony
├── obm
├── services
├── swd
├── uboot
└── webui
这些明显是marvell自己定制过的一些代码包，针对这个平台算是找到了正确的代码位置，但我们并不了解这个过程，并且开源代码部分是没有这种结构的，所以我们要继续探究下去，提出问题：
- 官方release的openwrt没有定制的情况下uboot和kernel在哪?
- marvell这种定制是如何实现的？
在目录结构和功能不了解的情况下会感觉无从下手而四处乱摸，通过上一章了解了目录结构，很自然的就会想到package目录，进去看看：
$ cd package
$ tree -L 1 boot
boot/
├── uboot-mvebu
├── uboot-mxs
├── uboot-omap
├── uboot-oxnas
├── ...
└── yamonenv
最终发现，package/boot/uboot-mmp/ :
$ cat package/boot/uboot-mmp/Makefile
include $(TOPDIR)/rules.mk
PKG_NAME:=u-boot
PKG_VERSION:=2014
PKG_RELEASE:=1
USE_SOURCE_DIR:=$(MRVLDIR)/uboot
PKG_SOURCE_SUBDIR:=$(PKG_NAME)-$(PKG_VERSION)
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)-$(BUILD_VARIANT)/$(PKG_NAME)-$(PKG_VERSION)
include $(INCLUDE_DIR)/package.mk
这里面看到了'MRVLDIR',查询后发现MRVLDIR:=$(TOPDIR)/marvell,
$ ls -l build_dir/target-arm_cortex-a7+neon-vfpv4_uClibc-1.0.25_eabi/u-boot-nezha3_dkb/
total 0
lrwxrwxrwx 1 tjd tjd 56 May 15 15:56 u-boot-2014 -> /home/tjd/datadisk/marvell/OpenWrt/openwrt/marvell/uboot
确认无疑了！！
package/boot/uboot-mmp/就是marvell的uboot定义的位置，变量USE_SOURCE_DIR指定了uboot使用的源码。
知道了marvell定制uboot源码的方法，再看一下官方release版本的uboot，已Beaglebone black板项目为例：
$ cat package/boot/uboot-omap/Makefile
include $(TOPDIR)/rules.mk
include $(INCLUDE_DIR)/kernel.mk
PKG_VERSION:=2017.01
PKG_RELEASE:=2
PKG_HASH:=6c425175f93a4bcf2ec9faf5658ef279633dbd7856a293d95bd1ff516528ecf2
include $(INCLUDE_DIR)/u-boot.mk
include $(INCLUDE_DIR)/package.mk
define U-Boot/Default
BUILD_TARGET:=omap
UBOOT_IMAGE:=u-boot.img MLO
UENV:=default
endef
define U-Boot/omap4_panda
NAME:=Pandaboard
BUILD_DEVICES:=omap4-panda
endef
define U-Boot/am335x_boneblack
NAME:=TI AM335x BeagleBone Black
BUILD_DEVICES:=am335x-boneblack
endef
...
很遗憾，这里没有USE_SOURCE_DIR指定源码位置，只是指定了使用的uboot的版本号，目录中也没有源码：
package/boot/uboot-omap/
├── files
│ └── uEnv-default.txt
├── Makefile
└── patches
├── 101-disable-thumb-omap3.patch
├── 102-minify-spl.patch
├── 103-disable-fat-write-spl.patch
├── 104-omap3-overo-enable-thumb.patch
└── 105-serial-ns16550-bugfix-ns16550-fifo-not-enabled.patch
2 directories, 7 files
这里保存了针对bbb的uboot相关patch，联想到与硬件相关的内容是以patch存在的，u-boot应该是官方发布的原版代码，上一章提到了dl目录保存了从网络上下载的代码包，编译时会解压源码包到build_dir中，现在看一下：
$ ls build_dir/target-arm_cortex-a8+vfpv3_musl_eabi/u-boot-am335x_boneblack/u-boot-2017.01/
api cmd configs drivers fs Kconfig MAINTAINERS MLO.byteswap README spl tools u-boot.cfg u-boot.lds u-boot.srec
arch common disk dts include lib Makefile net scripts System.map u-boot u-boot.cfg.configs u-boot.map u-boot.sym
board config.mk doc examples Kbuild Licenses MLO post snapshot.commit test u-boot.bin u-boot.img u-boot-nodtb.bin
前面看到package/boot/uboot-omap/中有patch，是不是已经合入了呢？ 对比看看：
$ cat package/boot/uboot-omap/patches/101-disable-thumb-omap3.patch
Index: u-boot-2017.01/include/configs/ti_omap3_common.h
===================================================================
--- u-boot-2017.01.orig/include/configs/ti_omap3_common.h
+++ u-boot-2017.01/include/configs/ti_omap3_common.h
@@ -80,4 +80,9 @@
/* Now bring in the rest of the common code. */
#include
+/* beagleboard doesnt boot with thumb */
+#ifdef CONFIG_SYS_THUMB_BUILD
+#undef CONFIG_SYS_THUMB_BUILD
+#endif
+
#endif /* __CONFIG_TI_OMAP3_COMMON_H__ */
$ diff dl/u-boot-2017.01_unpack/include/configs/ti_omap3_common.h build_dir/target-arm_cortex-a8+vfpv3_musl_eabi/u-boot-am335x_boneblack/u-boot-2017.01/include/configs/ti_omap3_common.h
82a83,87
> /* beagleboard doesnt boot with thumb */
> #ifdef CONFIG_SYS_THUMB_BUILD
> #undef CONFIG_SYS_THUMB_BUILD
> #endif
看来patch是已经打上去了。
- 官方的uboot先解压到
build_dir中，再打上package/boot/uboot-xxx/patches中的patch，再编译。- marvell通过变量
USE_SOURCE_DIR指定本地代码库，连接到build_dir中，然后编译。
再看看kernel应该是类似的：区别是kernel不在package中而是target中，引用外部库的变量不是USE_SOURCE_DIR而是CONFIG_EXTERNAL_KERNEL_TREE
$ cat target/linux/mmp/Makefile
include $(TOPDIR)/rules.mk
ARCH:=arm
BOARD:=mmp
BOARDNAME:=Marvell MMP
SUBTARGETS=pxa1826
FEATURES:=squashfs jffs2_nand nand ubifs
LINUX_VERSION:=3.10.33
CONFIG_EXTERNAL_KERNEL_TREE:=$(MRVLDIR)/linux
define Target/Description
Build firmware images for Marvell MMP SoC
endef
include $(INCLUDE_DIR)/target.mk
2. 如何编译自己的代码包？
用户态模块
新建getevent包最终如下：
$ tree package/xxx/
xxx/
└── getevent
├── Makefile
└── src
├── getevent.c
├── getevent.h
└── Makefile
配置顶层Makefile
$ vi package/xxx/getevent/Makefile
include $(TOPDIR)/rules.mk
PKG_NAME:=getevent
PKG_RELEASE:=1
# 定义编译软件包的路径
PKG_BUILD_DIR := $(BUILD_DIR)/$(PKG_NAME)
include $(INCLUDE_DIR)/package.mk
# 软件包描述，可以在make menuconfig中体现出来
define Package/getevent
SECTION:=Xxxx
CATEGORY:=xxx ddd
TITLE:=linux inputevent test tool
endef
# 本包安裝的配置文件，一行一个
define Package/dnsmasq/conffiles
#/etc/config/dhcp
#/etc/dnsmasq.conf
endef
# make package/xxx/getevent/prepare时被执行
define Build/Prepare
mkdir -p $(PKG_BUILD_DIR)
$(CP) ./src/* $(PKG_BUILD_DIR)/
endef
# 软件包的介绍信息
define Package/helloworld/description
helloworld,first self-made.
endef
# 对执行./configure时的特殊处理
define Build/Configure
endef
# 执行make或make package/xxx/getevent/compile时处理
define Build/Compile
$(MAKE) -C $(PKG_BUILD_DIR) \
CC="$(TARGET_CC)" \
CFLAGS="$(TARGET_CFLAGS) -Wall" \
LDFLAGS="$(TARGET_LDFLAGS)"
endef
# make install时处理
# INSTALL_DIR 决定了通过opkg安装ipk的目标目录
define Package/getevent/install
$(INSTALL_DIR) $(1)/usr/sbin
$(INSTALL_BIN) $(PKG_BUILD_DIR)/getevent $(1)/usr/sbin/
endef
$(eval $(call BuildPackage,getevent))
编辑src目录下的Makefile
CC = gcc
CFLAGS = -Wall
OBJS = getevent.o
all: getevent
%.o: %.c
$(CC) $(CFLAGS) -c -o $@ $<
getevent: $(OBJS)
$(CC) -o $@ $(OBJS)
clean:
rm -f getevent *.o
注意，这里一个代码包中有两个Makefile
下一步时make menuconfig，选择CONFIG_PACKAGE_getevent=m或CONFIG_PACKAGE_getevent=y否则编译不出东西，会报：
WARNING: skipping getevent -- package not selected
config配置成m时会编译成ipk文件，可以按需安装；配置成y时，是buildin，内置到release软件中，无需安装。
内核模块
最终包如下：
$ tree khello/
khello/
├── Makefile
└── src
├── khello.c
└── Makefile
1 directory, 3 files
配置顶层Makefile
$ cat khello/Makefile
#
# Copyright (C) 2008 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
include $(TOPDIR)/rules.mk
include $(INCLUDE_DIR)/kernel.mk
PKG_NAME:=khello
PKG_RELEASE:=2
include $(INCLUDE_DIR)/package.mk
define KernelPackage/khello
# 指定menuconfig的路径
SUBMENU:=Other modules
TITLE:=kernel Helloworld test
# 依赖的模块
DEPENDS:=
# 目标文件
FILES:=$(PKG_BUILD_DIR)/khello.ko
KCONFIG:=
endef
define KernelPackage/khello/description
Kernel module for helloword example.
endef
EXTRA_KCONFIG:= \
CONFIG_HELLO=m
EXTRA_CFLAGS:= \
$(patsubst CONFIG_%, -DCONFIG_%=1, $(patsubst %=m,%,$(filter %=m,$(EXTRA_KCONFIG)))) \
$(patsubst CONFIG_%, -DCONFIG_%=1, $(patsubst %=y,%,$(filter %=y,$(EXTRA_KCONFIG)))) \
MAKE_OPTS:= \
ARCH="$(LINUX_KARCH)" \
CROSS_COMPILE="$(TARGET_CROSS)" \
SUBDIRS="$(PKG_BUILD_DIR)" \
EXTRA_CFLAGS="$(EXTRA_CFLAGS)" \
$(EXTRA_KCONFIG)
define Build/Prepare
mkdir -p $(PKG_BUILD_DIR)
$(CP) ./src/* $(PKG_BUILD_DIR)/
endef
define Build/Compile
$(MAKE) -C "$(LINUX_DIR)" \
$(MAKE_OPTS) \
modules
endef
$(eval $(call KernelPackage,khello))
编辑src目录下的Makefile
$ cat package/revoview/khello/src/Makefile
obj-$(CONFIG_HELLO) += khello.o
编辑khello.c
$ cat package/revoview/khello/src/khello.c
#include
#include
#include
static int __init hello_init(void)
{
printk("Hello Kernel\n");
return 0;
}
module_init(hello_init);
static void __exit hell_exit(void)
{
printk("Bye Kernel");
}
module_exit(hell_exit);
MODULE_LICENSE("GPL");
3. 模块如何单独编译？
首先有个顶层Makefile，如上面的两个案例
如何模块需要区别于默认编译选项，Makefile中可以包含下面几个重要元素：
-
Build/Preparemake xxx/xxx/prepare在编译前解压或在build_dir/准备代码时调用，如：
define Build/Prepare mkdir -p $(PKG_BUILD_DIR) $(CP) ./src/* $(PKG_BUILD_DIR)/ endef -
Build/Configuremake xxx/xxx/configure在编译前执行./configure 做特殊处理，如：
define Build/Configure $(CP) $(SCRIPT_DIR)/config.guess $(SCRIPT_DIR)/config.sub $(PKG_BUILD_DIR)/support/ $(call Build/Configure/Default, \ --enable-shared \ --enable-static \ --without-curses \ ) endef也可以这样用：
define Build/Configure $(call Build/Configure/Default) $(if $(CONFIG_PCAP_HAS_USB),,$(SED) '/^#define PCAP_SUPPORT_USB/D' $(PKG_BUILD_DIR)/config.h) $(if $(CONFIG_PCAP_HAS_USB),,$(SED) 's/pcap-usb-linux.c *//' $(PKG_BUILD_DIR)/Makefile) $(if $(CONFIG_PCAP_HAS_BT),,$(SED) '/^#define PCAP_SUPPORT_BT/D' $(PKG_BUILD_DIR)/config.h) $(if $(CONFIG_PCAP_HAS_BT),,$(SED) 's/pcap-bt-linux.c *//' $(PKG_BUILD_DIR)/Makefile) endef需要注意的是：
$(call Build/Configure/Default)是必需的。 -
Build/Compilemake xxx/xxx/compilemake时执行，可以增加编译参数
define Build/Compile $(TARGET_CONFIGURE_OPTS) \ CFLAGS="$(TARGET_CFLAGS) -I$(STAGING_DIR)/usr/include -I$(PKG_BUILD_DIR)" \ LDFLAGS="$(TARGET_LDFLAGS)" \ $(MAKE) -C $(PKG_BUILD_DIR) endef -
Build/Install和Build/InstallDevmake xxx/xxx/install在编译完成，安装时执行的特殊处理,如：
define Build/InstallDev $(INSTALL_DIR) $(1)/usr/include $(CP) $(PKG_INSTALL_DIR)/usr/include/lua{,lib,conf}.h $(1)/usr/include/ $(CP) $(PKG_INSTALL_DIR)/usr/include/lauxlib.h $(1)/usr/include/ $(CP) $(PKG_INSTALL_DIR)/usr/include/lnum_config.h $(1)/usr/include/ $(INSTALL_DIR) $(1)/usr/lib $(CP) $(PKG_INSTALL_DIR)/usr/lib/liblua.{a,so*} $(1)/usr/lib/ ln -sf liblua.so.$(PKG_VERSION) $(1)/usr/lib/liblualib.so $(INSTALL_DIR) $(1)/usr/lib/pkgconfig $(CP) $(PKG_BUILD_DIR)/etc/lua.pc $(1)/usr/lib/pkgconfig/ endef
4. ipk是如何编译出来的
openwrt的编译要区分是不是buildin， 对于buildin是指，选定的包编译时就就安装到rootfs中，否则，就是编译成ipk文件，通过opkg命令安装。
区分是否buildin通过menuconfig配置，最终体现在.config中。
< CONFIG_PACKAGE_getevent=y
---
> CONFIG_PACKAGE_getevent=m
`=m` 代表编译成ipk, 目标：`bin/pxa1826/packages/base/getevent_1_pxa1826.ipk`
`=y` 代表buildin，目标：`staging_dir/target-arm_cortex-a7+neon-vfpv4_uClibc-1.0.25_eabi/root-mmp/usr/sbin/getevent`
5. Patches是如何自动打上去的？
通过make -n提取的make V=s package/utils/busybox/{clean,prepare}的过程如下
. /source/marvell/include/shell.sh; bzcat /source/marvell/dl/u-boot-2017.01.tar.bz2 | tar -C /source/marvell/build_dir/target-arm_cortex-a8+vfpv3_musl_eabi/u-boot-am335x_boneblack/u-boot-2017.01/.. -xf -
[ ! -d ./src/ ] || cp -fpR ./src/. /source/marvell/build_dir/target-arm_cortex-a8+vfpv3_musl_eabi/u-boot-am335x_boneblack/u-boot-2017.01
if [ -d "./patches" ] && [ "$(ls ./patches | wc -l)" -gt 0 ]; then export PATCH="patch"; if [ -s "./patches/series" ]; then sed -e s,\\\#.*,, ./patches/series | grep -E \[a-zA-Z0-9\] | xargs -n1 /source/marvell/scripts/patch-kernel.sh "/source/marvell/build_dir/target-arm_cortex-a8+vfpv3_musl_eabi/u-boot-am335x_boneblack/u-boot-2017.01" "./patches"; else /source/marvell/scripts/patch-kernel.sh "/source/marvell/build_dir/target-arm_cortex-a8+vfpv3_musl_eabi/u-boot-am335x_boneblack/u-boot-2017.01" "./patches"; fi; fi
-
- 解压
bzcat /source/marvell/dl/u-boot-2017.01.tar.bz2 | tar -C /source/marvell/build_dir/target-arm_cortex-a8+vfpv3_musl_eabi/u-boot-am335x_boneblack/u-boot-2017.01/.. -xf -
-
- 打patch
/source/marvell/scripts/patch-kernel.sh "/source/marvell/build_dir/target-arm_cortex-a8+vfpv3_musl_eabi/u-boot-am335x_boneblack/u-boot-2017.01“ "./patches"
可以在package/boot/uboot-omap目录下运行patch-kernel.sh测试。
6. 如何创建自己的patch？
-
安装quilt
sudo apt-get install quilt -
准备quilt的配置文件
cat > ~/.quiltrc < 创建第一个patch
$ make package/utils/fbtest/{clean,prepare} V=s QUILT=1
$ cd build_dir/target-*/fbtest
$ quilt new patches/0001-test.patch
$ quilt edit fbtest.c
查看修改的内容
$ quilt diff
刷新0001-test.patch
$ quilt refresh
把patch挪到package目录
$ cp build_dir/target-*/fbtest/patches/*.patch package/utils/fbtest/patches/
编译看看patch是否生效
$ make package/utils/fbtest/{clean,prepare} V=s
- 增加 patch
和上面步骤一样，只要注意patch的名字序号递增。
$ make package/utils/fbtest/{clean,prepare} V=s QUILT=1
$ cd build_dir/target-*/fbtest
$ quilt new patches/0002-test2.patch
$ quilt edit Makefile
查看修改的内容
$ quilt diff
刷新0002-test.patch
$ quilt refresh
把patch挪到package目录
$ cp build_dir/target-*/fbtest/patches/*.patch package/utils/fbtest/patches/
编译看看patch是否生效
$ make package/utils/fbtest/{clean,prepare} V=s
-
修改patch
$ make package/utils/fbtest/{clean,prepare} V=s QUILT=1 $ cd build_dir/target-*/fbtest $ quilt push patches/0002-test2.patch $ quilt edit Makefile 查看修改的内容 $ quilt diff 刷新0002-test.patch $ quilt refresh 把patch挪到package目录 $ cp build_dir/target-*/fbtest/patches/*.patch package/utils/fbtest/patches/ 编译看看patch是否生效 $ make package/utils/fbtest/{clean,prepare} V=s
生成patch特别要注意的是：
- 各个patch是不能有互相依赖关系的
- 如果有依赖关系需要做特殊处理，比如
0002依赖0001，需要将0001先导入（但不能用quilt导入，可以是patch -p1之类的），然后再用quilt生成0002- 最后一步
cp patch时不要将series文件也拷贝到package包中。
7. rootfs源头在哪？
有两个位置：
- package/base-file
- target/linux/xxx/base-files