Yocto:将kernel, u-boot, atf单独建库编译摒弃掉打patch方式

用过yocto都知道，如果要修改kernel源代码，需要生成patch, 并修改.bb文件让其编译的时候打上patch。但是这样其实个人觉得不是很方便。按之前安卓开发都是修改好代码后直接将其提交到git仓库。那么要如何将BSP开发经常用到几个库独立出来，编译的时候直接从指定的代码库里面去编译呢？下面就讲讲我摸索的办法。

工程默认的代码架构：

.
├── nxp-setup-alb.sh
└── sources

期望修改成如下：

.
├── nxp-setup-alb.sh
├── s32-atf
├── s32-kernel
├── s32-uboot
└── sources

让其编译kernel,u-boot, atf是源码从工程的根目录去获取。下面以kernel为例子进行讲解。

先看一下kernel有哪些task，其中有些task需要我们重新定义，让其按我们的想法走。

someone@ubt:/work/projects/b33_2/build_s32g274ardb2$ bitbake virtual/kernel -c listtasks
Loading cache: 100% |###########################################################################################################################| Time: 0:00:00
Loaded 4897 entries from dependency cache.
Parsing recipes: 100% |#########################################################################################################################| Time: 0:00:00
Parsing of 3299 .bb files complete (3298 cached, 1 parsed). 4895 targets, 289 skipped, 6 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.48.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-fsl-linux"
MACHINE              = "s32g274ardb2"
DISTRO               = "fsl-auto"
DISTRO_VERSION       = "33.0"
TUNE_FEATURES        = "aarch64 armv8a crc cortexa53 crypto"
TARGET_FPU           = ""
meta
meta-poky
meta-yocto-bsp       = "HEAD:6a751048e50c00261d99c2d8d69534f7a8da38a9"
meta-oe
meta-multimedia
meta-python          = "HEAD:cd80c236199d885c7e89ff184b28a29241bbbd95"
meta-python2         = "HEAD:a964c78e39a575bcb2ca15f461e6689ce61a07cd"
meta-networking
meta-gnome
meta-filesystems
meta-webserver
meta-perl
meta-xfce            = "HEAD:cd80c236199d885c7e89ff184b28a29241bbbd95"
meta-virtualization  = "HEAD:c9397c38ada45030a40cf2ed690ff4edd6fa3e97"
meta-optee           = "HEAD:608874173d66d40cdd41d38962576d539220b2cf"
meta-security        = "HEAD:cb1ada6f60e163b91c99c8c28c9e7b78e619e094"
meta-freescale       = "HEAD:211af419458cbb3c3af506320fff061b1711974f"
meta-alb             = "HEAD:12dda07b693c9d1ec68a69e3a12fd5ccef0072a8"

Initialising tasks: 100% |######################################################################################################################| Time: 0:00:01
Sstate summary: Wanted 0 Found 0 Missed 0 Current 0 (0% match, 0% complete)
NOTE: No setscene tasks
NOTE: Executing Tasks
do_build                              Default task for a recipe - depends on all other normal tasks required to 'build' a recipe
do_bundle_initramfs                   Combines an initial ramdisk image and kernel together to form a single image
do_checkuri                           Validates the SRC_URI value
do_clean                              Removes all output files for a target
do_cleanall                           Removes all output files, shared state cache, and downloaded source files for a target
do_cleansstate                        Removes all output files and shared state cache for a target
do_compile                            Compiles the source in the compilation directory
do_compile_kernelmodules              Compiles loadable modules for the Linux kernel
do_configure                          Configures the source by enabling and disabling any build-time and configuration options for the software being built
do_deploy                             Writes deployable output files to the deploy directory
do_deploy_setscene                    Writes deployable output files to the deploy directory (setscene version)
do_deploy_source_date_epoch
do_deploy_source_date_epoch_setscene   (setscene version)
do_devpyshell                         Starts an interactive Python shell for development/debugging
do_devshell                           Starts a shell with the environment set up for development/debugging
do_diffconfig                         Compares the old and new config files after running do_menuconfig for the kernel
do_fetch                              Fetches the source code
do_install                            Copies files from the compilation directory to a holding area
do_kernel_link_images                 Creates a symbolic link in arch/$arch/boot for vmlinux and vmlinuz kernel images
do_listtasks                          Lists all defined tasks for a target
do_menuconfig                         Runs 'make menuconfig' for the kernel
do_merge_delta_config
do_package                            Analyzes the content of the holding area and splits it into subsets based on available packages and files
do_package_qa                         Runs QA checks on packaged files
do_package_qa_setscene                Runs QA checks on packaged files (setscene version)
do_package_setscene                   Analyzes the content of the holding area and splits it into subsets based on available packages and files (setscene version)
do_package_write_rpm                  Creates the actual RPM packages and places them in the Package Feed area
do_package_write_rpm_setscene         Creates the actual RPM packages and places them in the Package Feed area (setscene version)
do_packagedata                        Creates package metadata used by the build system to generate the final packages
do_packagedata_setscene               Creates package metadata used by the build system to generate the final packages (setscene version)
do_patch                              Locates patch files and applies them to the source code
do_populate_lic                       Writes license information for the recipe that is collected later when the image is constructed
do_populate_lic_setscene              Writes license information for the recipe that is collected later when the image is constructed (setscene version)
do_populate_sysroot                   Copies a subset of files installed by do_install into the sysroot in order to make them available to other recipes
do_populate_sysroot_setscene          Copies a subset of files installed by do_install into the sysroot in order to make them available to other recipes (setscene version)
do_preconfigure
do_prepare_recipe_sysroot
do_savedefconfig                      Creates a minimal Linux kernel configuration file
do_shared_workdir
do_shared_workdir_setscene             (setscene version)
do_sizecheck                          Checks the size of the kernel image against KERNEL_IMAGE_MAXSIZE (if set)
do_strip                              Strips unneeded sections out of the Linux kernel image
do_symlink_kernsrc
do_unpack                             Unpacks the source code into a working directory
NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun and all succeeded.

方便讲述这里也贴一下kernel对应bb文件部分内容

DESCRIPTION = "Linux kernel for S32 platforms"
SECTION = "kernel"
LICENSE = "GPLv2"

inherit kernel

inherit fsl-kernel-localversion

SCMVERSION ?= "y"
LOCALVERSION = ""
DELTA_KERNEL_DEFCONFIG ?= ""

URL ?= "git://source.codeaurora.org/external/autobsps32/linux;protocol=https"
BRANCH ??= "${RELEASE_BASE}-${PV}-rt"
SRC_URI = "${URL};branch=${BRANCH}"

DEPENDS_append = " libgcc dtc-native"

KERNEL_CC_append = " ${TOOLCHAIN_OPTIONS}"
KERNEL_LD_append = " ${TOOLCHAIN_OPTIONS}"

S = "${WORKDIR}/git"
MAJ_VER =  "${@oe.utils.trim_version("${PV}", 2)}"

默认情况下编译kernel会做如下task

do_fetch:  从git://source.codeaurora.org/external/autobsps32/linux对应branch分支，以clone bare的方式指定的到downloads/git2/source.codeaurora.org.external.autobsps32.linux目录下

do_unpack: 从downloads/git2/source.codeaurora.org.external.autobsps32.linux git clone到指定的文件夹build_s32g274ardb2/tmp/work/s32g274ardb2-fsl-linux/linux-s32/5.10.109-r0/git

do_symlink_kernsrc: 做了两件事，先将build_s32g274ardb2/tmp/work/s32g274ardb2-fsl-linux/linux-s32/5.10.109-r0/git文件夹move到build_s32g274ardb2/tmp/work-shared/s32g274ardb2/kernel-source。 再将build_s32g274ardb2/tmp/work-shared/s32g274ardb2/kernel-source链接到build_s32g274ardb2/tmp/work/s32g274ardb2-fsl-linux/linux-s32/5.10.109-r0/git

someone@ubt:/work/projects/nxp_b33/build_s32g274ardb2$ ls tmp/work/s32g274ardb2-fsl-linux/linux-s32/5.10.109-r0/ -al
total 36
drwxrwxr-x 7 someone someone 4096 8月  29 11:13 .
drwxrwxr-x 3 someone someone 4096 8月  29 11:13 ..
drwxr-xr-x 2 someone someone 4096 8月  29 11:13 build
-rw-rw-r-- 1 someone someone 35 7月  29 11:31 gcc75compat.config
lrwxrwxrwx 1 someone someone 84 8月  29 11:13 git -> /work/projects/nxp_b33/build_s32g274ardb2/tmp/work-shared/s32g274ardb2/kernel-source
drwxrwxr-x 2 someone someone 4096 8月  29 11:13 recipe-sysroot
drwxrwxr-x 3 someone someone 4096 8月  29 11:13 recipe-sysroot-native
drwxr-xr-x 2 someone someone 4096 8月  29 11:13 source-date-epoch
drwxrwxr-x 2 someone someone 4096 8月  29 11:13 temp

根据上面的task步骤, 修改成让其kernel源码用项目根目录的s32-kernel。具体的修改如下

--- a/recipes-kernel/linux/linux-s32.inc
+++ b/recipes-kernel/linux/linux-s32.inc
@@ -10,9 +10,9 @@ SCMVERSION ?= "y"
 LOCALVERSION = ""
 DELTA_KERNEL_DEFCONFIG ?= ""

-URL ?= "git://source.codeaurora.org/external/autobsps32/linux;protocol=https"
-BRANCH ??= "${RELEASE_BASE}-${PV}-rt"
-SRC_URI = "${URL};branch=${BRANCH}"
+#URL ?= "git://source.codeaurora.org/external/autobsps32/linux;protocol=https"
+#BRANCH ??= "${RELEASE_BASE}-${PV}-rt"
+#SRC_URI = "${URL};branch=${BRANCH}"

 DEPENDS_append = " libgcc dtc-native"

@@ -21,6 +21,26 @@ KERNEL_LD_append = " ${TOOLCHAIN_OPTIONS}"

 S = "${WORKDIR}/git"
 MAJ_VER =  "${@oe.utils.trim_version("${PV}", 2)}"
+PROJECT_DIR = "${@os.path.dirname("${TOPDIR}")}/s32-kernel"
+
+do_fetch(){
+}
+
+do_unpck(){
+}
+
+do_symlink_kernsrc(){
+    rm -rf ${TMPDIR}/work-shared/${MACHINE}/kernel-source
+    ln -sf ${PROJECT_DIR} ${TMPDIR}/work-shared/${MACHINE}/kernel-source
+    rm -rf ${S}
+    ln -sf ${PROJECT_DIR} ${S}
+}
+
+do_clean[postfuncs] += "git_repository_checkout"
+git_repository_checkout(){
+       cd ${PROJECT_DIR}
+       git clean -fd && git checkout -f
+}

修改点说明如下：

1.注释掉git仓库地址，重写task do_fetch使其为空函数。也就是说不让其从网上git clone仓库下来。

2.重新do_unpack函数，使其为空，不用编译系统git clone kernel源代码到build_s32g274ardb2/tmp/work/s32g274ardb2-fsl-linux/linux-s32/5.10.109-r0/git

3.重写do_symlink_kernsrc，主要工作在这里。

1）将build_s32g274ardb2/tmp/work/s32g274ardb2-fsl-linux/linux-s32/5.10.109-r0/git软链接到s32-kernel目录去。

2)  将build_s32g274ardb2/tmp/work-shared/s32g274ardb2/kernel-source 也软链接到 s32-kernel目录去。链接好后如下

someone@ubt:/work/projects/b33_2/build_s32g274ardb2$ ls -al tmp/work/s32g274ardb2-fsl-linux/linux-s32/5.10.109-r0/ -al
total 248
drwxrwxr-x 26 someone someone   4096 8月  29 12:25 .
drwxrwxr-x  3 someone someone   4096 8月  26 17:56 ..
drwxr-xr-x 20 someone someone   4096 8月  29 12:25 build
-rw-rw-r--  1 someone someone 116205 8月  29 12:23 defconfig
drwxr-xr-x  2 someone someone   4096 8月  29 12:25 deploy-linux-s32
drwxr-xr-x  3 someone someone   4096 8月  29 12:25 deploy-rpms
drwxrwxr-x  2 someone someone   4096 8月  29 12:23 deploy-source-date-epoch
lrwxrwxrwx  1 someone someone     31 8月  29 12:23 git -> /work/projects/b33_2/s32-kernel
drwxr-xr-x  5 someone someone   4096 8月  29 12:25 image
drwxrwxr-x  3 someone someone   4096 8月  29 12:23 license-destdir
-rw-r--r--  1 someone someone   4365 8月  29 12:25 linux-s32.spec
drwxr-xr-x  4 someone someone   4096 8月  29 12:25 package
drwxr-xr-x 11 someone someone   4096 8月  29 12:25 packages-split
drwxr-xr-x  7 someone someone   4096 8月  29 12:25 pkgdata
drwxr-xr-x  7 someone someone   4096 8月  29 12:25 pkgdata-pdata-input
drwxr-xr-x  6 someone someone   4096 8月  29 12:25 pkgdata-sysroot
drwxrwxr-x  2 someone someone   4096 8月  29 12:25 pseudo
drwxrwxr-x  5 someone someone   4096 8月  29 12:25 recipe-sysroot
drwxrwxr-x  8 someone someone   4096 8月  29 12:25 recipe-sysroot-native
drwxrwxr-x  2 someone someone   4096 8月  29 12:23 source-date-epoch
drwxrwxr-x  2 someone someone   4096 8月  26 17:56 sstate-install-deploy
drwxrwxr-x  2 someone someone   4096 8月  26 17:56 sstate-install-deploy_source_date_epoch
drwxrwxr-x  2 someone someone   4096 8月  26 17:56 sstate-install-packagedata
drwxrwxr-x  2 someone someone   4096 8月  26 17:56 sstate-install-package_qa
drwxrwxr-x  2 someone someone   4096 8月  26 17:56 sstate-install-package_write_rpm
drwxrwxr-x  2 someone someone   4096 8月  26 17:56 sstate-install-populate_lic
drwxrwxr-x  2 someone someone   4096 8月  26 17:56 sstate-install-populate_sysroot
drwxr-xr-x  3 someone someone   4096 8月  29 12:25 sysroot-destdir
drwxrwxr-x  2 someone someone  20480 8月  29 12:25 temp
someone@ubt:/work/projects/b33_2/build_s32g274ardb2$ ls tmp/work-shared/s32g274ardb2/kernel-source -al
lrwxrwxrwx 1 someone someone 31 8月  29 12:23 tmp/work-shared/s32g274ardb2/kernel-source -> /work/projects/b33_2/s32-kernel

4. 最后追加写了git_repository_checkout函数，这个函数是相当于do_clean后追加执行的。有点类似do_clean_append函数。但为什么不直接重写do_clean_append函数呢？其实开始的时候我是重写了do_clean_append函数，但是do_clean_append不能直接执行shell命令。究其原因是do_clean_append函数使用python去解析执行的(PS, 应该是和yocto的版本有关，记得几年前刚接触yocto时do_clean_append也是用shell去解释执行的，现在新版本不行了)。而像do_clean是用shell去解析执行的。而我有不想重写do_clean函数，只想在do_clean函数后追加执行我像要执行的命令(cd s32-kernel文件夹中执行git clean -fd && git checkout -f)。后面找了下发现可以用do_clean[postfuncs]加一个自定义函数。

do_clean[postfuncs] += "git_repository_checkout"
git_repository_checkout(){
       cd ${PROJECT_DIR}
       git clean -fd && git checkout -f
}

最后说一点PROJECT_DIR这个是自己定义的。当时为了获取到工程的根目录也是想了大半天。

PROJECT_DIR = "${@os.path.dirname("${TOPDIR}")}/s32-kernel"最后用这个获取到的。

.bb文件其实是用python去解析的。由于知道TOPDIR=/work/projects/b33_2/build_s32g274ardb2

所以这里调用python库函数os.path.dirname("${TOPDIR}")获取到/work/projects/b33_2/的根目录，后面在连接上s32-kernel后就正确指定了PROJECT_DIR=/work/projects/b33_2/s32-kernel

后续修改kernel代码就可以在工程根目录s32-kernel下进行，修改后直接git add , git commit , git push到服务器。修改完成后就直接可以bitbake 编译了。bitbake virtual/kernel -c compile -f

不需要之前繁琐的修改完成kernel代码后需要先生成patch, 再把patch拷贝到sources/meta-alb/recipes-kernel/linux/files/目录下，再修改对应的bb文件。

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

关于重写do_clean_append函数，编译好像解析时就报错了。

+do_clean_append(){
+       cd ${PROJECT_DIR}
+       git clean -fd && git checkout -f
+}
+

t.DataSmart object at 0x7fef7ca97c70>):
                     parser = bb.codeparser.PythonParser(key, logger)
    >                parser.parse_python(value, filename=varflags.get("filename"), lineno=varflags.get("lineno"))
                     deps = deps | parser.references
  File "/work/projects/b33_2/sources/poky/bitbake/lib/bb/codeparser.py", line 308, in PythonParser.parse_python(node='\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n    """clear the build and temp directories"""\n    dir = d.expand("${WORKDIR}")\n    bb.note("Removing " + dir)\n    oe.path.remove(dir)\n\n    dir = "%s.*" % d.getVar(\'STAMP\')\n    bb.note("Removing " + dir)\n    oe.path.remove(dir)\n\n    for f in (d.getVar(\'CLEANFUNCS\') or \'\').split():\n        bb.build.exec_func(f, d)\n\tcd ${PROJECT_DIR}\n\tgit clean -fd && git checkout -f\n', lineno='25', filename='/work/projects/b33_2/sources/poky/meta/classes/utility-tasks.bbclass'):
             node = "\n" * int(lineno) + node
    >        code = compile(check_indent(str(node)), filename, "exec",
                            ast.PyCF_ONLY_AST)
  File "/work/projects/b33_2/sources/poky/meta/classes/utility-tasks.bbclass", line 37
    cd ${PROJECT_DIR}
                    ^
TabError: inconsistent use of tabs and spaces in indentation

上面例子是用的do_clean[postfuncs] 去执行git_repository_checkout

do_clean[postfuncs] += "git_repository_checkout"
git_repository_checkout(){
       cd ${PROJECT_DIR}
       git clean -fd && git checkout -f
}

其实也可以用这种方法

git_repository_checkout(){
     cd ${PROJECT_DIR}
     git clean -fd && git checkout -f
}
 
do_clean_append() {
     bb.build.exec_func("git_repository_checkout", d)
}

注意自定义函数时，取名的时候不要加_append, _prepend，否则会报如下错误

kernel_append(){
    cd ${PROJECT_DIR}
    git clean -fd && git checkout -f
}
 
do_clean_append() {
     bb.build.exec_func("kernel_append", d)
}

ds', d=):
                     parser = bb.codeparser.ShellParser(key, logger)
    >                parser.parse_shell(parsedvar.value)
                     deps = deps | shelldeps
  File "/work/projects/b33_2/sources/poky/bitbake/lib/bb/codeparser.py", line 343, in ShellParser.parse_shell(value=None):

    >        self._parse_shell(value)
             self.execs = set(cmd for cmd in self.allexecs if cmd not in self.funcdefs)
  File "/work/projects/b33_2/sources/poky/bitbake/lib/bb/codeparser.py", line 354, in ShellParser._parse_shell(value=None):
             except Exception:
    >            bb.error('Error during parse shell code, the last 5 lines are:\n%s' % '\n'.join(value.split('\n')[-5:]))
                 raise
AttributeError: 'NoneType' object has no attribute 'split'

最后贴一些.bb里面可能会用到的一些变量

TOPDIR=/work/projects/b33_2/build_s32g274ardb2

TMPDIR=/work/projects/b33_2/build_s32g274ardb2/tmp

B=/work/projects/b33_2/build_s32g274ardb2/tmp/work/s32g274ardb2-fsl-linux/linux-s32/5.10.109-r0/build

WORKDIR=/work/projects/b33_2/build_s32g274ardb2/tmp/work/s32g274ardb2-fsl-linux/linux-s32/5.10.109-r0

COMPONENTS_DIR=/work/projects/b33_2/build_s32g274ardb2/tmp/sysroots-components

DEPLOY_DIR=/work/projects/b33_2/build_s32g274ardb2/tmp/deploy

DL_DIR=/work/projects/b33_2/downloads

LOG_DIR=/work/projects/b33_2/build_s32g274ardb2/tmp/log

PKGDATA_DIR=/work/projects/b33_2/build_s32g274ardb2/tmp/pkgdata/s32g274ardb2

STAMPS_DIR=/work/projects/b33_2/build_s32g274ardb2/tmp/stamps

THISDIR=/work/projects/b33_2/sources/meta-alb/recipes-kernel/linux

SYSROOT_DIRS=     /usr/include     /usr/lib     /lib     /lib     /usr/share

STAMP=/work/projects/b33_2/build_s32g274ardb2/tmp/stamps/s32g274ardb2-fsl-linux/linux-s32/5.10.109-r0

MACHINE=s32g274ardb2

PN=linux-s32

PV=5.10.109

PR=r0