kehyuanyu

Android 镜像文件生成流程

Android编译基本流程

（1）source build/envsetup.sh

（2）lunch xxx // lunch 之后选择xxx index

（3）make -j16

envsetup.sh 主要是设置编译时的一些系统环境变量，加载 vendor，device 目录(及子目录)下所有vendorsetup.sh ，接着lunch来选择设备类型，编译类型，

并且运行完脚本之后会增加了一些命令mmm,mm,croot ....

Android 生成的目标文件

编译完整个android系统源码之后会生成一些文件也即是目标文件system.img，recovery.img, boot.img, userdata.img,cache.img ...,这些镜像文件就是直接烧写到

android各个对应的分区里面，有的平台是将所有的需要的镜像文件打包成USB bin文件进行增量升级，下面以system.img为例进行分析。

目标文件生成的基本流程

设置完环境变量之后， make systemimage

首先我们可以查看主目录的Makefile

### DO NOT EDIT THIS FILE ###
include build/core/main.mk
### DO NOT EDIT THIS FILE ###

此处主要就是导入了build/core/main.mk,我们看到一下目标

# Build files and then package it into the rom formats
.PHONY: droidcore
droidcore: files \
	systemimage \
	$(INSTALLED_BOOTIMAGE_TARGET) \
	$(INSTALLED_RECOVERYIMAGE_TARGET) \
	$(INSTALLED_USERDATAIMAGE_TARGET) \
	$(INSTALLED_CACHEIMAGE_TARGET) \
	$(INSTALLED_VENDORIMAGE_TARGET) \
	$(INSTALLED_FILES_FILE)

这里面systemimage就是生成system.img 文件的为目标，接着我们可以发现build/core/Makefile中有它的依赖关系

systemimage: $(INSTALLED_SYSTEMIMAGE)

INSTALLED_SYSTEMIMAGE名称定义

INSTALLED_SYSTEMIMAGE := $(PRODUCT_OUT)/system.img

INSTALLED_SYSTEMIMAGE依赖关系

$(INSTALLED_SYSTEMIMAGE): $(BUILT_SYSTEMIMAGE) $(RECOVERY_FROM_BOOT_PATCH) | $(ACP)
	@echo "Install system fs image: $@"
	$(copy-file-to-target)
	$(hide) $(call assert-max-image-size,$@ $(RECOVERY_FROM_BOOT_PATCH),$(BOARD_SYSTEMIMAGE_PARTITION_SIZE))

我们进一步看其子目标BUILT_SYSTEMIMAGE，名称定义以及依赖关系

BUILT_SYSTEMIMAGE := $(systemimage_intermediates)/system.img

BUILT_SYSTEMIMAGE依赖关系

$(BUILT_SYSTEMIMAGE): $(FULL_SYSTEMIMAGE_DEPS) $(INSTALLED_FILES_FILE)
	$(call build-systemimage-target,$@)

其中INSTALLED_FILES_FILE是system.img生成时需要所有安装文件，查看installed-files.txt即可知道system.img 打包了哪些文件， FULL_SYSTEMIMAGE_DEPS

也是其依赖文件

INSTALLED_FILES_FILE := $(PRODUCT_OUT)/installed-files.txt
$(INSTALLED_FILES_FILE): $(FULL_SYSTEMIMAGE_DEPS)
	@echo Installed file list: $@
	@mkdir -p $(dir $@)
	@rm -f $@
	$(hide) build/tools/fileslist.py $(TARGET_OUT) > $@

接着就会调用build-systemimage-target 来完成system.img 的打包流程

define build-systemimage-target
  @echo "Target system fs image: $(1)"
  $(call create-system-vendor-symlink)
  @mkdir -p $(dir $(1)) $(systemimage_intermediates) && rm -rf $(systemimage_intermediates)/system_image_info.txt
  $(call generate-userimage-prop-dictionary, $(systemimage_intermediates)/system_image_info.txt, \
      skip_fsck=true)
  $(hide) PATH=$(foreach p,$(INTERNAL_USERIMAGES_BINARY_PATHS),$(p):)$$PATH \
      ./build/tools/releasetools/build_image.py \
      $(TARGET_OUT) $(systemimage_intermediates)/system_image_info.txt $(1) \
      || ( echo "Out of space? the tree size of $(TARGET_OUT) is (MB): " 1>&2 ;\
           du -sm $(TARGET_OUT) 1>&2;\
           echo "The max is $$(( $(BOARD_SYSTEMIMAGE_PARTITION_SIZE) / 1048576 )) MB." 1>&2 ;\
           mkdir -p $(DIST_DIR); cp $(INSTALLED_FILES_FILE) $(DIST_DIR)/installed-files-rescued.txt; \
           exit 1 )

从上面定义的编译命令包来看，主要干活的脚本是build/tools/releasetools/build_image.py,调用该脚本时，传入了三个参数：

（1）打包的system.img需要的文件目录； // 通常为 ${ANDROID_PRODUCT_OUT}/system
（2）生成system.img 时，需要的配置信息文件； // ${ANDROID_PRODUCT_OUT}/obj/PACKAGING/systemimage_intermediates/system_image_info.txt
（3）输出的目标文件； // 也即生成的system.img 的完整输出路径和文件名，${ANDROID_PRODUCT_OUT}/obj/PACKAGING/systemimage_intermediates/system.img

接着我们分析一下build_image.py脚本和配置文件system_image_info.txt

System_image_info.txt 分析

fs_type=ext4
system_size=0x28A00000
userdata_size=0x40000000
cache_fs_type=ext4
cache_size=0x40000000
selinux_fc=out/target/product/cibn/root/file_contexts
skip_fsck=true

文件中之处了system.img 时，文件系统的类型ext4, system.img的大小，已经依赖的文件file_contexts文件路径

build_image.py脚本

下面先看一下主函数main

def main(argv):
  if len(argv) != 3:
    print __doc__
    sys.exit(1)

  in_dir = argv[0]
  glob_dict_file = argv[1]
  out_file = argv[2]

  glob_dict = LoadGlobalDict(glob_dict_file)
  image_filename = os.path.basename(out_file)
  mount_point = ""
  if image_filename == "system.img":
    mount_point = "system"
  elif image_filename == "userdata.img":
    mount_point = "data"
  elif image_filename == "cache.img":
    mount_point = "cache"
  elif image_filename == "vendor.img":
    mount_point = "vendor"
  elif image_filename == "oem.img":
    mount_point = "oem"
  else:
    print >> sys.stderr, "error: unknown image file name ", image_filename
    exit(1)

  image_properties = ImagePropFromGlobalDict(glob_dict, mount_point)
  if not BuildImage(in_dir, image_properties, out_file):
    print >> sys.stderr, "error: failed to build %s from %s" % (out_file, in_dir)
    exit(1)

LoadGlobalDict, ImagePropFromGlobalDict 主要解析BuildImage需要的参数

def LoadGlobalDict(filename):
  """Load "name=value" pairs from filename"""
  d = {}
  f = open(filename)
  for line in f:
    line = line.strip()
    if not line or line.startswith("#"):
      continue
    k, v = line.split("=", 1)
    d[k] = v
  f.close()
  return d

LoadGlobalDict 解析system_image_info.txt文件，而ImagePropFromGlobalDict解析得到system 的属性信息

def ImagePropFromGlobalDict(glob_dict, mount_point):
  """Build an image property dictionary from the global dictionary.

  Args:
    glob_dict: the global dictionary from the build system.
    mount_point: such as "system", "data" etc.
  """
  d = {}
  if "build.prop" in glob_dict:
    bp = glob_dict["build.prop"]
    if "ro.build.date.utc" in bp:
      d["timestamp"] = bp["ro.build.date.utc"]

  def copy_prop(src_p, dest_p):
    if src_p in glob_dict:
      d[dest_p] = str(glob_dict[src_p])

  common_props = (
      "extfs_sparse_flag",
      "mkyaffs2_extra_flags",
      "selinux_fc",
      "skip_fsck",
      "verity",
      "verity_key",
      "verity_signer_cmd"
      )
  for p in common_props:
    copy_prop(p, p)

  d["mount_point"] = mount_point
  if mount_point == "system":
    copy_prop("fs_type", "fs_type")
    copy_prop("system_size", "partition_size")
    copy_prop("system_verity_block_device", "verity_block_device")
  elif mount_point == "data":
    # Copy the generic fs type first, override with specific one if available.
    copy_prop("fs_type", "fs_type")
    copy_prop("userdata_fs_type", "fs_type")
    copy_prop("userdata_size", "partition_size")
  elif mount_point == "cache":
    copy_prop("cache_fs_type", "fs_type")
    copy_prop("cache_size", "partition_size")
  elif mount_point == "vendor":
    copy_prop("vendor_fs_type", "fs_type")
    copy_prop("vendor_size", "partition_size")
    copy_prop("vendor_verity_block_device", "verity_block_device")
  elif mount_point == "oem":
    copy_prop("fs_type", "fs_type")
    copy_prop("oem_size", "partition_size")

  return d

接着看最主要的动作BuildImage，打包system.img

def BuildImage(in_dir, prop_dict, out_file,
               fs_config=None,
               fc_config=None,
               block_list=None):
  """Build an image to out_file from in_dir with property prop_dict.

  Args:
    in_dir: path of input directory.
    prop_dict: property dictionary.
    out_file: path of the output image file.
    fs_config: path to the fs_config file (typically
      META/filesystem_config.txt).  If None then the configuration in
      the local client will be used.
    fc_config: path to the SELinux file_contexts file.  If None then
      the value from prop_dict['selinux_fc'] will be used.

  Returns:
    True iff the image is built successfully.
  """
  build_command = []
  fs_type = prop_dict.get("fs_type", "")
  run_fsck = False

  is_verity_partition = "verity_block_device" in prop_dict
  verity_supported = prop_dict.get("verity") == "true"
  # adjust the partition size to make room for the hashes if this is to be verified
  if verity_supported and is_verity_partition:
    partition_size = int(prop_dict.get("partition_size"))
    adjusted_size = AdjustPartitionSizeForVerity(partition_size)
    if not adjusted_size:
      return False
    prop_dict["partition_size"] = str(adjusted_size)
    prop_dict["original_partition_size"] = str(partition_size)

  if fs_type.startswith("ext"):
    build_command = ["mkuserimg.sh"]
    if "extfs_sparse_flag" in prop_dict:
      build_command.append(prop_dict["extfs_sparse_flag"])
      run_fsck = True
    build_command.extend([in_dir, out_file, fs_type,
                          prop_dict["mount_point"]])
    build_command.append(prop_dict["partition_size"])
    if "timestamp" in prop_dict:
      build_command.extend(["-T", str(prop_dict["timestamp"])])
    if fs_config is not None:
      build_command.extend(["-C", fs_config])
    if block_list is not None:
      build_command.extend(["-B", block_list])
    if fc_config is not None:
      build_command.append(fc_config)
    elif "selinux_fc" in prop_dict:
      build_command.append(prop_dict["selinux_fc"])
  elif fs_type.startswith("f2fs"):
    build_command = ["mkf2fsuserimg.sh"]
    build_command.extend([out_file, prop_dict["partition_size"]])
  else:
    build_command = ["mkyaffs2image", "-f"]
    if prop_dict.get("mkyaffs2_extra_flags", None):
      build_command.extend(prop_dict["mkyaffs2_extra_flags"].split())
    build_command.append(in_dir)
    build_command.append(out_file)
    if "selinux_fc" in prop_dict:
      build_command.append(prop_dict["selinux_fc"])
      build_command.append(prop_dict["mount_point"])

  exit_code = RunCommand(build_command)
  if exit_code != 0:
    return False

  # create the verified image if this is to be verified
  if verity_supported and is_verity_partition:
    if not MakeVerityEnabledImage(out_file, prop_dict):
      return False

  if run_fsck and prop_dict.get("skip_fsck") != "true":
    success, unsparse_image = UnsparseImage(out_file, replace=False)
    if not success:
      return False

    # Run e2fsck on the inflated image file
    e2fsck_command = ["e2fsck", "-f", "-n", unsparse_image]
    exit_code = RunCommand(e2fsck_command)

    os.remove(unsparse_image)

  return exit_code == 0

由于system.img 定义的文件系统类型的ext4,我们就从if fs_type.startswith("ext"): 处看起，一直在封装打包命令build_command，最后调用Rumcommand(cmd)运行

def RunCommand(cmd):
  """ Echo and run the given command

  Args:
    cmd: the command represented as a list of strings.
  Returns:
    The exit code.
  """
  print "Running: ", " ".join(cmd)
  p = subprocess.Popen(cmd)
  p.communicate()
  return p.returncode

其实运行的命令如下：

mkuserimg.sh out/target/product/cibn/system out/target/product/cibn/obj/PACKAGING/systemimage_intermediates/system.img ext4 system 0x28A00000 out/target/product/cibn/root/file_contexts

而mkuserimg.sh是编译完系统之后生成，mkuserimg.sh在${ANDROID_PRODUCT_OUT}/out/host/linux-x86/bin/目录下

xxxxxx@yf153:~/Mstar_828$ ls /home/backup/xxxxxx/Mstar_828/out/host/linux-x86/bin/
aapt           alignment.exe   bcc_strip_attr  clang-tblgen  dex2oat      dx                imgdiff        make_ext4fs  mkuserimg.sh  simg2img
acp            apicheck        bsdiff          crc           dexdeps      e2fsck            insertkeys.py  minigzip     oatdump       SubSecureInfoGen.exe
adb            aprotoc         checkfc         dalvikvm      dexdump      fastboot          llvm-rs-cc     mkbootfs     patchoat      validatekeymaps
aescrypt2.exe  backtrace_test  checkpolicy     dalvikvm32    dexlist      hierarchyviewer1  llvm-tblgen    mkimage      rmtypedefs    zipalign
aidl           bcc             checkseapp      dalvikvm64    dmtracedump  hprof-conv        lzop           mksdcard     rsa_sign

根据上面的分析，我们可以在编译完一次android系统之后，直接使用一下命令来生成system.img 文件

xxxxxx@yf153:~/project_name$ croot
xxxxxx@yf153:~/project_name$ mkuserimg.sh out/target/product/cibn/system out/target/product/cibn/obj/PACKAGING/systemimage_intermediates/system.img ext4 system 0x28A00000 out/target/product/cibn/root/file_contexts
make_ext4fs -T -1 -S out/target/product/cibn/root/file_contexts -l 0x28A00000 -a system out/target/product/cibn/obj/PACKAGING/systemimage_intermediates/system.img out/target/product/cibn/system
Creating filesystem with parameters:
    Size: 681574400
    Block size: 4096
    Blocks per group: 32768
    Inodes per group: 6944
    Inode size: 256
    Journal blocks: 2600
    Label: 
    Flexbg size: 8
    Flexbg groups: 1
    Blocks: 166400
    Block groups: 6
    Reserved block group size: 47
Created filesystem with 1762/41664 inodes and 130806/166400 blocks

可以看到直接输入一下参数：

out/target/product/cibn/system， // 输入打包目录

out/target/product/cibn/obj/PACKAGING/systemimage_intermediates/system.img， // 生成的输出文件

ext4， // 文件系统类型

system， // system.img 文件在系统的挂载点

0x28A00000，// system.img 文件的大小

out/target/product/cibn/root/file_contexts， // selinux_fc 系统安装策略配置文件
接下来我们看一下文件mkuserimg.sh。

mkuserimg.sh

android5.1 下的路径是 external/qemu/distrib/ext4_utils/src/mkuserimg.sh ，详细内容如下：

#!/bin/bash -x
#
# To call this script, make sure make_ext4fs is somewhere in PATH

function usage() {
cat<<EOT
Usage:
mkuserimg.sh [-s] SRC_DIR OUTPUT_FILE EXT_VARIANT MOUNT_POINT SIZE [FILE_CONTEXTS]
EOT
}

echo "in mkuserimg.sh PATH=$PATH"

ENABLE_SPARSE_IMAGE=
if [ "$1" = "-s" ]; then
  ENABLE_SPARSE_IMAGE="-s"
  shift
fi

if [ $# -ne 5 -a $# -ne 6 ]; then
  usage
  exit 1
fi

SRC_DIR=$1
if [ ! -d $SRC_DIR ]; then
  echo "Can not find directory $SRC_DIR!"
  exit 2
fi

OUTPUT_FILE=$2
EXT_VARIANT=$3
MOUNT_POINT=$4
SIZE=$5
FC=$6

case $EXT_VARIANT in
  ext4) ;;
  *) echo "Only ext4 is supported!"; exit 3 ;;
esac

if [ -z $MOUNT_POINT ]; then
  echo "Mount point is required"
  exit 2
fi

if [ -z $SIZE ]; then
  echo "Need size of filesystem"
  exit 2
fi

if [ -n "$FC" ]; then
    FCOPT="-S $FC"
fi

MAKE_EXT4FS_CMD="make_ext4fs $ENABLE_SPARSE_IMAGE $FCOPT -l $SIZE -a $MOUNT_POINT $OUTPUT_FILE $SRC_DIR"
echo $MAKE_EXT4FS_CMD
$MAKE_EXT4FS_CMD
if [ $? -ne 0 ]; then
  exit 4
fi

mkuserimg.sh调用make_ext4fs ，而工具make_ext4fs又是在哪里，我们可以找一下

kehuanyu@yf153:~/Mstar_828$ find . -name "*.mk" |xargs -i grep -rwnH "make_ext4fs" {}
./external/qemu/distrib/ext4_utils/sources.mk:12:    src/make_ext4fs.c \
./external/qemu/distrib/ext4_utils/src/Android.mk:6:    make_ext4fs.c \
./external/qemu/distrib/ext4_utils/src/Android.mk:37:LOCAL_MODULE := make_ext4fs
./external/qemu/distrib/ext4_utils/src/Android.mk:78:LOCAL_MODULE := make_ext4fs

在/external/qemu/distrib/ext4_utils/src/下，由make_ext4fs.c , make_ext4fs_main.c 文件编译生成，后面mkuserimg.sh转调make_ext4fs的命令是：

make_ext4fs -T -1 -S out/target/product/cibn/root/file_contexts -l 0x28A00000 -a system out/target/product/cibn/obj/PACKAGING/systemimage_intermediates/system.img out/target/product/cibn/system

mkuserimg.sh, make_ext4fs 的参数基本类似，有了这个命令，我们可以最大限度的去定制android所需要的镜像文件(*.img)

镜像文件的切分

有一些镜像文件比较大，而tftp烧录文件时，内存大小只有大约200M，所以必要时要进行拆分，压缩，使用的命令split , lzop

xxxxxx@yf153:~/projectName/out/target/product/cibn/temp$ ls
system.img
xxxxxx@yf153:~/projectName/out/target/product/cibn/temp$ split -b 150m system.img system.img 
xxxxxx@yf153:~/projectName/out/target/product/cibn/temp$ ls
system.img  system.imgaa  system.imgab  system.imgac  system.imgad  system.imgae
xxxxxx@yf153:~/projectName/out/target/product/cibn/temp$ lzop -f -o system.imga
system.imgaa  system.imgab  system.imgac  system.imgad  system.imgae  
xxxxxx@yf153:~/projectName/out/target/product/cibn/temp$ lzop -f -o system.imgaa.lzo system.imgaa 
xxxxxx@yf153:~/projectName/out/target/product/cibn/temp$ ls system.imgaa*
system.imgaa  system.imgaa.lzo
xxxxxx@yf153:~/projectName/out/target/product/cibn/temp$ ls system.imgaa* -al
-rw-rw-r-- 1 xxxxxx xxxxxx 157286400  7月 28 20:08 system.imgaa
-rw-rw-r-- 1 xxxxxx xxxxxx  95970609  7月 28 20:08 system.imgaa.lzo

可以看到命令：

split -b 150m system.img system.img // 按照150M进行拆分，生成文件system.img**

lzop -f -o system.imgaa.lzo system.imgaa // lzop 进行压缩，发现大小由157286400字节变成95970609