【详细深入分析OpenHarmony编译流程】

前言

OpenHarmony做为复杂的操作系统软件，其编译构建也比较复杂。本文主要记录作者自己在阅读编译脚本代码后对其的认识，不代表官方观点，不足之处还请批评指正。

本文顺着编译流程阅读和解释编译脚本代码，即执行时间靠前的代码先解释，执行时间靠后的代码后解释。

本文参考代码为2023年3月30日master分支代码。

为了方便查看OpenHarmony代码，作者搭建了一个服务器，欢迎使用https://www.lengqinjie.xyz。
对于master分支代码，此服务器上的代码每日更新一次，所以当你看到时，可能和本文有差异，属于正常情况。

你也可以自行从gitee下载此版本代码,使用如下命令，git工具下载方法请参考互联网。

git clone https://gitee.com/openharmony/build.git -b master
cd build
git reset --hard 9574463e6f568f81b8d89134306052782c1f4c14

编译入口

共有3种形式的编译入口

1. hb build
2. ./build.sh
3. ./build.py

3种形式的编译入口最终都汇总到hb build。即内部实际上是统一的。下面以build.py为例进行讲解。

源码根目录

源码根目录下有很多子目录，这些子目录大多数为OpenHarmony各软件子系统的实现，其中1个目录为build目录，即本文重点讨论的内容，此目录内主要为编译构建脚本，主要是python语言实现。

为了提供方便，源码根目录下有2个符号链接，引用了编译构建的脚本文件

build.py -> build/lite/build.py
build.sh -> build/build_scripts/build.sh

因此在源码根目录下执行这2个文件中的一个就能够启动编译构建过程。还有一种启动方式是hb命令，稍后再讲解。

编译命令

以build.py为例，在源码根目录下执行

./build.py -p rk3568

这里-p rk3568意思是编译构建产品(rk3568)。当然还支持很多其它参数，为了简化，这里只列出这个最重要的参数

记录源码根目录位置

build/lite/build.py

0071 def main():
0072     root_path = os.path.dirname(os.path.abspath(__file__))
0073     ret_code = set_root_path(root_path)
0074     if ret_code != 0:
0075         return ret_code
0076     return build(root_path, sys.argv[1:])
0077 
0078 
0079 if __name__ == "__main__":
0080     sys.exit(main())

注意72行代码中的__file__指的是源码根目录下的build.py，而不是build/lite/build.py
因为是在源码根目录下执行的，所以root_path就是源码根目录。
build/lite/build.py

0053 def set_root_path(path):
0054     sys.path.insert(0, os.path.join(path, 'build/lite'))
0055     module = importlib.import_module('hb_internal.set.set')
0056     return module.set_root_path(root_path=path)

然后将build/lite加入python程序搜索路径，方便后续python脚本查找自定义模块，55行代码就用上了54行代码的结果，在build/lite子目录内查找hb_internal.set.set模块并导入，最后执行此模块的设置源码根目录的方法
build/lite/hb_internal/set/set.py

0048 def set_root_path(root_path=None):
0049     config = Config()
0050     if root_path is None:
0051         try:
0052             hb_info(f'hb root path: {config.root_path}')
0053             return 0
0054         except OHOSException:
0055             root_path = get_input('[OHOS INFO] Input code path: ')
0056     config.root_path = root_path
0057     return 0

这里49行为一个配置对象，整个编译构建系统只有1个config对象(后续再解释)。这里就把root_path记录到config对象中(56行)。

开始构建

build/lite/build.py

0059 def build(path, args_list):
0060     python_executable = get_python()
0061     cmd = [python_executable, 'build/hb/main.py', 'build'] + args_list
0062     for args in args_list:
0063         if "using_hb_new=false" in args:
0064             cmd = [python_executable, 'build/lite/hb/__main__.py', 'build'] + args_list
0065             break
0066     if not os.path.exists(os.path.join(path,'build/hb/main.py')):
0067         cmd = [python_executable, 'build/lite/hb/__main__.py', 'build'] + args_list
0068     return check_output(cmd, cwd=path)

源码根目录记录成功以后，开始进行系统构建(build)。首先获取python解释器(60行), 然后再构造命令行参数(61行)。紧接着进行新老版本构建的判断，分别构造不同的命令行。本文不解释老版本构建方式，即不考虑(62到67行之间的内容）。最后执行构建命令并检查执行结果(68行)。

从61行代码可以看出，这里等价于执行

hb build args_list

前提条件是对build/hb目录中python代码进行了安装动作

pip install build/hb

获取python解释器

build/lite/build.py

0025 def get_python():
0026     hb_main = importlib.import_module("hb.__main__")
0027     topdir = hb_main.find_top()
0028     python_base_dir = os.path.join(topdir, 'prebuilts/python')
0029     if os.path.exists(python_base_dir):
0030         python_dir = hb_main.search(python_base_dir, 'python3')
0031         return os.path.join(python_dir, 'python3')
0032     else:
0033         print("please execute build/prebuilts_download.sh")
0034         sys.exit()

这里的主要目的是判断源码根目录下是否有prebuilts/python这个目录，如果没有的话，则需要执行
build/prebuilts_download.sh（当然是在源码根目录执行了）去下载对应的python解释器程序文件，当然这里还会下载其它一些工具，

这样做的目的是减少用户自行安装软件失败的可能性，因为这些软件对版本有要求，随着openharmony代码的更新，可能需要新的工具软件或者软件的新版本。通过上述脚本下载的工具软件都会存放在prebuilts目录下。
上述程序返回的是python解释器程序的文件路径。
build/lite/hb/__main__.py

0035 def find_top():
0036     cur_dir = os.getcwd()
0037     while cur_dir != "/":
0038         hb_internal = os.path.join(cur_dir, 'build/lite/hb_internal')
0039         if os.path.exists(hb_internal):
0040             return cur_dir
0041         cur_dir = os.path.dirname(cur_dir)
0042     raise Exception("Please call hb utilities inside source root directory")

上述代码的意思是从build/lite/hb目录开始一直往父目录查找，如果找到一个目录，其内部存在build/lite/hb_internal目录，则此目录为源码根目录，最后返回源码根目录(40行)

0045 def search(findir, target):
0046     for root, dirs, files in os.walk(findir):
0047         if target in files:
0048             return root
0049     return False

这里主要是从目录findir中去寻找target文件，可以是在子目录内(递归查找),如果文件存在，返回文件所在的目录

hb主程序

当python解释器成功获取后，就会使用构造的参数调用hb主程序
build/hb/main.py

0104     @staticmethod
0105     @throw_exception
0106     def main():
0107         main = Main()
0108         module_type = sys.argv[1]
0109         if module_type == 'build':
0110             module = main._init_build_module()
0111         elif module_type == 'set':
0112             module = main._init_set_module()
0113         elif module_type == 'env':
0114             module = main._init_env_module()
0115         elif module_type == 'clean':
0116             module = main._init_clean_module()
0117         elif module_type == 'tool':
0118             module = main._init_tool_module()
0119         elif module_type == 'help':
0120             for all_module_type in ModuleType:
0121                 LogUtil.hb_info(Separator.long_line)
0122                 LogUtil.hb_info(Arg.get_help(all_module_type))
0123             exit()
0124         else:
0125             raise OHOSException(
0126                 'There is no such option {}'.format(module_type), '0018')
0127         try:
0128             module.run()
0129         except KeyboardInterrupt:
0130             for file in os.listdir(ARGS_DIR):
0131                 if file.endswith('.json') and os.path.exists(os.path.join(ARGS_DIR, file)):
0132                     os.remove(os.path.join(ARGS_DIR, file))
0133             print('User abort')
0134             return -1
0135         else:
0136             return 0
0137 
0138 
0139 
0140 if __name__ == "__main__":
0141     sys.exit(Main.main())

本程序有2种使用方式

1. hb (build | set | clean | env | tool | help) args
2. python3 build/hb/main.py (build | set | clean | env | tool | help) args
   第一种方式需要安装此python模块，即

pip install build/hb

第二种方式即上文介绍的方式，参考build/lite/build.py第61行。

build模块初始化

build/hb/main.py

0062     def _init_build_module(self) -> BuildModuleInterface:
0063         args_dict = Arg.parse_all_args(ModuleType.BUILD)
0064 
0065         if args_dict.get("product_name").arg_value != '':
0066             set_args_dict = Arg.parse_all_args(ModuleType.SET)
0067             set_args_resolver = SetArgsResolver(set_args_dict)
0068             ohos_set_module = OHOSSetModule(set_args_dict, set_args_resolver, "")
0069             ohos_set_module.set_product()
0070 
0071         preloader = OHOSPreloader()
0072         loader = OHOSLoader()
0073         generate_ninja = Gn()
0074         ninja = Ninja()
0075         build_args_resolver = BuildArgsResolver(args_dict)
0076 
0077         return OHOSBuildModule(args_dict, build_args_resolver, preloader, loader, generate_ninja, ninja)

63行先将所有build能识别的参数进行解析，并存入args_dict字典中。
如果参数里面有"product_name", 那么还要执行set的逻辑(65到69行)。本文中的-p参数就是product_name的简写，所以这段逻辑会执行。
71到75行初始化几个对象，代表编译构建的几个阶段：预加载、加载、生成ninja脚本，执行ninja脚本，以及初始化build阶段各参数的解析器。
最后(77行)用这些对象初始化构建模块

build模块执行

build/hb/modules/ohos_build_module.py

0062     @throw_exception
0063     def run(self):
0064         try:
0065             super().run()
0066         except OHOSException as exception:
0067             raise exception
0068         else:
0069             LogUtil.hb_info('{} build success'.format(
0070                 self.args_dict.get('product_name').arg_value))
0071             LogUtil.hb_info('Cost time:  {}'.format(self.build_time))

run函数实际上使用的是父类的函数，如下
hb/modules/interface/build_module_interface.py

0061     def run(self):
0062         try:
0063             self._prebuild()
0064             self._preload()
0065             self._load()
0066             self._pre_target_generate()
0067             self._target_generate()
0068             self._post_target_generate()
0069             self._pre_target_compilation()
0070             self._target_compilation()
0071         except OHOSException as exception:
0072             raise exception
0073         else:
0074             self._post_target_compilation()
0075         finally:
0076             self._post_build()

可以看到实际上run定义了一个编译构造的流程，分别是

1. prebuild
2. preload
3. load
4. pre_target_generate
5. target_generate
6. post_target_generate
7. pre_target_compilation
8. target_compilation
9. post_target_compilation
10. post_build

后续会详细讲这10个流程

build和set参数解析

hb/containers/arg.py

0230     @staticmethod
0231     def parse_all_args(module_type: ModuleType) -> dict:
0232         args_dict = {}
0233         parser = argparse.ArgumentParser()
0234         all_args = Arg.read_args_file(module_type)
0235 
0236         for arg in all_args.values():
0237             arg = dict(arg)
0238             ArgsFactory.genetic_add_option(parser, arg)
0239             oh_arg = Arg.create_instance_by_dict(arg)
0240             args_dict[oh_arg.arg_name] = oh_arg
0241 
0242         parser.usage = 'hb {} [option]'.format(module_type.name.lower())
0243         parser_args = parser.parse_known_args(sys.argv[2:])
0244 
0245         for oh_arg in args_dict.values():
0246             if isinstance(oh_arg, Arg):
0247                 assigned_value = parser_args[0].__dict__[oh_arg.arg_name]
0248                 if oh_arg.arg_type == ArgType.LIST:
0249                     convert_assigned_value = TypeCheckUtil.tile_list(assigned_value)
0250                     convert_assigned_value = list(set(convert_assigned_value))
0251                 elif oh_arg.arg_type == ArgType.SUBPARSERS:
0252                     convert_assigned_value = TypeCheckUtil.tile_list(assigned_value)
0253                     if len(convert_assigned_value):
0254                         convert_assigned_value = list(set(convert_assigned_value))
0255                         convert_assigned_value.extend(parser_args[1])
0256                         convert_assigned_value.sort(key=sys.argv[2:].index)
0257                 elif oh_arg.arg_type == ArgType.BOOL or oh_arg.arg_type == ArgType.GATE:
0258                     if str(assigned_value).lower() == 'false':
0259                         convert_assigned_value = False
0260                     elif str(assigned_value).lower() == 'true':
0261                         convert_assigned_value = True
0262                 else:
0263                     convert_assigned_value = assigned_value
0264 
0265                 if oh_arg.arg_attribute.get('deprecated', None) and oh_arg.arg_value != convert_assigned_value:
0266                     LogUtil.hb_warning(
0267                         'compile option "{}" will be deprecated, \
0268                             please consider use other options'.format(oh_arg.arg_name))
0269                 oh_arg.arg_value = convert_assigned_value
0270                 Arg.write_args_file(
0271                     oh_arg.arg_name, oh_arg.arg_value, module_type)
0272 
0273         return args_dict

231行: 参数类型可以是build, set或者其它类型，
232行： 解析完的参数存储在此字典，并最后返回
233行：通用的参数解析器对象
234行：读取此类型支持的参数列表，由json配置文件存储(这些文件在build中的某个地方，后续分析)
236行：遍历本模块（如build）支持的参数
237行：每个参数的描述信息也是一个字典形式
238行：命令行加入对此参数的识别能力
239行：创建一个描述本参数的软件对象
240行：记录下参数名称与参数对象的映射关系
243行：先解析此模块能识别的参数，比如本文的“-p rk3568”，注意参数解析后，原来的参数表(sys.argv)并不会减少，下一个模块还能继续解析。
245行：遍历支持的参数表中的所有参数
247行：读取已解析出的参数值，如本例的rk3568，针对用户没有配置的参数，则读取的是默认值
248~263行：对参数值进行规范化处理
265~268行：如果使用了一个过期的参数，且和默认值不同，即用户指定了其他值，则打印警告提示日志
269~270行：更新参数对象的值并刷新参数配置文件
273行：最后返回参数对象表。

参数配置文件读写

hb/containers/arg.py

0297     @staticmethod
0298     @throw_exception
0299     def read_args_file(module_type: ModuleType):
0300         args_file_path = ''
0301         default_file_path = ''
0302         if module_type == ModuleType.BUILD:
0303             args_file_path = CURRENT_BUILD_ARGS
0304             default_file_path = DEFAULT_BUILD_ARGS
0305         elif module_type == ModuleType.SET:
0306             args_file_path = CURRENT_SET_ARGS
0307             default_file_path = DEFAULT_SET_ARGS
0308         elif module_type == ModuleType.CLEAN:
0309             args_file_path = CURRENT_CLEAN_ARGS
0310             default_file_path = DEFAULT_CLEAN_ARGS
0311         elif module_type == ModuleType.ENV:
0312             args_file_path = CURRENT_ENV_ARGS
0313             default_file_path = DEFAULT_ENV_ARGS
0314         elif module_type == ModuleType.TOOL:
0315             args_file_path = CURRENT_TOOL_ARGS
0316             default_file_path = DEFAULT_TOOL_ARGS
0317         else:
0318             raise OHOSException(
0319                 'You are trying to read args file, but there is no corresponding module "{}" args file'
0320                 .format(module_type.name.lower()), "0018")
0321         if not os.path.exists(CURRENT_ARGS_DIR):
0322             os.makedirs(CURRENT_ARGS_DIR)
0323         if not os.path.exists(args_file_path):
0324             IoUtil.copy_file(src=default_file_path, dst=args_file_path)
0325         return IoUtil.read_json_file(args_file_path)

以build类型的参数文件为例，其它类似。

• CURRENT_BUILD_ARGS： 生效的参数配置文件
• DEFAULT_BUILD_ARGS：默认的参数配置文件
  如果生效的参数文件存在，则直接读出，否则从默认参数文件拷贝后再读出。
  build/hb/resources/global_var.py

0022 CURRENT_OHOS_ROOT = os.path.dirname(os.path.dirname(
0023     os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
0024 CURRENT_BUILD_DIR = os.path.join(CURRENT_OHOS_ROOT, 'build')
0025 CURRENT_HB_DIR = os.path.join(CURRENT_BUILD_DIR, 'hb')
0026 DEFAULT_CCACHE_DIR = os.path.join(CURRENT_OHOS_ROOT, '.ccache')
0027 
0028 ARGS_DIR = os.path.join(CURRENT_HB_DIR, 'resources/args')
0029 
0030 DEFAULT_BUILD_ARGS = os.path.join(
0031     CURRENT_HB_DIR, 'resources/args/default/buildargs.json')
0032 DEFAULT_SET_ARGS = os.path.join(
0033     CURRENT_HB_DIR, 'resources/args/default/setargs.json')
0034 DEFAULT_CLEAN_ARGS = os.path.join(
0035     CURRENT_HB_DIR, 'resources/args/default/cleanargs.json')
0036 DEFAULT_ENV_ARGS = os.path.join(
0037     CURRENT_HB_DIR, 'resources/args/default/envargs.json')
0038 DEFAULT_TOOL_ARGS = os.path.join(
0039     CURRENT_HB_DIR, 'resources/args/default/toolargs.json')
0040 
0041 CURRENT_ARGS_DIR = os.path.join(CURRENT_OHOS_ROOT, 'out/hb_args')
0042 CURRENT_BUILD_ARGS = os.path.join(
0043     CURRENT_ARGS_DIR, 'buildargs.json')
0044 CURRENT_SET_ARGS = os.path.join(
0045     CURRENT_ARGS_DIR, 'setargs.json')
0046 CURRENT_CLEAN_ARGS = os.path.join(
0047     CURRENT_ARGS_DIR, 'cleanargs.json')
0048 CURRENT_ENV_ARGS = os.path.join(
0049     CURRENT_ARGS_DIR, 'envargs.json')
0050 CURRENT_TOOL_ARGS = os.path.join(
0051     CURRENT_ARGS_DIR, 'toolargs.json')

hb build, hb set, hb clean等参数文件位置和文件名称定义如上
以build为例

• 默认参数文件位置为：build/hb/resources/args/default/buildargs.json
• 生效参数文件位置为：out/hb_args/buildargs.json

hb/containers/arg.py

0275     @staticmethod
0276     @throw_exception
0277     def write_args_file(key: str, value, module_type: ModuleType):
0278         args_file_path = ''
0279         if module_type == ModuleType.BUILD:
0280             args_file_path = CURRENT_BUILD_ARGS
0281         elif module_type == ModuleType.SET:
0282             args_file_path = CURRENT_SET_ARGS
0283         elif module_type == ModuleType.CLEAN:
0284             args_file_path = CURRENT_CLEAN_ARGS
0285         elif module_type == ModuleType.ENV:
0286             args_file_path = CURRENT_ENV_ARGS
0287         elif module_type == ModuleType.TOOL:
0288             args_file_path = CURRENT_TOOL_ARGS
0289         else:
0290             raise OHOSException(
0291                 'You are trying to write args file, but there is no corresponding module "{}" args file'
0292                 .format(module_type), "0002")
0293         args_file = Arg.read_args_file(module_type)
0294         args_file[key]["argDefault"] = value
0295         IoUtil.dump_json_file(args_file_path, args_file)

上述函数根据用户配置的参数值，刷新参数配置文件。

配置对象的初始化

参考build/lite/hb_internal/set/set.py的第49行，这里是config对象的首次创建，我们来看看其怎么初始化的
build/lite/hb_internal/common/config.py

0034 class Config(metaclass=Singleton):
0035     def __init__(self):
0036         self.config_json = CONFIG_JSON
0037 
0038         config_content = read_json_file(self.config_json)
0039         self._root_path = config_content.get('root_path', None)
0040         self._board = config_content.get('board', None)
0041         self._kernel = config_content.get('kernel', None)
0042         self._product = config_content.get('product', None)
0043         self._product_path = config_content.get('product_path', None)
0044         self._device_path = config_content.get('device_path', None)
0045         self._device_company = config_content.get('device_company', None)
0046         self._patch_cache = config_content.get('patch_cache', None)
0047         self._version = config_content.get('version', '3.0')
0048         self._os_level = config_content.get('os_level', 'small')
0049         self._product_json = config_content.get('product_json', None)
0050         self._target_cpu = config_content.get('target_cpu', None)
0051         self._target_os = config_content.get('target_os', None)
0052         self._out_path = config_content.get('out_path', None)
0053         self._compile_config = config_content.get('compile_config', None)
0054         self._component_type = config_content.get('component_type', None)
0055         self._device_config_path = config_content.get('device_config_path',
0056                                                       None)
0057         self._product_config_path = config_content.get('product_config_path',
0058                                                        None)
0059         self._subsystem_config_json = config_content.get(
0060             'subsystem_config_json', None)
0061         self._subsystem_config_overlay_json = ''
0062         self.fs_attr = set()
0063         self.platform = platform.system()

34行：Config类是一个单实例类，即只会存在一个Config类型的对象
36行：记录config配置文件的位置
38行：读取config配置文件
39~60行：将读取到的值记录到config对象，如果值不存在，则记录为None。
61~62行：初始化2个空的配置属性
63行：记录当前平台类型，如windows, linux, macos等

配置文件位置

build/​lite/​hb_internal/​__init__.py

0022 def get_config_path():
0023     search_path = os.getcwd()
0024     root_path = os.path.abspath(os.sep)
0025     while search_path != root_path:
0026         config_path = os.path.join(search_path, 'ohos_config.json')
0027         if os.path.isfile(config_path):
0028             return config_path
0029         search_path = os.path.dirname(search_path)
0030     return os.path.abspath(
0031         os.path.join(os.path.dirname(__file__), 'common', 'config.json'))
0032 
0033 
0034 CONFIG_JSON = get_config_path()

从上述代码可知：
如果根目录下存在ohos_config.json，则配置文件为ohos_config.json
否则配置文件为build/lite/hb_internal/common/config.json
刚开始时ohos_config.json不存在，什么时候生成的呢？
请看如下代码逻辑
build/lite/hb_internal/common/config.py

0137     @root_path.setter
0138     def root_path(self, value):
0139         self._root_path = os.path.abspath(value)
0140         if not os.path.isdir(self._root_path):
0141             raise OHOSException(f'{self._root_path} is not a valid path')
0142 
0143         config_path = os.path.join(self._root_path, 'ohos_config.json')
0144         if not os.path.isfile(config_path):
0145             self.config_create(config_path)
0146         self.config_update('root_path', self._root_path)

0384     def config_create(self, config_path):
0385         dump_json_file(config_path, CONFIG_STRUCT)
0386         self.config_json = config_path

build/lite/hb_internal/init.py

0035 CONFIG_STRUCT = {
0036     "root_path": None,
0037     "board": None,
0038     "kernel": None,
0039     "product": None,
0040     "product_path": None,
0041     "device_path": None,
0042     "device_company": None,
0043     "os_level": None,
0044     "version": None,
0045     "patch_cache": None
0046 }

build/lite/hb_internal/set/set.py

0056     config.root_path = root_path

56行： 触发setter的执行，从而创建ohos_config.json

产品配置信息的获取

刚开始时，除了在配置对象中设置了源码根目录的位置，其它数据都是None, 那么某产品(如rk3568)的实际配置是怎么获取到的呢
让我们回到build/hb/main.py第69行，这里会刷新产品信息
build/hb/main.py

0069             ohos_set_module.set_product()

然后我们看看set_product的实现
build/​hb/​modules/​ohos_set_module.py

0046     def set_product(self):
0047         self.args_resolver.resolve_arg(self.args_dict['product_name'], self)

进一步查看对应的resolve_arg函数
​build/​hb/​resolver/​set_args_resolver.py

0035     @staticmethod
0036     def resolve_product_name(target_arg: Arg, set_module: SetModuleInterface):
0037         config = Config()
0038         product_info = dict()
0039         device_info = dict()
0040         if target_arg.arg_value == '':
0041             product_info = set_module._menu.select_product()
0042         elif target_arg.arg_value.__contains__('@'):
0043             product_name, company_name = target_arg.arg_value.split('@', 2)
0044             product_info = ProductUtil.get_product_info(
0045                 product_name, company_name)
0046         else:
0047             product_info = ProductUtil.get_product_info(target_arg.arg_value)
0048 
0049         config.product = product_info.get('name')
0050         config.product_path = product_info.get('product_path')
0051         config.version = product_info.get('version')
0052         config.os_level = product_info.get('os_level')
0053         config.product_json = product_info.get('config')
0054         config.component_type = product_info.get('component_type')
0055         if product_info.get('product_config_path'):
0056             config.product_config_path = product_info.get(
0057                 'product_config_path')
0058         else:
0059             config.product_config_path = product_info.get('path')
0060 
0061         device_info = ProductUtil.get_device_info(config.product_json)
0062         config.board = device_info.get('board')
0063         config.kernel = device_info.get('kernel')
0064         config.target_cpu = device_info.get('target_cpu')
0065         config.target_os = device_info.get('target_os')
0066         config.support_cpu = device_info.get("support_cpu")
0067         kernel_version = device_info.get('kernel_version')
0068         config.device_company = device_info.get('company')
0069         board_path = device_info.get('board_path')
0070 
0071         if product_info.get('build_out_path'):
0072             config.out_path = os.path.join(config.root_path,
0073                                            product_info.get('build_out_path'))
0074         else:
0075             if config.os_level == 'standard':
0076                 config.out_path = os.path.join(config.root_path, 'out',
0077                                                config.board)
0078             else:
0079                 config.out_path = os.path.join(config.root_path, 'out',
0080                                                config.board, config.product)
0081 
0082         if product_info.get('subsystem_config_json'):
0083             config.subsystem_config_json = product_info.get(
0084                 'subsystem_config_json')
0085         else:
0086             config.subsystem_config_json = 'build/subsystem_config.json'
0087 
0088         subsystem_config_overlay_path = os.path.join(
0089             config.product_path, 'subsystem_config_overlay.json')
0090         if os.path.isfile(subsystem_config_overlay_path):
0091             if product_info.get('subsystem_config_overlay_json'):
0092                 config.subsystem_config_overlay_json = product_info.get(
0093                     'subsystem_config_overlay_json')
0094             else:
0095                 config.subsystem_config_overlay_json = subsystem_config_overlay_path
0096 
0097         if config.version == '2.0':
0098             config.device_path = board_path
0099         else:
0100             if config.os_level == "standard":
0101                 config.device_path = board_path
0102             else:
0103                 config.device_path = DeviceUtil.get_device_path(
0104                     board_path, config.kernel, kernel_version)
0105 
0106         if device_info.get('board_config_path'):
0107             config.device_config_path = device_info.get('board_config_path')
0108         else:
0109             config.device_config_path = config.device_path
0110 
0111         Arg.write_args_file(target_arg.arg_name,
0112                             product_info.get('name'), ModuleType.BUILD)
0113         Arg.write_args_file(target_arg.arg_name,
0114                             product_info.get('name'), ModuleType.SET)

41行：产品名称为空，则弹出菜单让用户选择产品并获取产品配置
43行：产品名称含@符号，则用公司名和产品名一起查找产品配置
47行：直接用产品名查找产品配置
49~59行：根据产品配置刷新config对象
61行：获取设备配置信息
62~69行：根据设备配置信息刷新config对象
71~80行：设置编译构建的输出目录
82~95行：与产品相关的子系统配置信息
97~109行：设备路径和设备配置路径信息
111~114行：将产品名称记录到set和build对应的参数文件中

这里需要注意设备和产品的概念，理论上可以基于设备进行二次开发，形成不同的产品。所以设备和产品的关系可以是一对多的关系。这里的术语设备，有时我们也称为开发板。
至此，config对象的主要信息获取到了。
这里也许你会有一个疑惑，命令行参数product_name和处理函数resolve_product_name是怎么对应上的，这里给出解答
build/​hb/​resources/​args/​default/​setargs.json

0002     "product_name": {
0003         "arg_name": "--product-name",
0004         "argDefault": "",
0005         "arg_help": "Default:''. Help:build a specified product. You could use this option like this: 1.'hb set --product-name rk3568@hihope' 2.'hb set --product-name rk3568' 3.'hb set'[graphical ui]",
0006         "arg_phase": "prebuild",
0007         "arg_type": "str",
0008         "arg_attribute": {
0009             "abbreviation": "-p"
0010         },
0011         "resolve_function": "resolve_product_name",
0012         "testFunction": "testProductName"
0013     },

可以从11行看到product_name的处理函数为resolve_product_name。
build/hb/resolver/interface/args_resolver_interface.py

0031     @throw_exception
0032     def resolve_arg(self, target_arg: Arg, module):
0033         if target_arg.arg_name not in self._args_to_function.keys():
0034             raise OHOSException(
0035                 'You are tring to call {} resolve function, but it has not been defined yet', '0000')
0036         if not hasattr(self._args_to_function[target_arg.arg_name], '__call__'):
0037             raise OHOSException()
0038 
0039         resolve_function = self._args_to_function[target_arg.arg_name]
0040         return resolve_function(target_arg, module)
0041 
0042     @throw_exception
0043     def _map_args_to_function(self, args_dict: dict):
0044         for entity in args_dict.values():
0045             if isinstance(entity, Arg):
0046                 args_name = entity.arg_name
0047                 function_name = entity.resolve_function
0048                 if not hasattr(self, function_name) or \
0049                         not hasattr(self.__getattribute__(function_name), '__call__'):
0050                     raise OHOSException(
0051                         'There is no resolution function for arg: {}'.format(
0052                             args_name),
0053                         "0004")
0054                 entity.resolve_function = self.__getattribute__(function_name)
0055                 self._args_to_function[args_name] = self.__getattribute__(
0056                     function_name)

这里可以洞见，通用的resolve_arg函数怎么调用到具体的与参数对应的解析函数上的
39行：通过参数名称查询对应的解析函数
40行：调用对应的解析函数
那么名称和解析函数映射表是怎么建立的呢，请看43~56行代码
44行：遍历所有的参数对象
46~47行：分别获取参数名和解析函数名称
48行：判断此名称的函数是否存在
54~55行：如果存在则取出函数对象，再建立函数对象与参数名称的映射

0025 class ArgsResolverInterface(metaclass=ABCMeta):
0026 
0027     def __init__(self, args_dict: dict):
0028         self._args_to_function = dict()
0029         self._map_args_to_function(args_dict)

如上所示，解析器对象(子类)初始化的时候，就把参数名与函数映射表建立好了。
此时，config对象的信息也写入配置文件了，因为config对象中可以配置的属性基本都有一个setter方法和一个property方法，下面举个例子
build/hb/resources/config.py

0181     @property
0182     def device_company(self):
0183         if self._device_company is None:
0184             raise OHOSException('Failed to init compile config', '0019')
0185         return self._device_company
0186 
0187     @device_company.setter
0188     def device_company(self, value):
0189         self._device_company = value
0190         self.config_update('device_company', self._device_company)

如上。setter方法用于修改device_company属性时，property方法用于读取device_company属性时。而config_update会刷新配置文件
build/hb/resources/config.py

0378     def config_update(self, key, value):
0379         config_content = IoUtil.read_json_file(self.config_json)
0380         config_content[key] = value
0381         IoUtil.dump_json_file(self.config_json, config_content)

因此ohos_config.json会得到及时的刷新

prebuild

build/hb/modules/ohos_build_module.py

0073     def _prebuild(self):
0074         self._run_phase(BuildPhase.PRE_BUILD)

0111     def _run_phase(self, phase: BuildPhase):
0112         '''Description: Traverse all registered parameters in build process and 
0113             execute the resolver function of the corresponding phase
0114         @parameter: [phase]:  Build phase corresponding to parameter
0115         @return :none
0116         '''
0117         for phase_arg in [arg for arg in self.args_dict.values()if arg.arg_phase == phase]:
0118             self.args_resolver.resolve_arg(phase_arg, self)

刚开始遍历执行参数中为prebuild的阶段的这些处理函数，根据配置文件查询，这些函数有
build/​hb/​resources/​args/​default/​buildargs.json

0011     "resolve_function": "resolve_product",
0028     "resolve_function": "resolve_target_cpu",
0048     "resolve_function": "resolve_ccache",
0058     "resolve_function": "resolve_pycache",
0094     "resolve_function": "resolve_build_target",
0104     "resolve_function": "resolve_ninja_args",
0116     "resolve_function": "resolve_full_compilation",

还有2个参数标记成deprecated（过期）状态，这里没有列出

产品名称解析

build/hb/resolver/build_args_resolver.py

0049     @staticmethod
0050     def resolve_product(target_arg: Arg, build_module: BuildModuleInterface):
0051         """resolve '--product-name' arg.
0052         :param target_arg: arg object which is used to get arg value.
0053         :param build_module [maybe unused]: build module object which is used to get other services.
0054         :phase: prebuild.
0055         """
0056         config = Config()
0057         target_generator = build_module.target_generator
0058         target_generator.regist_arg('product_name', config.product)
0059         target_generator.regist_arg('product_path', config.product_path)
0060         target_generator.regist_arg(
0061             'product_config_path', config.product_config_path)
0062 
0063         target_generator.regist_arg('device_name', config.board)
0064         target_generator.regist_arg('device_path', config.device_path)
0065         target_generator.regist_arg('device_company', config.device_company)
0066         target_generator.regist_arg(
0067             'device_config_path', config.device_config_path)
0068 
0069         target_generator.regist_arg('target_cpu', config.target_cpu)
0070         target_generator.regist_arg(
0071             'is_{}_system'.format(config.os_level), True)
0072 
0073         target_generator.regist_arg('ohos_kernel_type', config.kernel)
0074         target_generator.regist_arg('ohos_build_compiler_specified',
0075                                     ProductUtil.get_compiler(config.device_path))
0076 
0077         target_generator.regist_arg('ohos_build_time',
0078                                     SystemUtil.get_current_time(time_type='timestamp'))
0079         target_generator.regist_arg('ohos_build_datetime',
0080                                     SystemUtil.get_current_time(time_type='datetime'))
0081 
0082         features_dict = ProductUtil.get_features_dict(config.product_json)
0083         for key, value in features_dict.items():
0084             target_generator.regist_arg(key, value)
0085 
0086         if ProductUtil.get_compiler(config.device_path) == 'clang':
0087             target_generator.regist_arg(
0088                 'ohos_build_compiler_dir', config.clang_path)
0089 
0090         if target_arg.arg_value == 'ohos-sdk':
0091             target_generator = build_module.target_generator
0092             target_generator.regist_arg('build_ohos_sdk', True)
0093             target_generator.regist_arg('build_ohos_ndk', True)
0094             if len(build_module.args_dict['build_target'].arg_value) == 0:
0095                 build_module.args_dict['build_target'].arg_value = [
0096                     'build_ohos_sdk']
0097             build_module.args_dict['target_cpu'].arg_value = 'arm64'

56~80行：将config配置对象的数据读出并记录到target_generator对象中（后续gn程序会使用到），这些config配置对象在之前hb set 处理中已设置好。
82~84行：读取产品对应的特性配置，对于rk3568来说，其在vendor/hihope/rk3568/config.json中，并记录下来
86~88行：读取并记录clang编译器的目录位置
90~97行：如果编译的产品是openharmony sdk, 则需要记录一些特殊的参数

目标CPU解析

build/hb/resolver/build_args_resolver.py

0099     @staticmethod
0100     def resolve_target_cpu(target_arg: Arg, build_module: BuildModuleInterface):
0101         """resolve '--target-cpu' arg.
0102         :param target_arg: arg object which is used to get arg value.
0103         :param build_module [maybe unused]: build module object which is used to get other services.
0104         :phase: prebuild.
0105         """
0106         config = Config()
0107         default_build_args = IoUtil.read_json_file(DEFAULT_BUILD_ARGS)
0108         if config.target_cpu == "":
0109             config.target_cpu = target_arg.arg_value
0110         elif target_arg.arg_value != default_build_args.get("target_cpu").get("argDefault"):
0111             config.target_cpu = target_arg.arg_value

target_cpu表示此代码最终编译出来后在什么cpu架构上执行。
对于./build.py -p rk3568场景
由于hb set逻辑已经设置了target_cpu所以，这里if条件不满足
110~111行：用户指定了target_cpu参数，且与默认值不同，则更新一下。

ccache参数解析

build/hb/resolver/build_args_resolver.py

0163     @staticmethod
0164     def resolve_ccache(target_arg: Arg, build_module: BuildModuleInterface):
0165         """resolve '--ccache' arg
0166         :param target_arg: arg object which is used to get arg value.
0167         :param build_module [maybe unused]: build module object which is used to get other services.
0168         :phase: prebuild.
0169         """
0170         if target_arg.arg_value:
0171             config = Config()
0172             ccache_path = find_executable('ccache')
0173             if ccache_path is None:
0174                 LogUtil.hb_warning('Failed to find ccache, ccache disabled.')
0175                 return
0176             else:
0177                 target_generator = build_module.target_generator
0178                 target_generator.regist_arg(
0179                     'ohos_build_enable_ccache', target_arg.arg_value)
0180 
0181             ccache_local_dir = os.environ.get('CCACHE_LOCAL_DIR')
0182             ccache_base = os.environ.get('CCACHE_BASE')
0183             if not ccache_local_dir:
0184                 ccache_local_dir = '.ccache'
0185             if not ccache_base:
0186                 ccache_base = os.environ.get('HOME')
0187             ccache_base = os.path.join(ccache_base, ccache_local_dir)
0188             if not os.path.exists(ccache_base):
0189                 os.makedirs(ccache_base)
0190 
0191             ccache_log_suffix = os.environ.get('CCACHE_LOG_SUFFIX')
0192             if ccache_log_suffix:
0193                 logfile = os.path.join(
0194                     ccache_base, "ccache.{}.log".format(ccache_log_suffix))
0195             else:
0196                 logfile = os.path.join(ccache_base, "ccache.log")
0197             if os.path.exists(logfile):
0198                 oldfile = os.path.join(ccache_base, '{}.old'.format(logfile))
0199                 if os.path.exists(oldfile):
0200                     os.unlink(oldfile)
0201                 os.rename(logfile, oldfile)
0202 
0203             os.environ['CCACHE_EXEC'] = ccache_path
0204             os.environ['CCACHE_LOGFILE'] = logfile
0205             os.environ['USE_CCACHE'] = '1'
0206             os.environ['CCACHE_DIR'] = ccache_base
0207             os.environ['CCACHE_UMASK'] = '002'
0208             os.environ['CCACHE_BASEDIR'] = config.root_path
0209             ccache_max_size = os.environ.get('CCACHE_MAXSIZE')
0210             if not ccache_max_size:
0211                 ccache_max_size = '50G'
0212 
0213             cmd = ['ccache', '-M', ccache_max_size]
0214 
0215             SystemUtil.exec_command(cmd, log_path=config.log_path)

ccache是一个工具，可以缓存C/C++编译过程的中间结果，这样下次编译的时候可以从中间结果继续往下执行，加快编译速度。
本段代码处理ccache功能打开或关闭的逻辑。当前此功能默认打开。
172~175行：查找ccache工具程序路径
176~179行：记录ccache功能开关
181~189行：查找或创建ccache输出文件夹
191~201行：ccache功能日志文件创建或重命名
203~208行：将ccache一些参数记入环境变量
209~215行：设置ccache的缓冲区尺寸

pycache参数解析

pycache是一个可以加快python程序性能的工具，默认是关闭的
build/hb/resolver/build_args_resolver.py

0217     @staticmethod
0218     def resolve_pycache(target_arg: Arg, build_module: BuildModuleInterface):
0219         """resolve '--enable-pycache' arg
0220         :param target_arg: arg object which is used to get arg value.
0221         :param build_module [maybe unused]: build module object which is used to get other services.
0222         :phase: prebuild.
0223         """
0224         if target_arg.arg_value:
0225             config = Config()
0226             pycache_dir = os.environ.get('CCACHE_BASE')
0227             # The default value is HOME for local users
0228             if not pycache_dir:
0229                 pycache_dir = os.environ.get('HOME')
0230             pycache_dir = os.path.join(pycache_dir, '.pycache')
0231             os.environ['PYCACHE_DIR'] = pycache_dir
0232             pyd_start_cmd = [
0233                 'python3',
0234                 '{}/build/scripts/util/pyd.py'.format(config.root_path),
0235                 '--root',
0236                 pycache_dir,
0237                 '--start',
0238             ]
0239             cmd = ['/bin/bash', '-c', ' '.join(pyd_start_cmd), '&']
0240             subprocess.Popen(cmd)

224行：如果开启了pycache功能，则执行后面的逻辑
主要目标就是执行build/scripts/util/pyd.py脚本，此脚本不做分析，读者自己阅读代码即可。

构建目标解析

操作系统编译构建可以有很多目标，如果不做指定，则会是整个操作系统镜像(非常大)，也可以指定成一些具体的软件模块(相对较小)-比如二进制程序或者共享库等。可以通过hb tool查看当前支持哪些构建目标。
build/hb/resolver/build_args_resolver.py

0113     @staticmethod
0114     @throw_exception
0115     def resolve_build_target(target_arg: Arg, build_module: BuildModuleInterface):
0116         """resolve '--build-target' arg.
0117         :param target_arg: arg object which is used to get arg value.
0118         :param build_module [maybe unused]: build module object which is used to get other services.
0119         :phase: prebuild.
0120         :raise OHOSException: when build target not exist in compiling product.
0121         """
0122         config = Config()
0123         build_executor = build_module.target_compiler
0124         target_list = []
0125         if len(target_arg.arg_value):
0126             target_list = target_arg.arg_value
0127         else:
0128             if os.getcwd() == CURRENT_OHOS_ROOT:
0129                 target_list = ['images']
0130             elif ComponentUtil.is_in_component_dir(os.getcwd()) and \
0131                     ComponentUtil.is_component_in_product(
0132                     ComponentUtil.get_component_name(os.getcwd()), Config().product):
0133                 component_name = ComponentUtil.get_component_name(os.getcwd())
0134                 LogUtil.write_log(Config().log_path, 'In the component "{}" directory,'
0135                                   'this compilation will compile only this component'.format(
0136                                       component_name),
0137                                   'warning')
0138                 target_list.append(component_name)
0139                 target_list.append(component_name + '_test')
0140             else:
0141                 component_name = ComponentUtil.get_component_name(os.getcwd())
0142                 component_name = os.path.basename(
0143                     os.getcwd()) if component_name == '' else component_name
0144                 raise OHOSException('There is no target component "{}" for the current product "{}"'
0145                                     .format(component_name, Config().product), "4001")
0146         build_executor.regist_arg('build_target', target_list)

125~126行，如果参数指明了构建目标，则直接记录下来
否则
128~129行，当前在代码根目录，则直接生成系统镜像
130~139行，如果在某组件目录，且此产品含有这个组件，则构建对应的组件
140~145行，组件不属于产品，报错提示

ninja配置参数解析

build/hb/resolver/build_args_resolver.py

0309     @staticmethod
0310     @throw_exception
0311     def resolve_ninja_args(target_arg: Arg, build_module: BuildModuleInterface):
0312         """resolve '--ninja-args' arg
0313         :param target_arg: arg object which is used to get arg value.
0314         :param build_module [maybe unused]: build module object which is used to get other services.
0315         :phase: prebuild.
0316         :raise OHOSException: when the value of the ninja parameter does not use quotation marks.
0317         """
0318         build_executor = build_module.target_compiler
0319         ninja_args_list = []
0320         for ninja_arg in target_arg.arg_value:
0321             ninja_arg = re.sub("'", "", ninja_arg)
0322             ninja_args_list.append(ninja_arg)
0323         build_executor.regist_arg('ninja_args', ninja_args_list)

主要就是把所有额外添加的单引号删除掉，回归ninja参数本来面目

全量编译开关

build/hb/resolver/build_args_resolver.py

0242     @staticmethod
0243     def resolve_full_compilation(target_arg: Arg, build_module: BuildModuleInterface):
0244         """resolve '--full-compilation' arg
0245         :param target_arg: arg object which is used to get arg value.
0246         :param build_module [maybe unused]: build module object which is used to get other services.
0247         :phase: prebuild.
0248         """
0249         if target_arg.arg_value:
0250             build_executor = build_module.target_compiler
0251             target_list = build_executor.args_dict.get('build_target', None)
0252             if isinstance(target_list, list):
0253                 target_list.append('make_all')
0254                 target_list.append('make_test')
0255             else:
0256                 build_executor.regist_arg(
0257                     'build_target', ['make_all', 'make_test'])

在已有构建目标的基础上，增加全量构建目标，即会构建整个系统镜像

preload

build/hb/modules/ohos_build_module.py

0076     def _preload(self):
0077         self._run_phase(BuildPhase.PRE_LOAD)
0078         if self.args_dict.get('fast_rebuild', None) and not self.args_dict.get('fast_rebuild').arg_value:
0079             self.preloader.run()

77行：先运行preload。
78~79行：如果开启了fast_rebuild,则会跳过preloader。
目前在buildarg json文件中没有查询到preload阶段的参数，所以，77行代码为空操作
对于preloader, 继续查看如下代码
build/hb/services/interface/preload_interface.py

0045     def run(self):
0046         self.__post_init__()
0047         self._generate_build_prop()
0048         self._generate_build_config_json()
0049         self._generate_parts_json()
0050         self._generate_parts_config_json()
0051         self._generate_build_gnargs_prop()
0052         self._generate_features_json()
0053         self._generate_syscap_json()
0054         self._generate_exclusion_modules_json()
0055         self._generate_platforms_build()
0056         self._generate_subsystem_config_json()
0057         self._generate_systemcapability_json()

可以看出，其主要作用是生成一系列的json文件，接下来逐一进行描述

preloader初始化

build/hb/services/preloader.py

0030     def __init__(self):
0031         super().__init__()
0032         self._dirs = ""
0033         self._outputs = ""
0034         self._product = ""
0035         self._os_level = ""
0036         self._target_cpu = ""
0037         self._target_os = ""
0038         self._toolchain_label = ""
0039         self._subsystem_info = {}
0040         self._all_parts = {}
0041         self._build_vars = {}
0042 
0043     def __post_init__(self):
0044         self._dirs = Dirs(self._config)
0045         self._outputs = Outputs(self._dirs.preloader_output_dir)
0046         self._product = Product(self._dirs, self._config)
0047         self._all_parts = self._product._parts
0048         self._build_vars = self._product._build_vars
0049         self._os_level = self._build_vars.get('os_level')
0050         self._target_os = self._build_vars.get('target_os')
0051         self._target_cpu = self._build_vars.get('target_cpu')
0052         self._toolchain_label = self._build_vars.get('product_toolchain_label')
0053         self._subsystem_info = self._get_org_subsystem_info()

preloader初始化时，上述2个函数会依次调用。
44行: 记录一些输入目录和文件的路径
45行：记录需要输出的一些文件路径名
46行：记录下产品相关的信息，并解析
47~53行：记录下其它信息

记录输入目录和文件

build/hb/util/preloader/preloader_process_data.py

0050 class Dirs:
0051 
0052     def __init__(self, config):
0053         self.__post_init__(config)
0054 
0055     def __post_init__(self, config):
0056         self.source_root_dir = config.root_path
0057         self.built_in_product_dir = config.built_in_product_path
0058         self.productdefine_dir = os.path.join(
0059             self.source_root_dir, 'productdefine/common')
0060         self.built_in_base_dir = os.path.join(self.productdefine_dir, 'base')
0061 
0062         # Configs of vendor specified products are stored in ${vendor_dir} directory.
0063         self.vendor_dir = config.vendor_path
0064         # Configs of device specified products are stored in ${device_dir} directory.
0065         self.device_dir = os.path.join(config.root_path, 'device')
0066 
0067         self.subsystem_config_json = os.path.join(
0068             config.root_path, config.subsystem_config_json)
0069         self.subsystem_config_overlay_json = os.path.join(config.product_path,
0070                                                           'subsystem_config_overlay.json')
0071         self.lite_components_dir = os.path.join(
0072             config.root_path, 'build/lite/components')
0073 
0074         self.preloader_output_dir = os.path.join(
0075             config.root_path, 'out/preloader', config.product)

56~75行：各种目录的记录

记录各输出文件的路径

build/hb/util/preloader/preloader_process_data.py

0025 class Outputs:
0026 
0027     def __init__(self, output_dir):
0028         self.__post_init__(output_dir)
0029 
0030     def __post_init__(self, output_dir):
0031         os.makedirs(output_dir, exist_ok=True)
0032         self.build_prop = os.path.join(output_dir, 'build.prop')
0033         self.build_config_json = os.path.join(output_dir, 'build_config.json')
0034         self.parts_json = os.path.join(output_dir, 'parts.json')
0035         self.parts_config_json = os.path.join(output_dir, 'parts_config.json')
0036         self.build_gnargs_prop = os.path.join(output_dir, 'build_gnargs.prop')
0037         self.features_json = os.path.join(output_dir, 'features.json')
0038         self.syscap_json = os.path.join(output_dir, 'syscap.json')
0039         self.exclusion_modules_json = os.path.join(output_dir,
0040                                                    'exclusion_modules.json')
0041         self.subsystem_config_json = os.path.join(output_dir,
0042                                                   'subsystem_config.json')
0043         self.subsystem_config_overlay_json = os.path.join(output_dir,
0044                                                   'subsystem_config_overlay.json')
0045         self.platforms_build = os.path.join(output_dir, 'platforms.build')
0046         self.systemcapability_json = os.path.join(
0047             output_dir, 'SystemCapability.json')

31行：创建输出目录
32~47行：记录各输出文件的路径

记录产品相关信息

build/hb/util/preloader/preloader_process_data.py

0078 class Product():
0079 
0080     def __init__(self, config_dirs: Dirs, ohos_config: Config):
0081         self._ohos_config = None
0082         self._dirs = None
0083         self._name = ""
0084         self._config = {}
0085         self._build_vars = {}
0086         self._parts = {}
0087         self._syscap_info = {}
0088         self._device_name = ""
0089         self._device_info = {}
0090         self._config_file = ""
0091         self._version = ''
0092         self.__post_init__(config_dirs, ohos_config)
0093 
0094     def __post_init__(self, config_dirs: Dirs, config: Config):
0095         self._ohos_config = config
0096         self._dirs = config_dirs
0097         self._name = config.product
0098         self._config_file = config.product_json
0099         self._config = self._get_full_product_config()
0100         self._version = self._config.get('version', '3.0')
0101         self._do_parse()
0102 
0103     # parse product configuration, then generate parts list and build vars
0104     def _do_parse(self):
0105         self._update_syscap_info()
0106         self._update_device()
0107         self._update_parts()
0108         self._update_build_vars()
0109         self._remove_excluded_components()

105~106行：解析产品信息
107行：生成部件列表信息
108行：生成build var列表
109行：移除不需要的组件

记录系统能力信息

build/hb/util/preloader/preloader_process_data.py

0114     def _update_syscap_info(self):
0115         product_name = self._config.get('product_name')
0116         if product_name is None:
0117             product_name = ""
0118         os_level = self._config.get('type')
0119         if os_level is None:
0120             os_level = ""
0121         api_version = self._config.get('api_version')
0122         if api_version is None:
0123             api_version = 0
0124         manufacturer_id = self._config.get('manufacturer_id')
0125         if manufacturer_id is None:
0126             manufacturer_id = 0
0127         self._syscap_info = {'product': product_name, 'api_version': api_version,
0128                              'system_type': os_level, 'manufacturer_id': manufacturer_id}

主要记录一些基础信息，如果没有配置，则记录成空字符串或0。

记录设备信息

build/hb/util/preloader/preloader_process_data.py

0131     def _update_device(self):
0132         if self._version == "2.0":
0133             device_name = self._config.get('product_device')
0134             if device_name:
0135                 self._device_name = device_name
0136                 self._device_info = self._get_device_info_v2(
0137                     device_name, self._dirs.built_in_device_dir)
0138         else:
0139             device_name = self._config.get('board')
0140             if device_name:
0141                 self._device_name = device_name
0142                 self._device_info = self._get_device_info_v3(self._config)
0143         if self._ohos_config.target_cpu:
0144             self._device_info["target_cpu"] = self._ohos_config.target_cpu
0145         if self._ohos_config.compile_config:
0146             self._device_info[self._ohos_config["compile_config"]] = True

132~142行：记录device name和device info
144行：记录目标CPU
145行：记录编译工具类型

生成部件信息列表

build/hb/util/preloader/preloader_process_data.py

0149     def _update_parts(self):
0150         if self._version == "1.0":
0151             _parts = {}
0152             self._parts = _parts
0153         else:
0154             # 1. inherit parts information from base config
0155             if self._version == "2.0":
0156                 os_level = self._config.get("type", "standard")
0157             else:
0158                 os_level = self._config.get("type", "mini")
0159             # 2. product config based on default minimum system
0160             based_on_mininum_system = self._config.get(
0161                 'based_on_mininum_system')
0162             if based_on_mininum_system == "true":
0163                 self._parts = self._get_base_parts(
0164                     self._dirs.built_in_base_dir, os_level)
0165             # 3. inherit parts information from inherit config
0166             inherit = self._config.get('inherit')
0167             if inherit:
0168                 self._parts.update(
0169                     self._get_inherit_parts(inherit, self._dirs.source_root_dir))
0170 
0171             # 4. chipset products relate system parts config
0172             sys_info_path = self._config.get('system_component')
0173             if sys_info_path:
0174                 sys_parts = self._get_sys_relate_parts(
0175                     sys_info_path, self._parts, self._dirs.source_root_dir)
0176                 self._parts.update(sys_parts)
0177             all_parts = {}
0178             if self._version == "2.0":
0179                 current_product_parts = self._config.get("parts")
0180                 if current_product_parts:
0181                     all_parts.update(current_product_parts)
0182             else:
0183                 all_parts.update(get_vendor_parts_list(self._config))
0184                 all_parts.update(self._get_product_specific_parts())
0185 
0186                 device_name = self._config.get('board')
0187                 if device_name:
0188                     all_parts.update(self._get_device_specific_parts())
0189             self._parts.update(all_parts)

160~164行：先生成最小系统的部件集合，目前搜索整个代码仓没有发现使用。
166~169行：从某些地方继承合并一些部件
172~176行：补充一些系统部件，目前搜索整个代码仓没有发现使用。
178~188行：补充只属于本产品的一些部件

继承合并一些其它部件集合

build/hb/util/preloader/preloader_process_data.py

0337     def _get_inherit_parts(self, inherit, source_root_dir) -> dict:
0338         inherit_parts = {}
0339         for _config in inherit:
0340             _file = os.path.join(source_root_dir, _config)
0341             _info = IoUtil.read_json_file(_file)
0342             parts = _info.get('parts')
0343             if parts:
0344                 inherit_parts.update(parts)
0345             else:
0346                 inherit_parts.update(get_vendor_parts_list(_info))
0347         return inherit_parts

这里的inherit是一个列表，每个元素描述待继承的部件描述json文件路径(相对于代码根目录)
即从其它若干产品继承部件集合。
注意：由于历史原因，软件组件早期按子系统和组件模型划分，即系统拆分成若干子系统，每个子系统下面再拆分成若干组件，后来组件术语改变成部件(不再强调子系统概念)。即子系统与组件整体变更成部件概念。所以，这里get_vendor_parts_list就是从组件信息推导出部件信息。

获取产品部件集合

build/hb/util/preloader/parse_vendor_product_config.py

0108 def transform(config):
0109     subsystems = config.get('subsystems')
0110     if subsystems:
0111         config.pop('subsystems')
0112         parts = from_ss_to_parts(subsystems)
0113         config['parts'] = parts
0114     return config

0140 def get_vendor_parts_list(config):
0141     return transform(config).get('parts')

将config.json中的子系统转换成部件并记录

获取本产品额外部件

build/hb/util/preloader/preloader_process_data.py

0361     def _get_product_specific_parts(self) -> dict:
0362         part_name = 'product_{}'.format(self._name)
0363         subsystem_name = part_name
0364         info = {}
0365         info['{}:{}'.format(subsystem_name, part_name)] = {}
0366         return info

这里得到的info[‘product_xxx:product_xxx’]

增加本设备额外部件

build/hb/util/preloader/preloader_process_data.py

0316     def _get_device_specific_parts(self) -> dict:
0317         info = {}
0318         if self._device_info and self._device_info.get('device_build_path'):
0319             subsystem_name = 'device_{}'.format(self._device_name)
0320             part_name = subsystem_name
0321             info['{}:{}'.format(subsystem_name, part_name)] = {}
0322         return info

319~321行：增加设备额外的部件

生成build var列表

build/hb/util/preloader/preloader_process_data.py

0192     def _update_build_vars(self):
0193         config = self._config
0194         build_vars = {}
0195         if self._version == "1.0":
0196             build_vars = {"os_level": 'large'}
0197         else:
0198             if self._version == "2.0":
0199                 build_vars['os_level'] = config.get("type", "standard")
0200                 device_name = config.get('product_device')
0201                 if device_name:
0202                     build_vars['device_name'] = device_name
0203                 else:
0204                     build_vars['device_name'] = ''
0205                 build_vars['product_company'] = config.get('product_company')
0206             else:
0207                 build_vars['os_level'] = config.get('type', 'mini')
0208                 build_vars['device_name'] = config.get('board')
0209                 if config.get('product_company'):
0210                     build_vars['product_company'] = config.get(
0211                         'product_company')
0212                 elif os.path.dirname(self._config_file) != self._dirs.built_in_product_dir:
0213                     relpath = os.path.relpath(
0214                         self._config_file, self._dirs.vendor_dir)
0215                     build_vars['product_company'] = relpath.split('/')[0]
0216                 else:
0217                     build_vars['product_company'] = config.get(
0218                         'device_company')
0219             build_vars['product_name'] = config.get('product_name')
0220             if 'enable_ramdisk' in config:
0221                 build_vars['enable_ramdisk'] = config.get('enable_ramdisk')
0222             if 'enable_absystem' in config:
0223                 build_vars['enable_absystem'] = config.get('enable_absystem')
0224             if 'build_selinux' in config:
0225                 build_vars['build_selinux'] = config.get('build_selinux')
0226             if 'build_seccomp' in config:
0227                 build_vars['build_seccomp'] = config.get('build_seccomp')
0228             if 'support_jsapi' in config:
0229                 build_vars['support_jsapi'] = config.get('support_jsapi')
0230             if 'chipprod_config_path' in config:
0231                 chipprod_config_path = os.path.join(
0232                     self._dirs.source_root_dir, config.get('chipprod_config_path'))
0233                 if os.path.exists(chipprod_config_path):
0234                     build_vars['chipprod_config_path'] = chipprod_config_path
0235             if 'ext_sdk_config_file' in config:
0236                 ext_sdk_config_file = os.path.join(
0237                     self._dirs.source_root_dir, config.get('ext_sdk_config_file'))
0238                 if os.path.exists(ext_sdk_config_file):
0239                     build_vars['ext_sdk_config_file'] = ext_sdk_config_file
0240             if 'ext_ndk_config_file' in config:
0241                 ext_ndk_config_file = os.path.join(
0242                     self._dirs.source_root_dir, config.get('ext_ndk_config_file'))
0243                 if os.path.exists(ext_ndk_config_file):
0244                     build_vars['ext_ndk_config_file'] = ext_ndk_config_file
0245         build_vars.update(self._device_info)
0246         if build_vars['os_level'] == 'mini' or build_vars['os_level'] == 'small':
0247             toolchain_label = ""
0248         else:
0249             toolchain_label = '//build/toolchain/{0}:{0}_clang_{1}'.format(
0250                 self._device_info.get('target_os'), self._device_info.get('target_cpu'))
0251         build_vars['product_toolchain_label'] = toolchain_label
0252         self._build_vars = build_vars

记录下各种各样的build参数

移除不需要的部件

build/hb/util/preloader/preloader_process_data.py

0255     def _remove_excluded_components(self):
0256         items_to_remove = []
0257         for part, val in self._parts.items():
0258             if "exclude" in val and val["exclude"] == "true":
0259                 items_to_remove.append(part)
0260         for item in items_to_remove:
0261             del self._parts[item]

生成build.prop

build/hb/services/preloader.py

0176     '''Description: generate build prop info to "out/preloader/product_name/build.prop"
0177     @parameter:none
0178     @return :none
0179     '''
0180 
0181     def _generate_build_prop(self):
0182         build_vars_list = []
0183         for key, value in self._build_vars.items():
0184             build_vars_list.append('{}={}'.format(key, value))
0185         with os.fdopen(os.open(self._outputs.build_prop,
0186                                os.O_RDWR | os.O_CREAT, stat.S_IWUSR | stat.S_IRUSR), 'w') as fobj:
0187             fobj.write('\n'.join(build_vars_list))
0188         LogUtil.hb_info(
0189             'generated build prop info to {}/build.prop'.format(self._dirs.preloader_output_dir))

将所有的build参数写入build.prop文件

生成build_config.json

build/hb/services/preloader.py

0165     '''Description: generate build config info to "out/preloader/product_name/build_config.json"
0166     @parameter:none
0167     @return :none
0168     '''
0169 
0170     def _generate_build_config_json(self):
0171         IoUtil.dump_json_file(
0172             self._outputs.build_config_json, self._build_vars)
0173         LogUtil.hb_info(
0174             'generated build config info to {}/build_config.json'.format(self._dirs.preloader_output_dir))

仍然是build参数，不过以json文件格式记录下来

生成部件json

build/hb/services/preloader.py

0191     '''Description: generate parts to "out/preloader/product_name/parts.json"
0192     @parameter:none
0193     @return :none
0194     '''
0195 
0196     def _generate_parts_json(self):
0197         parts_info = {"parts": sorted(list(self._all_parts.keys()))}
0198         IoUtil.dump_json_file(self._outputs.parts_json, parts_info)
0199         LogUtil.hb_info(
0200             'generated product parts info to {}/parts.json'.format(self._dirs.preloader_output_dir))

将部件列表信息按部件名排序后，写入json文件

生成部件配置json

build/hb/services/preloader.py

0202     '''Description: generate parts config to "out/preloader/product_name/parts_config.json"
0203     @parameter:none
0204     @return :none
0205     '''
0206 
0207     def _generate_parts_config_json(self):
0208         parts_config = {}
0209         for part in self._all_parts:
0210             part = part.replace(":", "_")
0211             part = part.replace("-", "_")
0212             part = part.replace(".", "_")
0213             part = part.replace("/", "_")
0214             parts_config[part] = True
0215         IoUtil.dump_json_file(self._outputs.parts_config_json, parts_config)
0216         LogUtil.hb_info(
0217             'generated parts config info to {}/parts_config.json'.format(self._dirs.preloader_output_dir))

将部件中的名称进行规范化处理后写入json文件

生成gn参数属性文件

build/hb/services/preloader.py

0075     '''Description: generate build gnargs prop info to "out/preloader/{product_name}/build_gnargs.prop"
0076     @parameter:none
0077     @return :none
0078     '''
0079 
0080     def _generate_build_gnargs_prop(self):
0081         all_features = {}
0082         for _part_name, vals in self._all_parts.items():
0083             _features = vals.get('features')
0084             if _features:
0085                 all_features.update(_features)
0086         attr_list = []
0087         for key, val in all_features.items():
0088             _item = ''
0089             if isinstance(val, bool):
0090                 _item = f'{key}={str(val).lower()}'
0091             elif isinstance(val, int):
0092                 _item = f'{key}={val}'
0093             elif isinstance(val, str):
0094                 _item = f'{key}="{val}"'
0095             else:
0096                 raise Exception("part feature '{key}:{val}' type not support.")
0097             attr_list.append(_item)
0098         with os.fdopen(os.open(self._outputs.build_gnargs_prop,
0099                                os.O_RDWR | os.O_CREAT, stat.S_IWUSR | stat.S_IRUSR), 'w') as fobj:
0100             fobj.write('\n'.join(attr_list))
0101         LogUtil.hb_info(
0102             'generated build gnargs prop info to {}/build_gnargs.prop'.format(self._dirs.preloader_output_dir))

81~85行：读出所有部件的特性集合
86~97行：对特性字符串进行规范化处理，只支持整数，布尔类型和字符串
98~102行：写入规范化后的结果

生成特性集合

build/hb/services/preloader.py

0104     '''Description: generate features to "out/preloader/{product_name}/features.json"
0105     @parameter:none
0106     @return :none
0107     '''
0108 
0109     def _generate_features_json(self):
0110         all_features = {}
0111         part_feature_map = {}
0112         for _part_name, vals in self._all_parts.items():
0113             _features = vals.get('features')
0114             if _features:
0115                 all_features.update(_features)
0116             if _features:
0117                 part_feature_map[_part_name.split(
0118                     ':')[1]] = list(_features.keys())
0119         parts_feature_info = {
0120             "features": all_features,
0121             "part_to_feature": part_feature_map
0122         }
0123         IoUtil.dump_json_file(self._outputs.features_json, parts_feature_info)
0124         LogUtil.hb_info(
0125             'generated features info to {}/features.json'.format(self._dirs.preloader_output_dir))

114~115行：汇总所有的特性
116~118行：记录每一个部件下的特性集合(部件名使用简称)
119~125行：结果写入文件

生成系统能力记录

build/hb/services/preloader.py

0127     '''Description: generate syscap to "out/preloader/product_name/syscap.json"
0128     @parameter:none
0129     @return :none
0130     '''
0131 
0132     def _generate_syscap_json(self):
0133         all_syscap = {}
0134         part_syscap_map = {}
0135         for _part_name, vals in self._all_parts.items():
0136             _syscap = vals.get('syscap')
0137             if _syscap:
0138                 all_syscap.update(_syscap)
0139                 part_syscap_map[_part_name.split(':')[1]] = _syscap
0140         parts_syscap_info = {
0141             "syscap": all_syscap,
0142             "part_to_syscap": part_syscap_map
0143         }
0144         IoUtil.dump_json_file(self._outputs.syscap_json, parts_syscap_info)
0145         LogUtil.hb_info(
0146             'generated syscap info to {}/syscap.json'.format(self._dirs.preloader_output_dir))

136~139行：获取系统能力以及组件和能力集的映射
然后写入结果

生成排除模块记录

build/hb/services/preloader.py

0148     '''Description: generate exclusion modules info to "out/preloader/product_name/exclusion_modules.json"
0149     @parameter:none
0150     @return :none
0151     '''
0152 
0153     def _generate_exclusion_modules_json(self):
0154         exclusions = {}
0155         for _part_name, vals in self._all_parts.items():
0156             _exclusions = vals.get('exclusions')
0157             if _exclusions:
0158                 pair = dict()
0159                 pair[_part_name] = _exclusions
0160                 exclusions.update(pair)
0161         IoUtil.dump_json_file(self._outputs.exclusion_modules_json, exclusions)
0162         LogUtil.hb_info(
0163             'generated exclusion modules info to {}/exclusion_modules.json'.format(self._dirs.preloader_output_dir))

将各部件排除的模块整合起来
再记录到json文件中
本能力目前在各产品还没有看见使用。

生成子系统配置json

build/hb/services/preloader.py

0219     '''Description: generate subsystem config info to "out/preloader/product_name/subsystem_config.json"
0220     @parameter:none
0221     @return :none
0222     '''
0223 
0224     def _generate_subsystem_config_json(self):
0225         if self._subsystem_info:
0226             self._subsystem_info.update(
0227                 self._product._get_product_specific_subsystem())
0228             self._subsystem_info.update(
0229                 self._product._get_device_specific_subsystem())
0230         IoUtil.dump_json_file(
0231             self._outputs.subsystem_config_json, self._subsystem_info)
0232         LogUtil.hb_info(
0233             'generated subsystem config info to {}/subsystem_config.json'.format(self._dirs.preloader_output_dir))

汇总设备和产品的配置信息，然后记录到json文件中

生成系统能力配置json

build/hb/services/preloader.py

0235     '''Description: generate systemcapability_json to "out/preloader/product_name/systemcapability.json"
0236     @parameter:none
0237     @return :none
0238     '''
0239 
0240     def _generate_systemcapability_json(self):
0241         IoUtil.dump_json_file(
0242             self._outputs.systemcapability_json, self._product._syscap_info)
0243         LogUtil.hb_info(
0244             'generated system capability info to {}/systemcapability.json'.format(self._dirs.preloader_output_dir))

写入当前的系统能力信息

load

build/hb/modules/ohos_build_module.py

0081     def _load(self):
0082         self._run_phase(BuildPhase.LOAD)
0083         if self.args_dict.get('fast_rebuild', None) and not self.args_dict.get('fast_rebuild').arg_value:
0084             self.loader.run()

82行：先支持LOAD阶段的处理
这些处理有

• resolve_strict_mode
• resolve_scalable_build
• resolve_build_example
• resolve_build_platform_name
• resolve_build_xts
• resolve_ignore_api_check
• resolve_load_test_config

82~84行：在没有开启fast_rebuild的情况下，还要运行loader

严格模式

build/hb/resolver/build_args_resolver.py

0327     def resolve_strict_mode(target_arg: Arg, build_module: BuildModuleInterface):
0328         """resolve '--strict-mode' arg.
0329         :param target_arg: arg object which is used to get arg value.
0330         :param build_module [maybe unused]: build module object which is used to get other services.
0331         :phase: load.
0332         :raise OHOSException: when preloader or loader results not correct
0333         """
0334         if target_arg.arg_value:
0335             preloader = build_module.preloader
0336             loader = build_module.loader
0337             if not preloader.outputs.check_outputs():
0338                 raise OHOSException('Preloader result not correct', "1001")
0339             if not loader.outputs.check_outputs():
0340                 raise OHOSException('Loader result not correct ', "2001")

目前这2个check_outputs函数没有看到定义的地方
这个命令行参数当前不可用，后续可能会支持

增量构建

build/hb/resolver/build_args_resolver.py

0343     def resolve_scalable_build(target_arg: Arg, build_module: BuildModuleInterface):
0344         """resolve '--scalable-build' arg.
0345         :param target_arg: arg object which is used to get arg value.
0346         :param build_module [maybe unused]: build module object which is used to get other services.
0347         :phase: load.
0348         """
0349         loader = build_module.loader
0350         loader.regist_arg("scalable_build", target_arg.arg_value)

记录增量构建开关

构建范例

build/hb/resolver/build_args_resolver.py

0353     def resolve_build_example(target_arg: Arg, build_module: BuildModuleInterface):
0354         """resolve '--build-example' arg.
0355         :param target_arg: arg object which is used to get arg value.
0356         :param build_module [maybe unused]: build module object which is used to get other services.
0357         :phase: load.
0358         """
0359         loader = build_module.loader
0360         loader.regist_arg("build_example", target_arg.arg_value)

记录build_example开关

构建平台名称

build/hb/resolver/build_args_resolver.py

0363     def resolve_build_platform_name(target_arg: Arg, build_module: BuildModuleInterface):
0364         """resolve '---build-platform-name' arg
0365         :param target_arg: arg object which is used to get arg value.
0366         :param build_module [maybe unused]: build module object which is used to get other services.
0367         :phase: load.
0368         """
0369         loader = build_module.loader
0370         loader.regist_arg("build_platform_name", target_arg.arg_value)

记录构建平台的名称

构建测试用例

build/hb/resolver/build_args_resolver.py

0373     def resolve_build_xts(target_arg: Arg, build_module: BuildModuleInterface):
0374         """resolve '--build-xts' arg
0375         :param target_arg: arg object which is used to get arg value.
0376         :param build_module [maybe unused]: build module object which is used to get other services.
0377         :phase: load.
0378         """
0379         loader = build_module.loader
0380         for gn_arg in build_module.args_dict['gn_args'].arg_value:
0381             if 'build_xts' in gn_arg:
0382                 variable, value = gn_arg.split('=')
0383                 if str(value).lower() == 'false':
0384                     value = False
0385                 elif str(value).lower() == 'true':
0386                     value = True
0387                 loader.regist_arg(variable, value)
0388                 return
0389         loader.regist_arg("build_xts", target_arg.arg_value)

如果在gn_args参数里面配置了build_xts, 则优先使用gn_args里面的开关
否则使用外面这个开关

忽略API检查

build/hb/resolver/build_args_resolver.py

0392     def resolve_ignore_api_check(target_arg: Arg, build_module: BuildModuleInterface):
0393         """resolve '--ignore-api-check' arg
0394         :param target_arg: arg object which is used to get arg value.
0395         :param build_module [maybe unused]: build module object which is used to get other services.
0396         :phase: load.
0397         """
0398         loader = build_module.loader
0399         if len(target_arg.arg_value):
0400             loader.regist_arg("ignore_api_check", target_arg.arg_value)
0401         else:
0402             loader.regist_arg("ignore_api_check", [
0403                               'xts', 'common', 'developertest'])

记录用户指定的忽略api检查的部件集合
如果用户没有指定，默认有3个部件不检查API

加载测试配置

build/hb/resolver/build_args_resolver.py

0406     def resolve_load_test_config(target_arg: Arg, build_module: BuildModuleInterface):
0407         """resolve '--load-test-config' arg
0408         :param target_arg: arg object which is used to get arg value.
0409         :param build_module [maybe unused]: build module object which is used to get other services.
0410         :phase: load.
0411         """
0412         loader = build_module.loader
0413         loader.regist_arg("load_test_config", target_arg.arg_value)

记录下对应开关值

loader

build/hb/services/interface/load_interface.py

0044     def run(self):
0045         self.__post_init__()
0046         self._execute_loader_args_display()
0047         self._check_parts_config_info()
0048         self._generate_subsystem_configs()
0049         self._generate_target_platform_parts()
0050         self._generate_system_capabilities()
0051         self._generate_stub_targets()
0052         self._generate_platforms_part_by_src()
0053         self._generate_target_gn()
0054         self._generate_phony_targets_build_file()
0055         self._generate_required_parts_targets()
0056         self._generate_required_parts_targets_list()
0057         self._generate_src_flag()
0058         self._generate_auto_install_part()
0059         self._generate_platforms_list()
0060         self._generate_part_different_info()
0061         self._generate_infos_for_testfwk()
0062         self._check_product_part_feature()
0063         self._generate_syscap_files()

loader初始化

build/hb/services/loader.py

0058     def __post_init__(self):
0059         self.source_root_dir = self.config.root_path + '/'
0060         self.gn_root_out_dir = self.config.out_path if not self.config.out_path.startswith(
0061             '/') else os.path.relpath(self.config.out_path, self.config.root_path)
0062         self.os_level = self.config.os_level if self.config.os_level else "standard"
0063         self.target_cpu = self.config.target_cpu if self.config.target_cpu else "arm"
0064         self.target_os = self.config.target_os if self.config.target_os else "ohos"
0065         self.config_output_relpath = os.path.join(
0066             self.gn_root_out_dir, 'build_configs')
0067         self.config_output_dir = os.path.join(
0068             self.source_root_dir, self.config_output_relpath)
0069         self.target_arch = '{}_{}'.format(self.target_os, self.target_cpu)
0070         self.subsystem_config_file = os.path.join(
0071             self.config.root_path, 'out/preloader', self.config.product, 'subsystem_config.json')
0072         self.platforms_config_file = os.path.join(
0073             self.config.root_path, 'out/preloader', self.config.product, 'platforms.build')
0074         self.exclusion_modules_config_file = os.path.join(
0075             self.config.root_path, 'out/preloader', self.config.product, 'exclusion_modules.json')
0076         self.example_subsystem_file = os.path.join(
0077             self.config.root_path, 'build', 'subsystem_config_example.json')
0078 
0079         # check config args
0080         self._check_args()
0081 
0082         self.build_example = self.args_dict.get('build_example')
0083         if not self.build_example:
0084             self.example_subsystem_file = ""
0085         self.scalable_build = self.args_dict.get('scalable_build')
0086         self.build_platform_name = self.args_dict.get('build_platform_name')
0087         self.build_xts = self.args_dict.get('build_xts')
0088         self.ignore_api_check = self.args_dict.get('ignore_api_check')
0089         self.load_test_config = self.args_dict.get('load_test_config')
0090         self.subsystem_configs = subsystem_scan.scan(self.subsystem_config_file,
0091                                                      self.example_subsystem_file,
0092                                                      self.source_root_dir)
0093 
0094         self._subsystem_info = subsystem_info.get_subsystem_info(
0095             self.subsystem_config_file,
0096             self.example_subsystem_file,
0097             self.source_root_dir,
0098             self.config_output_relpath,
0099             self.os_level)
0100         overrided_components = self._override_components()
0101 
0102         self._platforms_info = platforms_loader.get_platforms_info(
0103             self.platforms_config_file,
0104             self.source_root_dir,
0105             self.gn_root_out_dir,
0106             self.target_arch,
0107             self.config_output_relpath,
0108             self.scalable_build)
0109         self.variant_toolchains = self._platforms_info.get(
0110             'variant_toolchain_info').get('platform_toolchain')
0111         self._all_platforms = self.variant_toolchains.keys()
0112         self.build_platforms = self._get_build_platforms()
0113         self.parts_config_info = load_ohos_build.get_parts_info(
0114             self.source_root_dir,
0115             self.config_output_relpath,
0116             self._subsystem_info,
0117             self.variant_toolchains,
0118             self.target_arch,
0119             self.ignore_api_check,
0120             self.exclusion_modules_config_file,
0121             self.load_test_config,
0122             overrided_components,
0123             self.build_xts)
0124         self.parts_targets = self.parts_config_info.get('parts_targets')
0125         self.phony_targets = self.parts_config_info.get('phony_target')
0126         self.parts_info = self.parts_config_info.get('parts_info')
0127         self.target_platform_parts = self._get_platforms_all_parts()
0128         self.target_platform_stubs = self._get_platforms_all_stubs()
0129         self.required_parts_targets_list = self._get_required_build_parts_list()
0130         self.required_phony_targets = self._get_required_phony_targets()
0131         self.required_parts_targets = self._get_required_build_targets()

59~89行：记录本阶段解析好的命令行参数，以及一些文件的路径
90~92行：扫描subsystem_config.json和subsystem_config_example.json内的所有subsystem，含ohos.build或bundle.json文件的，即为合法的部件，否则为不参与编译的源码。
94~99行：将扫描出的subsystem写入json文件，并返回扫描结果
out/productname/build_configs/subsystem_info/src_subsystem_info.json
out/productname/build_configs/subsystem_info/no_src_subsystem_info.json
out/productname/build_configs/subsystem_info/subsystem_build_config.json
100行：处理被替换的部件, 厂商自己实现的部件替换掉系统默认实现的部件
102~108行：获取平台信息
109~110行：获取平台对应的工具链信息
111行：获取本次编译的所有平台，当前只支持phone
112行：获取本次构建的品台，仅支持phone
113~123行：获取ohos.build两个的部件信息
124~125行：记录部件编译目标和伪目标
126~131行：记录相关的需要构建的目标

扫描已知子系统中的部件

build/hb/util/loader/subsystem_scan.py

0075 def scan(subsystem_config_file, example_subsystem_file, source_root_dir):
0076     subsystem_infos = _read_config(subsystem_config_file,
0077                                    example_subsystem_file)
0078     # add common subsystem info
0079     subsystem_infos.update(_default_subsystem)
0080 
0081     no_src_subsystem = {}
0082     _build_configs = {}
0083     for key, val in subsystem_infos.items():
0084         _all_build_config_files = []
0085         if not isinstance(val, list):
0086             val = [val]
0087         else:
0088             if not _check_path_prefix(val):
0089                 raise OHOSException(
0090                     "subsystem '{}' path configuration is incorrect.".format(
0091                         key), "2013")
0092         _info = {'path': val}
0093         for _path in val:
0094             _subsystem_path = os.path.join(source_root_dir, _path)
0095             _build_config_files = _scan_build_file(_subsystem_path)
0096             _all_build_config_files.extend(_build_config_files)
0097         if _all_build_config_files:
0098             _info['build_files'] = _all_build_config_files
0099             _build_configs[key] = _info
0100         else:
0101             no_src_subsystem[key] = val
0102 
0103     scan_result = {
0104         'source_path': source_root_dir,
0105         'subsystem': _build_configs,
0106         'no_src_subsystem': no_src_subsystem
0107     }
0108     LogUtil.hb_info('subsytem config scan completed')
0109     return scan_result

76行：读取已知子系统的名称和路径列表
79行：添加common子系统
83行：遍历所有子系统
85~86行: path只有一项，直接存入
87~91行：path是一个列表，那么列表中最多只能含一个(device或vendor前缀)的路径
93~96行：遍历path列表，在这些路径中找出软件包配置文件
97~99行：记录下这些配置文件
100~101行：无软件包的子系统也记录一下
103~109行：记录结果并返回

根据子系统路径配置读取子系统信息

build/hb/util/loader/subsystem_scan.py

0028 def _read_config(subsystem_config_file, example_subsystem_file):
0029     if not os.path.exists(subsystem_config_file):
0030         raise OHOSException(
0031             "config file '{}' doesn't exist.".format(subsystem_config_file), "2013")
0032     subsystem_config = read_json_file(subsystem_config_file)
0033     if subsystem_config is None:
0034         raise OHOSException("read file '{}' failed.".format(
0035             subsystem_config_file), "2013")
0036 
0037     # example subsystem
0038     if example_subsystem_file:
0039         example_subsystem_config = read_json_file(example_subsystem_file)
0040         if example_subsystem_config is not None:
0041             subsystem_config.update(example_subsystem_config)
0042 
0043     subsystem_info = {}
0044     for key, val in subsystem_config.items():
0045         if 'path' not in val:
0046             raise OHOSException(
0047                 "subsystem '{}' not config path.".format(key), "2013")
0048         subsystem_info[key] = val.get('path')
0049     return subsystem_info

48行：将每个子系统的名称和路径记录到新的字典并返回

检查非法路径

build/hb/util/loader/subsystem_scan.py

0066 def _check_path_prefix(paths):
0067     allow_path_prefix = ['vendor', 'device']
0068     result = list(
0069         filter(lambda x: x is False,
0070                map(lambda p: p.split('/')[0] in allow_path_prefix, paths)))
0071     return len(result) <= 1

在路径列表中，最多只能出现一次不是device或vendor开头的路径
map表示针对对每条路径按/切分获取首个字符串，并判断字符串是否device或vendor
filter表示，将map计算结果为False的进行汇总
最终结果就是路径列表中允许出现device和vendor开头，其它开头的最多只能一条。

寻找部件

build/hb/util/loader/subsystem_scan.py

0052 def _scan_build_file(subsystem_path):
0053     _files = []
0054     _bundle_files = []
0055     for root, dirs, files in os.walk(subsystem_path):
0056         for name in files:
0057             if name == 'ohos.build':
0058                 _files.append(os.path.join(root, name))
0059             elif name == 'bundle.json':
0060                 _bundle_files.append(os.path.join(root, name))
0061     if _bundle_files:
0062         _files.extend(_bundle_files)
0063     return _files

遍历子系统目录，寻找obos.build或bundle.json文件，返回文件路径集合。

获取存在部件的子系统列表

build/hb/util/loader/subsystem_info.py

0046 def get_subsystem_info(subsystem_config_file, example_subsystem_file,
0047                        source_root_dir, config_output_path, os_level):
0048     if not subsystem_config_file:
0049         subsystem_config_file = 'build/subsystem_config.json'
0050 
0051     subsystem_configs = {}
0052     output_dir_realpath = os.path.join(source_root_dir, config_output_path)
0053     subsystem_configs = subsystem_scan.scan(subsystem_config_file,
0054                                             example_subsystem_file,
0055                                             source_root_dir)
0056     config = Config()
0057     subsystem_config_overlay_file = os.path.join(
0058         config.product_path, "subsystem_config_overlay.json")
0059     if os.path.isfile(subsystem_config_overlay_file):
0060         subsystem_config_overlay = {}
0061         subsystem_config_overlay = subsystem_scan.scan(subsystem_config_overlay_file,
0062                                                        example_subsystem_file,
0063                                                        source_root_dir)
0064         subsystem_configs['subsystem'].update(
0065             subsystem_config_overlay['subsystem'])
0066         subsystem_configs['no_src_subsystem'].update(
0067             subsystem_config_overlay['no_src_subsystem'])
0068     _output_subsystem_configs(output_dir_realpath, subsystem_configs)
0069     return subsystem_configs.get('subsystem')

57~67行：如果存在overlay配置，则需要将overlay相关子系统也考虑进来
68行：保存查找到的部件信息
69行：返回部件信息

保存部件信息

build/hb/util/loader/subsystem_info.py

0024 def _output_subsystem_configs(output_dir, subsystem_configs):
0025     build_config_file_name = "subsystem_build_config.json"
0026     build_config_file = os.path.join(output_dir, 'subsystem_info',
0027                                      build_config_file_name)
0028     write_json_file(build_config_file, subsystem_configs)
0029 
0030     src_output_file_name = "src_subsystem_info.json"
0031     no_src_output_file_name = "no_src_subsystem_info.json"
0032 
0033     src_subsystem = {}
0034     for key, val in subsystem_configs.get('subsystem').items():
0035         src_subsystem[key] = val.get('path')
0036     src_output_file = os.path.join(output_dir, 'subsystem_info',
0037                                    src_output_file_name)
0038     write_json_file(src_output_file, src_subsystem)
0039 
0040     no_src_output_file = os.path.join(output_dir, 'subsystem_info',
0041                                       no_src_output_file_name)
0042     write_json_file(no_src_output_file,
0043                     subsystem_configs.get('no_src_subsystem'))

25~28行：保存完整部件信息
33~38行：保存含部件的子系统路径映射表
40~43行：保存不含部件的子系统列表

替换冗余部件

build/hb/services/loader.py

0794     def _override_components(self):
0795         '''Description: Check whether there are components that need to be replaced, and if so, 
0796             replace the component configuration file bundle.json in subsystem_info and update 
0797             the component list generated by the preloader.
0798         @parameter:none
0799         @return :overrided_components
0800         '''
0801         parts_file = self.platforms_config_file.replace(
0802             "platforms.build", "parts.json")
0803         all_parts = read_json_file(parts_file)
0804         if "parts" not in all_parts:
0805             LogUtil.hb_warning("{} does not contain parts!".format(parts_file))
0806             return {}
0807         overrided = False
0808         overrided_components = {}
0809         all_parts = all_parts["parts"]
0810         component_override_map = {}
0811         for subsystem_name, build_config_info in self._subsystem_info.items():
0812             if "build_files" not in build_config_info:
0813                 continue
0814 
0815             # scan all bundle.json or ohos.build files with named groups
0816             for build_file in build_config_info["build_files"]:
0817 
0818                 # ohos.build does not support overrided components
0819                 if not build_file.endswith('bundle.json'):
0820                     continue
0821 
0822                 # Only device or vendor components can do named groups extensions
0823                 if (not build_file.startswith(self.source_root_dir + 'device/')) \
0824                         and (not build_file.startswith(self.source_root_dir + 'vendor/')):
0825                     continue
0826 
0827                 # "subsystem", "name" and "override" is required
0828                 component = read_json_file(build_file).get("component")
0829 
0830                 if (not component) or (not all(key in component for key in ("subsystem", "name", "override"))):
0831                     continue
0832 
0833                 full_part_name = f"{component.get('subsystem')}:{component.get('name')}"
0834                 if full_part_name not in all_parts:
0835                     LogUtil.hb_warning("{} was not configured for this product: {}".format(
0836                         build_file, full_part_name))
0837                     continue
0838 
0839                 if self._override_one_component(self._subsystem_info, component, build_file, all_parts, overrided_components, component_override_map):
0840                     overrided = True
0841 
0842         if overrided:
0843             # Update parts.json and parts_config.json generated by preloader
0844             write_json_file(parts_file, {"parts": all_parts})
0845             parts_file = self.platforms_config_file.replace(
0846                 "platforms.build", "parts_config.json")
0847             self._output_parts_config_json(all_parts, parts_file)
0848 
0849         write_json_file(
0850             f"{self.config_output_dir}/component_override_map.json", component_override_map)
0851         return overrided_components

801~810行：读取部件列表
811行：遍历子系统
816行：遍历子系统中部件列表
819~820行：只处理bundle.json描述的部件
823~825行：只处理device或vendor开头的部件
828~831行：部件中必须包含subsystem， name, override属性
833~837行：子系统与组件名和部件名需匹配
839~840行：移除被本组件替换(覆盖)的组件
842~850行：更新相关文件，并记录下覆盖关系
851行：返回被覆盖的部件集合

替换一个部件

build/hb/services/loader.py

0853     def _override_one_component(self, subsystem_info, component, build_file, all_parts, overrided_components, component_override_map):
0854         '''Description: Perform a replacement of a single component and return the component list update result.
0855         @parameter:subsystem_info, component, build_file, all_parts, overrided_components
0856         @return :True or False(Whether replacement has been performed)
0857         '''
0858         splits = component["override"].split(":")
0859         if len(splits) != 2:
0860             LogUtil.hb_warning(
0861                 "{} override value is invalid format. Skip override process".format(build_file))
0862             return False
0863         overrided_subsystem = splits[0]
0864         overrided_component = splits[1]
0865         if overrided_subsystem not in subsystem_info:
0866             LogUtil.hb_warning(
0867                 "{} override invalid subsystem. Skip override process".format(build_file))
0868             return False
0869 
0870         founded_bundle = ""
0871 
0872         for bundle in subsystem_info[overrided_subsystem]["build_files"]:
0873             if not bundle.endswith('bundle.json'):
0874                 continue
0875 
0876             bundle_obj = read_json_file(bundle)
0877 
0878             if bundle_obj.get("component", {}).get("name") == overrided_component:
0879                 founded_bundle = bundle
0880                 break
0881 
0882         if founded_bundle:
0883             origin_component = read_json_file(build_file).get('component')
0884             LogUtil.hb_warning(
0885                 f"You are trying to override \"{component['override']}\" with \"{origin_component.get('subsystem')}:{origin_component.get('name')}\". \nPlease ensure that the modules in \"{component['override']}\" only rely on the interfaces of other components through \"external_deps\"")
0886 
0887             # replace bundle.json in subsystem_info's build_files
0888             subsystem_info[overrided_subsystem]["build_files"].remove(
0889                 founded_bundle)
0890 
0891             # Update parts.json generated by preloader, which means that new added components will not be installed
0892             # Ensure that the overrided components will be installed
0893             full_partname = f"{overrided_subsystem}:{overrided_component}"
0894             if full_partname in all_parts:
0895                 all_parts.remove(full_partname)
0896 
0897             overrided_components[f"{component['subsystem']}:{component['name']}"] = {
0898                 'subsystem': overrided_subsystem,
0899                 'partName': overrided_component
0900             }
0901             component_override_map[overrided_component] = component["name"]
0902             return True
0903         LogUtil.hb_warning(
0904             "{}:{} is not configured in product, \new add component will be installed!".format(
0905                 overrided_subsystem, overrided_component))
0906         return False

858~864行：解析被替换的子系统和组件名
865~868行：子系统不存在
872行：遍历子系统中的包
873~874行：只支持bundle.json类型的包
878~880行：找到匹配的包
883~885行：获取原始组件，并输出覆盖动作日志
888~889行：移除被覆盖的bundle.json路径
893~895行：移除对应的部件名
897~900行：记录下部件覆盖关系–覆盖了谁
901行：记录下部件覆盖关系–被谁覆盖

获取平台信息

build/hb/util/loader/platforms_loader.py

0183 def get_platforms_info(platforms_config_file, source_root_dir, root_build_dir,
0184                        target_arch, config_output_relpath, scalable_build):
0185     platform_loader = PlatformsLoader(platforms_config_file, source_root_dir,
0186                                       root_build_dir, target_arch,
0187                                       scalable_build)
0188 
0189     platforms_info_output_dir = 'platforms_info'
0190     all_parts = platform_loader.get_all_parts()
0191     all_parts_file = os.path.join(source_root_dir, config_output_relpath,
0192                                   platforms_info_output_dir, "all_parts.json")
0193     write_json_file(all_parts_file, all_parts)
0194     LogUtil.hb_info(
0195         "generate all parts of platforms info to '{}'".format(all_parts_file))
0196 
0197     # variant to toolchain and toolchain to variant
0198     toolchain_to_variant_dict = platform_loader.platforms_toolchain()
0199     toolchain_variant_info_file = os.path.join(source_root_dir,
0200                                                config_output_relpath,
0201                                                platforms_info_output_dir,
0202                                                "toolchain_to_variant.json")
0203     write_json_file(toolchain_variant_info_file,
0204                     toolchain_to_variant_dict,
0205                     check_changes=True)
0206     LogUtil.hb_info("generate toolchain to variant of platforms info to '{}'".format(
0207         toolchain_variant_info_file))
0208 
0209     result = {}
0210     result['all_parts'] = all_parts
0211     result['all_stubs'] = platform_loader.get_all_stubs()
0212     result['variant_toolchain_info'] = toolchain_to_variant_dict
0213     return result

189~196行：将所有部件信息换一种形式存储起来
198~207行：将平台与工具链之间的映射信息存储起来
209~213行：记录并返回上述信息

获取平台和工具链的映射

build/hb/util/loader/platforms_loader.py

0168     def platforms_toolchain(self):
0169         self._loading()
0170         platform_toolchain = {}
0171         toolchain_platform = {}
0172         for key, val in self._platforms_info.items():
0173             _toolchain = val.get('toolchain')
0174             platform_toolchain[key] = _toolchain
0175             toolchain_platform[_toolchain] = key
0176         _result = {
0177             "platform_toolchain": platform_toolchain,
0178             "toolchain_platform": toolchain_platform
0179         }
0180         return _result

174行：从平台到工具链的映射
175行：从工具链到平台的映射
177~178行：记录下上述2个映射表

显示loader参数

检查部件配置信息

生成子系统配置

生成目标平台部件

生成系统能力

生成桩目标

基于源码生成平台部件

生成目标gn

生成伪目标构建文件

生成必要的部件目标

生成必要的部件目标列表

生成源码对应的标识

生成自动安装部件

生成平台列表

生成部件差异化信息

生成测试框架信息

检查产品部件特性

生成系统能力文件

gn前处理

gn处理

构建类型

日志等级

测试集选择

gn参数

gn flag

快速重建

gn执行

gn后处理

ninja前处理

ninja处理

ninja错误后继续

只生成gn

ninja执行

ninja后处理

root或user构建

设备类型

打包镜像

rom尺寸统计

ccache性能度量

汇总编译警告

ninja性能度量

计算overlap rate

依赖分析清理

postbuild

清理构建参数

未完待续