[Mark] pyinstaller打包含有transformers依赖程序时出现PackageNotFoundError

背景

最近有一个项目在使用到了transformers这个包，因为加密需要所以使用了pyinstaller对程序打了包，程序执行的时候发现了以下报错：

Traceback (most recent call last):
  File "transformers/utils/versions.py", line 105, in require_version
  File "importlib_metadata/__init__.py", line 994, in version
  File "importlib_metadata/__init__.py", line 967, in distribution
  File "importlib_metadata/__init__.py", line 561, in from_name

importlib_metadata.PackageNotFoundError: No package metadata was found for XXX

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "service/server.py", line 6, in 
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "model/tagging/infer_handler.py", line 1, in 
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "transformers/__init__.py", line 43, in 
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "transformers/dependency_versions_check.py", line 41, in 
  File "transformers/utils/versions.py", line 120, in require_version_core
  File "transformers/utils/versions.py", line 108, in require_version

importlib_metadata.PackageNotFoundError: No package metadata was found for The 'XXX' distribution was not found and is required by this application. 

Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git master

解决方法

For every package listed in pkgs_to_check_at_runtime (except python), you will need to collect metadata, by using copy_metadata(name) in your spec file.
For quick prototyping, you can also use --copy-metadata command-line switch (i.e., add --copy-metadata tqdm --copy-metadata regex --copy-metadata sacremoses ... to your PyInstaller command-line).
And I seem to recall that at least one of those packages also requires a --collect-data (or collect_data_files() in spec).

原帖 重点的部分我进行了加粗，参考原帖的思路为采用修改 xxx.spec (pyinstall生成的打包文件)解决这个问题

1. 找到需要使用copy_metadata方法告知pyinstaller需要添加的数据文件
   在你程序对应的python环境中查看需要拷贝的数据文件的包有哪些

   from transformers.dependency_versions_check import pkgs_to_check_at_runtime
   print(pkgs_to_check_at_runtime)
   

   out:

   ['python', 'tqdm', 'regex', 'sacremoses', 'requests', 'packaging', 'filelock', 'numpy', 'tokenizers', 'importlib_metadata']

2. 修改. spec文件

   ...
   
   from PyInstaller.utils.hooks import 
   collect_data_files, copy_metadata
   
   ...
   
   # tqdm issue for https://github.com/pyinstaller/pyinstaller/discussions/6033
   datas = copy_metadata('tqdm')
   datas += copy_metadata('torch')
   datas += copy_metadata('regex')
   datas += copy_metadata('sacremoses')
   datas += copy_metadata('requests')
   datas += copy_metadata('packaging')
   datas += copy_metadata('filelock')
   datas += copy_metadata('numpy')
   datas += copy_metadata('tokenizers')
   datas += copy_metadata('importlib_metadata')
   
   ...
   
   a = Analysis(...
            datas=datas,
            ...)
   ...

   希望能帮助你解决问题！