python 打包 太大 精简_极简 Python 打包指南

本文由 qbit 整理自《pytest 测试实战·附录D 打包和发布 Python 项目》·Brian Okken 著·华中科技大学出版社

qbit 的实验环境

Windows 10 x64

Python 3.6.7 x64

项目的打包和发布很重要。大部分 Python 开发者对这一块并不熟悉，实际上，我们需要严肃地看待这个问题。毕竟，共享代码也是 Python 开发工作的一部分。因此，合理地使用 Python 内置的工具开共享代码很重要。虽然这是一个很大的话题，但由于篇幅的限制我无法全面介绍。这里只介绍常规的共享代码的方法。掌握这些方法后，至少你不必再用电子邮件发送压缩的文件和模块了。

我将介绍如何设置项目让它可以通过 pip 安装；如何以源代码形式发布项目；如何将项目打包成 wheel 文件。这些技巧足以让你在小型团队内部共享代码。如果你还希望通过 PyPI 将代码共享到互联网上，请阅读我推荐的文档。现在让我们开始吧。

创建可安装的模块

Creating an Installable Module

先学习如何让一个小项目可以被 pip 安装。我们以一个只有单一模块的项目为例。实际项目通常不会这么简单，我选择这个示例只是为了展示如何创建一个可维护的项目，以及 setup.py 文件可以多么简单。下面是一个简单的目录结构：

some_module_proj

|--setup.py

|--some_module.py

我们想要共享的代码在 some_molule.py 文件里：

# D:\Python3Project\test\some_module_proj\some_module.py

def some_func():

return 42

要使其能被 pip 安装，我们需要一个 setup.py 文件。下面是最简洁的 setup.py 代码：

# D:\Python3Project\test\some_module_proj\setup.py

from setuptools import setup

setup(

name='some_module',

py_modules=['some_module']

)

一个目录、一个模块、一个 setup.py 文件，就足以使项目能够被 pip 安装了。

D:\Python3Project\test

λ pip3 install .\some_module_proj\

Processing d:\python3project\test\some_module_proj

Installing collected packages: some-module

Running setup.py install for some-module ... done

Successfully installed some-module-0.0.0

现在可以在 Python 程序里(或者从测试用例里)使用 some_molule 了。

D:\Python3Project>python

Python 3.6.7 (v3.6.7:6ec5cf24b7, Oct 20 2018, 13:35:33) [MSC v.1900 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.

>>> from some_module import some_func

>>> some_func()

42

>>> exit()

这是一种理想化的情况。在实际工作中，更常见的情况是将项目进行打包。下一节将介绍如何修改 setup.py 文件来完成项目打包。

创建可安装的包

Creating an Installable Package

先创建一个以包名名命的目录，然后将 __init__.py 文件和相关模块一同放入该目录里。

some_package_proj

|--setup.py

|--src

|--some_package

|--__init__.py

|--some_module.py

some_module.py 文件的内容不变。__init__.py 文件需要把模块的功能通过包命名空间暴露给外部。有很多方式可以做到这一点，请阅读 Python 文档中关于该主题的两节内容(https://docs.python.org/3/tutorial/modules.html#packages)。

如果在 __init__.py 文件里这样写：

import some_package.some_module

那么调用端的代码必须指明 some_module。

import some_package

some_package.some_module.some_func()

但是我认为 some_module.py 是 API 功能的一部分，应该提供包这一层级的信息。所以，我们应该这样写：

# D:\Python3Project\test\some_package_proj\src\some_package\__init__.py

from some_package.some_module import *

# or

from .some_module import *

现在调用端代码可以这样写了：

import some_package

some_package.some_func()

还需要对 setup.py 文件稍作修改。

# D:\Python3Project\test\some_package_proj\setup.py

from setuptools import setup, find_packages

setup(

name='some_package',

packages=find_packages(where='src'),

package_dir={'': 'src'},

)

以后再调用就不需要提到 py_modules 了，只需指明包。现在它可以被安装了。

D:\Python3Project\test

λ pip3 install .\some_package_proj

Processing d:\python3project\test\some_package_proj

Installing collected packages: some-package

Running setup.py install for some-package ... done

Successfully installed some-package-0.0.0

而且可以直接使用。

D:\Python3Project\test\some_package_proj\src>python

Python 3.6.7 (v3.6.7:6ec5cf24b7, Oct 20 2018, 13:35:33) [MSC v.1900 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.

>>> from some_package import some_func

>>> some_func()

42

项目可以被安装了，而且很容易进行模块调用。你可以在与 src 同级的目录中添加 tests 目录并且放入测试用例。但 setup.py 文件还缺少重要内容，它还不能正常发布源码或创建 wheel 文件。不过，我们只需要略作修改即可。

创建源码发布包和 Wheel 文件

Creating a Source Distribution and Wheel

如果只是个人使用，上一节介绍的方法足以创建源码发布包和 wheel 文件了。下面来试试。

打包 wheel 格式需先安装 wheel 包

pip3 install wheel

λ python3 setup.py sdist bdist_wheel

running sdist

running egg_info

creating src\some_package.egg-info

writing src\some_package.egg-info\PKG-INFO

writing dependency_links to src\some_package.egg-info\dependency_links.txt

writing top-level names to src\some_package.egg-info\top_level.txt

writing manifest file 'src\some_package.egg-info\SOURCES.txt'

reading manifest file 'src\some_package.egg-info\SOURCES.txt'

writing manifest file 'src\some_package.egg-info\SOURCES.txt'

warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md

......

D:\Python3Project\test\some_package_proj

λ dir /B dist

some_package-0.0.0-py3-none-any.whl

some_package-0.0.0.tar.gz

虽然出现了一些警告信息，但我们还是成功创建了 .whl 文件和 .tar.gz 文件。下面尝试消除这些警告。

步骤如下。

添加 README、README.rst、README.txt 或者 README.md 文件

补充配置中的 url 数据

补充作者和作者邮箱(或者维护者和维护者邮箱)信息

同时还要添加以下信息。

版本号

软件使用许可

变更记录

README 文件告诉用户如何使用你的包。url、作者信息告诉用户遇到问题时与谁联系。软件使用许可告诉用户使用包的注意事项(分发、贡献、复用代码有哪些限制)。如果你不希望开源，应当在使用许可中写清楚限制。如果开源，则推荐访问 https://choosealicense.com/ 选择合适的软件许可。

添加这些信息用不了多长时间，下面是一个精简的例子。

setup.py 文件：

# D:\Python3Project\test\some_package_proj\setup.py

from setuptools import setup, find_packages

setup(

name='some_package',

description='打包和分发示例',

version='1.0',

author='walker',

author_email='[email protected]',

url='https://segmentfault.com/a/1190000021065589',

packages=find_packages(where='src'),

package_dir={'': 'src'},

)

许可条文应该放到 LICENSE 文件里。(walker 抄的 MIT 许可)

# D:\Python3Project\test\some_package_proj\setup.py

MIT License

Copyright (c) 2014-present walker and other contributors

Permission is hereby granted, free of charge, to any person obtaining a copy

of this software and associated documentation files (the "Software"), to deal

in the Software without restriction, including without limitation the rights

to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

copies of the Software, and to permit persons to whom the Software is

furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all

copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE

AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

SOFTWARE.

README.md 文件内容如下：

D:\Python3Project\test\some_package_proj\README.md

# some_package: 打包和分发示例

`some_package` is the Python package to demostrate how easy it is

to create installable, maintainable, shareable packages and

distributions.

\``` # 斜线为本文转义

>>> import some_package

>>> some_package.some_func()

42

\```

That's it, really.

我习惯从现成的开源项目中复制 README 文件，删掉那些我不需要的条文，然后根据自己项目的要求进行修改。

如果是一个严肃的项目，建议逐字输入 README。个人项目复制粘贴修改即可。

再添加一个记录项目变更的日志文件。下面是一个例子：

D:\Python3Project\test\some_package_proj\CHANGELOG.md

# Changelog

All notable changes to this project will be documented in this file.

## [1.0.0] - 2019-11-21

### Added

- Something be added.

- Something be added.

### Changed

- Something be changed.

- Something be changed.

### Removed

- Something be removed.

## [0.0.5] - 2019-11-15

### Added

- Something be added.

### Changed

- Something be changed.

### Fixed

- Something be fixed.

## [0.0.4] - 2019-11-14

### Added

- Something be added.

## [0.0.3] - 2019-11-13

### Added

- Something be added.

### Changed

- Something be changed.

### Removed

- Something be removed.

## [0.0.2] - 2019-11-12

### Added

- Something.

## [0.0.1] - 2019-11-11

### Added

- Initial version.

请访问 https://keepachangelog.com ，阅读编写变更日志的建议。

让我们看看这些改动是否足以消除警告。

D:\Python3Project\test\some_package_proj

λ python3 setup.py sdist bdist_wheel

running sdist

running egg_info

creating src\some_package.egg-info

writing src\some_package.egg-info\PKG-INFO

writing dependency_links to src\some_package.egg-info\dependency_links.txt

writing top-level names to src\some_package.egg-info\top_level.txt

writing manifest file 'src\some_package.egg-info\SOURCES.txt'

reading manifest file 'src\some_package.egg-info\SOURCES.txt'

writing manifest file 'src\some_package.egg-info\SOURCES.txt'

running check

creating some_package-1.0

creating some_package-1.0\src

creating some_package-1.0\src\some_package

creating some_package-1.0\src\some_package.egg-info

copying files to some_package-1.0...

copying README.rst -> some_package-1.0

copying setup.py -> some_package-1.0

copying src\some_package\__init__.py -> some_package-1.0\src\some_package

copying src\some_package\some_module.py -> some_package-1.0\src\some_package

copying src\some_package.egg-info\PKG-INFO -> some_package-1.0\src\some_package.egg-info

copying src\some_package.egg-info\SOURCES.txt -> some_package-1.0\src\some_package.egg-info

copying src\some_package.egg-info\dependency_links.txt -> some_package-1.0\src\some_package.egg-info

copying src\some_package.egg-info\top_level.txt -> some_package-1.0\src\some_package.egg-info

Writing some_package-1.0\setup.cfg

creating dist

Creating tar archive

removing 'some_package-1.0' (and everything under it)

running bdist_wheel

running build

running build_py

creating build

creating build\lib

creating build\lib\some_package

copying src\some_package\some_module.py -> build\lib\some_package

copying src\some_package\__init__.py -> build\lib\some_package

installing to build\bdist.win-amd64\wheel

running install

running install_lib

creating build\bdist.win-amd64

creating build\bdist.win-amd64\wheel

creating build\bdist.win-amd64\wheel\some_package

copying build\lib\some_package\some_module.py -> build\bdist.win-amd64\wheel\.\some_package

copying build\lib\some_package\__init__.py -> build\bdist.win-amd64\wheel\.\some_package

running install_egg_info

Copying src\some_package.egg-info to build\bdist.win-amd64\wheel\.\some_package-1.0-py3.6.egg-info

running install_scripts

adding license file "LICENSE" (matched pattern "LICEN[CS]E*")

creating build\bdist.win-amd64\wheel\some_package-1.0.dist-info\WHEEL

creating 'dist\some_package-1.0-py3-none-any.whl' and adding 'build\bdist.win-amd64\wheel' to it

adding 'some_package/__init__.py'

adding 'some_package/some_module.py'

adding 'some_package-1.0.dist-info/LICENSE'

adding 'some_package-1.0.dist-info/METADATA'

adding 'some_package-1.0.dist-info/WHEEL'

adding 'some_package-1.0.dist-info/top_level.txt'

adding 'some_package-1.0.dist-info/RECORD'

removing build\bdist.win-amd64\wheel

D:\Python3Project\test\some_package_proj

λ dir /B dist

some_package-0.0.0-py3-none-any.whl

some_package-0.0.0.tar.gz

很好， 没有警告了。

现在可以把 .whl 文件和 .tar.gz 文件放到同一个目录，然后运行 pip install。

D:\Python3Project\test\some_package_proj

λ mkdir .\packages

D:\Python3Project\test\some_package_proj

λ copy .\dist\some_package-1.0-py3-none-any.whl .\packages\

已复制 1 个文件。

D:\Python3Project\test\some_package_proj

λ copy .\dist\some_package-1.0.tar.gz .\packages\

已复制 1 个文件。

D:\Python3Project\test\some_package_proj

λ pip3 install --no-index --find-links=.\packages\ some_package

Looking in links: .\packages\

Processing d:\python3project\test\some_package_proj\packages\some_package-1.0-py3-none-any.whl

Installing collected packages: some-package

Successfully installed some-package-1.0

D:\Python3Project\test\some_package_proj

λ pip3 install --no-index --find-links=.\packages\ some_package==1.0

Looking in links: .\packages\

Requirement already satisfied: some_package==1.0 in

c:\program files\python36\lib\site-packages (1.0)

现在可以为本地项目创建自己的包，它安装起来就像从 PyPI 安装那样容易。

创建可以从 PyPI 安装的包

Creating a PyPI-Installable Package

如果希望在 PyPI 上发布自己的包，那么需要在 setup.py 中添加更多的配置。同时还需要类似 Twine 这样的工具把包推送到 PyPI。Twine 是一组工具，它可以让你更容易、更安全地与 PyPI 交互。它通过 HTTPS 认证保护 PyPI 密钥信息，并负责将包上传到 PyPI。

这些内容已经超出了本书的范围，细节请查阅 Python Packaging User Guide(Python 程序以用户分发指南)和 Python 文档中关于 PyPI 的部分。