深入openstack rpm打包原理(openstack cinder rpm打包完全分析)

在这篇博文openstack软件包管理我已经介绍了下openstack打包原理最核心的原理和问题，但是并没有以实际的例子来介绍，因此这篇博文主要是通过cinder的spec文件来深入的讲解如何通过rpmbuild工具打出自己的版本，顺带对涉及到rpmbuild相关的知识做讲解，本次博文的主要笔记直接记录在spec文件里中。

我的rpmbuild的测试环境：

RPM 版本 4.11.3

rpmbuild构建路径采用默认值，具体如下

%_topdir %{getenv:HOME}/rpmbuild

%_rpmdir        %{_topdir}/RPMS

%_sourcedir     %{_topdir}/SOURCES

%_specdir       %{_topdir}/SPECS

%_srcrpmdir     %{_topdir}/SRPMS

%_buildrootdir      %{_topdir}/BUILDROOT

%buildroot      %{_buildrootdir}/%{name}-%{version}-%{release}.%{_arch}

关于如何配置自定义的宏环境，可以参考rpm 宏定义。

将下面的cinder.spec放入~/rpmbuild/SPECS/

将spec用到的source原文件(tarball、patch文件、控制脚本等)手动拷贝到~/rpmbuild/SOURCES

执行打包编译

rpmbuild -ba ~/rpmbuild/SPECS/

-ba：打源码包和二进制包

-bb：只构建二进制包

何为打源码包？生成在SRPMS下面的.src.rpm包

何为二进制包？生成在RPMS下面的.rpm包

其他的可以通过rpmbuild --help自行查看

生成的.rpm包在~/rpmbuild/RPMS/

生成的.src.rpm包在~/rpmbuild/SRPMS/

最终生成的包有：

openstack-cinder-7.0.0-123.el7.centos.noarch.rpm

python-cinder-7.0.0-1.el7.centos.noarch.rpm

openstack-cinder-7.0.0-1.el7.centos.src.rpm

包名称的构成：{Name}-{Version}-{Release}.{BuildArch}.rpm or .src.rpm

下面就是cinder.spec的内容，里面带有每个关键点的解释。

%global with_doc %{!?_without_doc:1}%{?_without_doc:0}
%global pypi_name cinder
%global release_name liberty
#global是用来定义一些全局变量，变量的反问语法是%{},如果是%{?}，则表示变量如果有定义才取值
%{!?upstream_version: %global upstream_version %{version}%{?milestone}}

Name:             openstack-cinder

#打出来的rpm包的名字
# Liberty semver reset
# https://review.openstack.org/#/q/I6a35fa0dda798fad93b804d00a46af80f08d475c,n,z
Epoch:            1
Version:          7.0.0
Release:          123%{?milestone}%{?dist}

#dist的值在/etc/rpm/macros.dist里面定义：%dist .el7.centos
Summary:          OpenStack Volume service

License:          ASL 2.0
URL:              http://www.openstack.org/software/openstack-storage/
Source0:          http://tarballs.openstack.org/%{pypi_name}/%{pypi_name}-%{upstream_version}.tar.gz
#源码包下载路径，这里目前我没看到rpmbuild有相关参数可以控制自动下，我的做法是：1、手动打执行版本的tarball，将其拷贝到SOURCES下。2、通过spectool工具来自动下载

#如果是自己手动打的tarball，最好要通过python标准的打包方式：python setup.py sdist来发布，否则会涉及到解压出来的发布软件包的版本问题.

#下面这些都是原来打包进rpm包的配置和systemd的控制文件
Source1:          cinder-dist.conf
Source2:          cinder.logrotate
Source3:          cinder-tgt.conf

Source10:         openstack-cinder-api.service
Source11:         openstack-cinder-scheduler.service
Source12:         openstack-cinder-volume.service
Source13:         openstack-cinder-backup.service
Source20:         cinder-sudoers

BuildArch:        noarch

#以BuildRequires定义的包是指在python setup.py build编译的时候需要用到的依赖
BuildRequires:    intltool
BuildRequires:    python-d2to1
BuildRequires:    python-oslo-sphinx
BuildRequires:    python-pbr
BuildRequires:    python-sphinx
BuildRequires:    python2-devel
BuildRequires:    python-setuptools
BuildRequires:    python-netaddr
BuildRequires:    systemd
BuildRequires:    git
BuildRequires:    os-brick
BuildRequires:    python-oslo-concurrency
BuildRequires:    python-oslo-messaging
BuildRequires:    python-oslo-service
# Required to build cinder.conf
BuildRequires:    python-keystonemiddleware
BuildRequires:    python-glanceclient
BuildRequires:    python-novaclient
BuildRequires:    python-swiftclient
BuildRequires:    python-oslo-db
BuildRequires:    python-oslo-config >= 2:1.11.0
BuildRequires:    python-oslo-policy
BuildRequires:    python-oslo-reports
BuildRequires:    python-oslotest
BuildRequires:    python-oslo-versionedobjects
BuildRequires:    python-oslo-vmware
BuildRequires:    python-crypto
BuildRequires:    python-lxml
BuildRequires:    python-osprofiler
BuildRequires:    python-paramiko
BuildRequires:    python-suds
BuildRequires:    python-taskflow
#以Requires定义的包是指在安装rpm包的时候依赖或者说对python来说就是指：python setup.py install
Requires:         openstack-utils
Requires:         python-cinder = %{epoch}:%{version}-%{release}

# we dropped the patch to remove PBR for Delorean
Requires:         python-pbr

# as convenience
Requires:         python-cinderclient

Requires(post):   systemd
Requires(preun):  systemd
Requires(postun): systemd
Requires(pre):    shadow-utils

Requires:         lvm2
Requires:         python-osprofiler
Requires:         python-rtslib

#这是默认包即openstack-cinder包的描述
%description
OpenStack Volume (codename Cinder) provides services to manage and
access block storage volumes for use by Virtual Machine instances.

#这里是指定义一个新的包，包名为python-cinder，所有关于这个包的信息，依赖都在这个字段下面书写
%package -n       python-cinder
Summary:          OpenStack Volume Python libraries
Group:            Applications/System

Requires:         sudo

Requires:         MySQL-python

Requires:         qemu-img
Requires:         sysfsutils
Requires:         os-brick
Requires:         python-paramiko

Requires:         python-qpid
Requires:         python-kombu
Requires:         python-amqplib

Requires:         python-eventlet
Requires:         python-greenlet
Requires:         python-iso8601
Requires:         python-netaddr
Requires:         python-lxml
Requires:         python-anyjson
Requires:         python-cheetah
Requires:         python-stevedore
Requires:         python-suds

Requires:         python-sqlalchemy
Requires:         python-migrate

Requires:         python-paste-deploy
Requires:         python-routes
Requires:         python-webob

Requires:         python-glanceclient >= 1:0
Requires:         python-swiftclient >= 1.2
Requires:         python-keystoneclient
Requires:         python-novaclient >= 1:2.15

Requires:         python-oslo-config >= 1:1.2.0
Requires:         python-oslo-db >= 1.0.0
Requires:         python-six >= 1.5.0
Requires:         python-psutil >= 1.1.1

Requires:         python-babel
Requires:         python-lockfile
Requires:         python-jinja2

Requires:         python-oslo-rootwrap
Requires:         python-oslo-utils
Requires:         python-oslo-serialization
Requires:         python-oslo-db
Requires:         python-oslo-context
Requires:         python-oslo-concurrency
Requires:         python-oslo-middleware
Requires:         python-taskflow >= 0.7.1
Requires:         python-oslo-messaging >= 1.8.0
Requires:         python-keystonemiddleware >= 1.5.0
Requires:         python-oslo-reports
Requires:         python-oslo-service
Requires:         python-oslo-versionedobjects
Requires:         libcgroup-tools
Requires:         iscsi-initiator-utils

Requires:         python-osprofiler

#新包python-cinder的描述
%description -n   python-cinder
OpenStack Volume (codename Cinder) provides services to manage and
access block storage volumes for use by Virtual Machine instances.

This package contains the cinder Python library.
#spec语法的逻辑控制块，如果定义with_doc，则会打出一个doc包
%if 0%{?with_doc}
%package doc
Summary:          Documentation for OpenStack Volume
Group:            Documentation

Requires:         %{name} = %{epoch}:%{version}-%{release}

BuildRequires:    graphviz

# Required to build module documents
BuildRequires:    python-eventlet
BuildRequires:    python-routes
BuildRequires:    python-sqlalchemy
BuildRequires:    python-webob
# while not strictly required, quiets the build down when building docs.
BuildRequires:    python-migrate, python-iso8601

%description      doc
OpenStack Volume (codename Cinder) provides services to manage and
access block storage volumes for use by Virtual Machine instances.

This package contains documentation files for cinder.
%endif

#到这里关于包的定义，描述都结束，接下来就是开始编译前准备，做的事无非就是将源码包解压在BUILD构建

#目录下，再通过一些扩展宏对构建包做一些处理，比如将其初始化为一个git仓库。
%prep
%autosetup -n cinder-%{upstream_version} -S git
#autosetup是rpm替代setup的宏，相关用法参考rpm宏定义及openstack软件包管理

#这段相当于直接执行shell的脚本
find . \( -name .gitignore -o -name .placeholder \) -delete

find cinder -name \*.py -exec sed -i '/\/usr\/bin\/env python/{d;q}' {} +

# Remove the requirements file so that pbr hooks don't add it
# to distutils requires_dist config
rm -rf {test-,}requirements.txt tools/{pip,test}-requires

#开始build阶段，先设置PYTHONPAHT，再执行build动作
%build
# Generate config file
PYTHONPATH=. tools/config/generate_sample.sh from_tox

%{__python2} setup.py build
#安装rpm的时候执行的动作
%install
%{__python2} setup.py install -O1 --skip-build --root %{buildroot}
#下面这些动作都是rpm安装的时候做的事，装完源代码之后还会做一些系统环境相关的配置
# docs generation requires everything to be installed first
export PYTHONPATH="$( pwd ):$PYTHONPATH"

pushd doc

%if 0%{?with_doc}
SPHINX_DEBUG=1 sphinx-build -b html source build/html
# Fix hidden-file-or-dir warnings
rm -fr build/html/.doctrees build/html/.buildinfo
%endif

# Create dir link to avoid a sphinx-build exception
mkdir -p build/man/.doctrees/
ln -s .  build/man/.doctrees/man
SPHINX_DEBUG=1 sphinx-build -b man -c source source/man build/man
mkdir -p %{buildroot}%{_mandir}/man1
install -p -D -m 644 build/man/*.1 %{buildroot}%{_mandir}/man1/

popd

# Setup directories
install -d -m 755 %{buildroot}%{_sharedstatedir}/cinder
install -d -m 755 %{buildroot}%{_sharedstatedir}/cinder/tmp
install -d -m 755 %{buildroot}%{_localstatedir}/log/cinder

# Install config files
install -d -m 755 %{buildroot}%{_sysconfdir}/cinder
install -p -D -m 640 %{SOURCE1} %{buildroot}%{_datadir}/cinder/cinder-dist.conf
install -d -m 755 %{buildroot}%{_sysconfdir}/cinder/volumes
install -p -D -m 644 %{SOURCE3} %{buildroot}%{_sysconfdir}/tgt/conf.d/cinder.conf
install -p -D -m 640 etc/cinder/rootwrap.conf %{buildroot}%{_sysconfdir}/cinder/rootwrap.conf
install -p -D -m 640 etc/cinder/api-paste.ini %{buildroot}%{_sysconfdir}/cinder/api-paste.ini
install -p -D -m 640 etc/cinder/policy.json %{buildroot}%{_sysconfdir}/cinder/policy.json
install -p -D -m 640 etc/cinder/cinder.conf.sample %{buildroot}%{_sysconfdir}/cinder/cinder.conf

# Install initscripts for services
install -p -D -m 644 %{SOURCE10} %{buildroot}%{_unitdir}/openstack-cinder-api.service
install -p -D -m 644 %{SOURCE11} %{buildroot}%{_unitdir}/openstack-cinder-scheduler.service
install -p -D -m 644 %{SOURCE12} %{buildroot}%{_unitdir}/openstack-cinder-volume.service
install -p -D -m 644 %{SOURCE13} %{buildroot}%{_unitdir}/openstack-cinder-backup.service

# Install sudoers
install -p -D -m 440 %{SOURCE20} %{buildroot}%{_sysconfdir}/sudoers.d/cinder

# Install logrotate
install -p -D -m 644 %{SOURCE2} %{buildroot}%{_sysconfdir}/logrotate.d/openstack-cinder

# Install pid directory
install -d -m 755 %{buildroot}%{_localstatedir}/run/cinder

# Install rootwrap files in /usr/share/cinder/rootwrap
mkdir -p %{buildroot}%{_datarootdir}/cinder/rootwrap/
install -p -D -m 644 etc/cinder/rootwrap.d/* %{buildroot}%{_datarootdir}/cinder/rootwrap/


# Symlinks to rootwrap config files
mkdir -p %{buildroot}%{_sysconfdir}/cinder/rootwrap.d
for filter in %{_datarootdir}/os-brick/rootwrap/*.filters; do
ln -s $filter %{buildroot}%{_sysconfdir}/cinder/rootwrap.d/
done
# Remove unneeded in production stuff
rm -f %{buildroot}%{_bindir}/cinder-all
rm -f %{buildroot}%{_bindir}/cinder-debug
rm -fr %{buildroot}%{python2_sitelib}/cinder/tests/
rm -fr %{buildroot}%{python2_sitelib}/run_tests.*
rm -f %{buildroot}/usr/share/doc/cinder/README*

#安装前的准备阶段，执行在%install阶段前
%pre
getent group cinder >/dev/null || groupadd -r cinder --gid 165
if ! getent passwd cinder >/dev/null; then
  useradd -u 165 -r -g cinder -G cinder,nobody -d %{_sharedstatedir}/cinder -s /sbin/nologin -c "OpenStack Cinder Daemons" cinder
fi
exit 0
#安装后的收尾阶段,执行在%install之后
%post
%systemd_post openstack-cinder-volume
%systemd_post openstack-cinder-api
%systemd_post openstack-cinder-scheduler
%systemd_post openstack-cinder-backup

#卸载rpm包之前的适合动作
%preun
%systemd_preun openstack-cinder-volume
%systemd_preun openstack-cinder-api
%systemd_preun openstack-cinder-scheduler
%systemd_preun openstack-cinder-backup

#卸载rpm包之后的动作
%postun
%systemd_postun_with_restart openstack-cinder-volume
%systemd_postun_with_restart openstack-cinder-api
%systemd_postun_with_restart openstack-cinder-scheduler
%systemd_postun_with_restart openstack-cinder-backup

#默认的包openstack-cinder所包含的文件列表
%files
%dir %{_sysconfdir}/cinder
%config(noreplace) %attr(-, root, cinder) %{_sysconfdir}/cinder/cinder.conf
%config(noreplace) %attr(-, root, cinder) %{_sysconfdir}/cinder/api-paste.ini
%config(noreplace) %attr(-, root, cinder) %{_sysconfdir}/cinder/rootwrap.conf
%config(noreplace) %attr(-, root, cinder) %{_sysconfdir}/cinder/policy.json
%config(noreplace) %{_sysconfdir}/logrotate.d/openstack-cinder
%config(noreplace) %{_sysconfdir}/sudoers.d/cinder
%config(noreplace) %{_sysconfdir}/tgt/conf.d/cinder.conf
%{_sysconfdir}/cinder/rootwrap.d/
%attr(-, root, cinder) %{_datadir}/cinder/cinder-dist.conf

%dir %attr(0750, cinder, root) %{_localstatedir}/log/cinder
%dir %attr(0755, cinder, root) %{_localstatedir}/run/cinder
%dir %attr(0755, cinder, root) %{_sysconfdir}/cinder/volumes

%{_bindir}/cinder-*
%{_unitdir}/*.service
%{_datarootdir}/cinder
%{_mandir}/man1/cinder*.1.gz

%defattr(-, cinder, cinder, -)
%dir %{_sharedstatedir}/cinder
%dir %{_sharedstatedir}/cinder/tmp

#python-cinder包所包含的文件列表,cinder项目的源代码都在这个包里面，这个包相当于公共基础包
%files -n python-cinder
%{?!_licensedir: %global license %%doc}
%license LICENSE
%{python2_sitelib}/cinder
%{python2_sitelib}/cinder-*.egg-info

#doc包所包含的文件列表
%if 0%{?with_doc}
%files doc
%doc doc/build/html
%endif
#这个不重要
%changelog
* Fri Oct 16 2015 Haikel Guemar  1:7.0.0-1
- Update to upstream 7.0.0

* Wed Oct 07 2015 Haikel Guemar  7.0.0-0.2.0rc2
- Update to upstream 7.0.0.0rc2

* Wed Sep 30 2015 Haikel Guemar  7.0.0-0.1rc1
- Update to upstream 7.0.0.0rc1

* Wed Jun 17 2015 Haïkel Guémar  2015.1.0-3
- Fix CVE-2015-1851 (RHBZ #1231822)

* Mon May 04 2015 Alan Pevec  2015.1.0-2
- update keystone_authtoken section in sample conf

* Thu Apr 30 2015 Alan Pevec  2015.1.0-1
- OpenStack Kilo release