迁移服务到Openshift完整实例
概述
openshift提供了3种方式构建镜像：Docker, S2I, Pipeline。
-
Docker方式是典型的一种方式。
-
S2I（source to image) 适用于java, golang等需要编译的应用，因为它提供依赖包保存的功能以保证后续的镜像构建可以复用第一次下载的依赖包，减少了构建镜像的时间。
-
Pipeline是采用jenkins的方式来构建镜像，jenkins插件丰富，功能全面。
kubernetes是没有构建镜像的功能的，也就是缺失了CI这一块的功能。
大部分情况下你其实只需要第一种方式即可。本文以第一种方式来说明。
应用说明
把应用迁移到openshift或kubernetes中主要需要两个东西，一个是Dockerfile,一个是模版。本文示例的应用是一个以ansible为基础开发的批量部署系统，以下简称AU。整个应用设计到的组件包含django, celery, mysql, nginx, uwsgi, supervisor，按照标准的做法应该把每个组件都单独放到一个容器中跑，但本文为了简化流程就把所有组件放到supervisor中托管，容器启动时只启动supervisor进程。
上传项目源代码AU到git仓库
git init
git add -A .
git commit -m "init"
git remote add origin [email protected]:au
git push -u origin master
4.编写Dockerfile
编写Dockerfile有自己的规则，每一条命令就代表一层镜像，所以写Dockerfile的时候能合并到一个命令的尽量合并，以下是AU的Dockerfile
FROM ubuntu:14.04
MAINTAINER zhanghh
#define volume /var/log/AU and volume for mysql storage
#VOLUME ["/var/log/AU", "/var/lib/mysql"]
#copy source code to container
COPY . /data/app/AU/
#set source list
RUN \
##change owner and group for /var/lib/mysql
#chown -R mysql:mysql /var/lib/mysql
#chmod 700 /var/lib/mysql
#install basic packages
apt-get install -y \
openssh-client \
mysql-server \
libmysqlclient-dev \
python-dev \
libldap2-dev \
libsasl2-dev \
libssl-dev \
libffi-dev \
supervisor \
nginx \
libjpeg8-dev \
zlib1g-dev \
python-setuptools \
python-dev \
build-essential \
python-pip \
&& pip install --upgrade setuptools;\
\
#add user ansible
adduser --home /home/ansible --shell /bin/bash ansible;\
\
#install python libraries
pip install -r /data/app/AU/requirements.txt; \
\
###mysql setting ###
#modify /etc/mysql/my.cnf
sed -i 's/bind_address/# bind_address/g' /etc/mysql/my.cnf; \
\
#start mysql service
service mysql start && \
\
#set root password
#RUN mysqladmin -u root password xxxxxxx
#create user and grant privileges
mysql -u root -e "create database ansible CHARACTER SET utf8; \
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'xxxxxxx'; \
GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'xxxxxxx';"; \
\
#create dir /var/run/uwsgi
mkdir /var/run/uwsgi \
&& chown ansible:ansible /var/run/uwsgi \
&& chmod 755 /var/run/uwsgi;\
\
#create dir /var/log/AU
mkdir /var/log/AU \
&& chmod 777 /var/log/AU;\
#initialize database
python /data/app/AU/manage.py makemigrations account && \
python /data/app/AU/manage.py makemigrations ansible && \
python /data/app/AU/manage.py migrate ansible && \
python /data/app/AU/manage.py migrate account && \
python /data/app/AU/manage.py migrate djcelery && \
python /data/app/AU/manage.py migrate guardian && \
python /data/app/AU/manage.py migrate && \
\
#remove /var/log/AU/AU_debug.log
rm -f /var/log/AU/AU_debug.log && \
\
#remove /etc/nginx/sites-enabled/default
rm -f /etc/nginx/sites-enabled/default
#copy supervisor config file
COPY ./supervisord.conf /etc/supervisor/supervisord.conf
#copy nginx config file
COPY ./nginx-conf/nginx_ansible.conf /etc/nginx/sites-enabled/
#copy ansible config file
COPY ./ansible-conf/ansible.cfg /etc/ansible/ansible.cfg
EXPOSE 80 9001 3306
CMD ["/usr/bin/supervisord"]
Dockerfile里面主要就是应用的部署脚本。
在容器中服务需要以no-daemon的方式运行，所以在supervisor中服务也需要以no-daemon方式启动。
supervisor.conf示例如下：
[supervisord]
logfile=/var/log/AU/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/var/run/supervisord.pid
nodaemon=true ; (start in foreground if true;default false)
minfds=10240 ; (min. avail startup file descriptors;default 1024)
minprocs=2000 ; (min. avail process descriptors;default 200)
[program:AU-celery]
autorestart = true
redirect_stderr=true
command = /usr/bin/python /data/app/AU/manage.py celeryd -B -l info --autoscale=40,2 --concurrency=40
user = root
;must specify the environment for HOME, or there will be permission problem
environment=HOME="/root",USER="root",C_FORCE_ROOT="true"
stdout_logfile=/var/log/AU/celery.log
loglevel=info
; kill all children processes when killing the master process
stopasgroup=true
killasgroup=true
[program:AU-django]
autorestart = true
redirect_stderr=true
command = /usr/local/bin/uwsgi --ini /data/app/AU/ansible_uwsgi.ini
user = root
stdout_logfile=/var/log/AU/django.log
loglevel=info
; kill all children processes when killing the master process
stopasgroup=true
killasgroup=true
[program:AU-nginx]
autorestart = true
redirect_stderr=true
command = /usr/sbin/nginx -g "daemon off;"
user = root
stdout_logfile=/var/log/AU/nginx.log
loglevel=info
; kill all children processes when killing the master process
stopasgroup=true
killasgroup=true
[program:AU-mysql]
autorestart = true
redirect_stderr=true
command = /usr/bin/pidproxy /var/run/mysqld/mysqld.pid /usr/bin/mysqld_safe
user = root
stdout_logfile=/var/log/AU/mysql.log
loglevel=info
; kill all children processes when killing the master process
stopasgroup=true
killasgroup=true
测试调试Dockerfile
在openshift中构建镜像之前，你最好用docker run自己测试测试，没有问题后在迁移到openshift中，因为在本地测试还可以利用本地的cache，大大的缩短了调试时间。
构建镜像命令：
docker build -t au:v1.0 .
启动镜像并进入交互模式命令：
docker run --rm -it au:v1.0 /bin/bash
进入容器后手动启动supervisor来调试Dockerfile:
/usr/bin/supervisord
编写模版文件
template文件包含容器生命周期的所有东西，类型包括buildconfig, deploymentconfig, service, imagestream, secret等。
au_template.yaml如下：
apiVersion: v1
kind: Template
labels:
template: demo-au
message: |-
The following service(s) have been created in your project: demo-au
metadata:
annotations:
description: demo-au used for deployment in parallel
iconClass: icon-django
openshift.io/display-name: demo-au
openshift.io/long-description: demo-au used for deployment in parallel
openshift.io/provider-display-name: Red Hat, Inc.
creationTimestamp: null
name: demo-au
objects:
- apiVersion: v1
kind: Service
metadata:
annotations:
description: Exposes and load balances the application pods
name: demo-au
spec:
ports:
- name: nginx
port: 80
targetPort: 80
selector:
name: demo-au
- apiVersion: v1
kind: Route
metadata:
name: demo-au
spec:
host: demo-au.example.com
to:
kind: Service
name: demo-au
- apiVersion: v1
kind: ImageStream
metadata:
annotations:
description: Keeps track of changes in the application image
name: demo-au
spec:
lookupPolicy:
local: false
tags:
- annotations: null
from:
kind: DockerImage
name: docker-registry.default.svc:5000/demo-au/demo-au:v1.0
generation: 3
importPolicy:
insecure: true
name: v1.0
referencePolicy:
type: Source
status:
dockerImageRepository: ""
- apiVersion: v1
kind: BuildConfig
metadata:
annotations:
description: Defines how to build the application
name: demo-au
spec:
nodeSelector:
region: infra
output:
to:
kind: ImageStreamTag
name: demo-au:v1.0
source:
git:
ref: master
uri: https://gitlab.example.com/au.git
sourceSecret:
name: git-secret
type: Git
strategy:
type: Docker
- apiVersion: v1
kind: DeploymentConfig
metadata:
annotations:
description: Defines how to deploy the application server
name: demo-au
spec:
replicas: 1
selector:
name: demo-au
strategy:
type: Rolling
template:
metadata:
labels:
name: demo-au
name: demo-au
spec:
nodeSelector:
region: production
containers:
- name: demo-au
env: []
image: ' '
#livenessProbe:
# httpGet:
# path: /
# port: 80
# initialDelaySeconds: 30
# timeoutSeconds: 3
ports:
- containerPort: 80
#readinessProbe:
# httpGet:
# path: /
# port: 80
# initialDelaySeconds: 3
# timeoutSeconds: 3
resources:
limits:
cpu: 2000m
memory: 2Gi
volumeMounts:
- mountPath: /var/lib/mysql
name: demo-au-mysql
- mountPath: /data/app/AU/projects
name: demo-au-projects
volumes:
- name: demo-au-mysql
persistentVolumeClaim:
claimName: demo-au-mysql
- name: demo-au-projects
persistentVolumeClaim:
claimName: demo-au-projects
triggers:
- imageChangeParams:
automatic: true
containerNames:
- demo-au
from:
kind: ImageStreamTag
name: demo-au:v1.0
type: ImageChange
- type: ConfigChange
- apiVersion: v1
data:
password: xxxxxxxxxx
username: xxxxxxxxxx
kind: Secret
metadata:
creationTimestamp: null
name: git-secret
type: kubernetes.io/basic-auth
创建新的项目
oc new-project demo-au
上传template文件到openshift，构建镜像并部署
打开openshift web console，进入到特定的project, 通过右上角的“Add to project" -→ “Import YAML/JSON” , 把template文件复制到里面，然后创建。
创建成功后，到"Build" → “Build” -→"创建镜像“ 创建镜像。
镜像创建的过程中可以通过命令来查看过程：
oc logs -f au-1-build
镜像构建好后会自动触发部署，部署成功后Pod会显示蓝色，否则表示部署失败。
更多容器技术请关注公众号：
