Centos7上部署Openshift Origin3.9

概述

openshift origin是红帽研发的一套开源容器平台，是基于kubernetes又封装了一些对象，不同于kubernetes，openshift origin有完整的CI/CD流程，更适合公司或商业使用。后面我会陆续推出一系列关于openshift origin的文章，版本是3.9。

本文说的是用ansible playbook安装高可用openshift origin集群。

准备至少5台机器，3台做master，1台做负载均衡，1台做子节点，保证每台机器都可以访问外网，可以是物理机，也可以是虚拟机，操作系统为centos7.3, 机器IP、主机名和角色如下：

10.131.31.37 openshift-4.test  master1
10.131.31.39 openshift-5.test  master2
10.131.31.8 openshift-2.tet    master3
10.131.31.14 openshift-3.test  node1
10.131.31.7 openshift-1.test    lb

架构图：

启动NetworkManager，2-6步需要在所有机器上操作

systemctl enable NetworkManager
systemctl start NetworkManager

开启ip_forward,docker进程访问外网用

vim /etc/sysctl.conf
net.ipv4.ip_forward = 1
sysctl -p /etc/sysctl.conf

手动绑定IP和主机名到/etc/hosts，如果你们有DNS服务，可以正常解析主机名则可以忽略此步骤

10.131.31.37 openshift-4.test
10.131.31.39 openshift-5.test
10.131.31.8 openshift-2.test
10.131.31.14 openshift-3.test
10.131.31.7 openshift-1.test

安装docker

yum install -y wget git net-tools bind-utils iptables-services bridge-utils bash-completion
yum install -y docker
systemctl enable docker
systemctl start docker

使用一台机器作为ansible管理机，6-9步是在ansible管理机上操作。安装ansible,pyOpenSSL（ansible版本要高于2.4.3并且低于2.6.0，建议用2.5.3）

yum -y install https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm
sed -i -e "s/^enabled=1/enabled=0/" /etc/yum.repos.d/epel.repo
yum -y --enablerepo=epel install ansible pyOpenSSL

生成ssh key并向其他机器分发公钥

ssh-keygen -f /root/.ssh/id_rsa -N ''

把生成的公钥/root/.ssh/id_rsa.pub 拷贝到其他机器上的/root/.ssh/authorized_keys里面，并把权限修改为600

下载部署代码openshift-ansible

git clone https://github.com/openshift/openshift-ansible.git
cd openshift-ansible
git checkout release-3.9

编辑ansible inventory , vim /etc/ansible/hosts

#Create an OSEv3 group that contains the masters and nodes groups
[OSEv3:children]
masters
nodes
etcd
lb

#Set variables common for all OSEv3 hosts
[OSEv3:vars]
#SSH user, this user should allow ssh based auth without requiring a password
ansible_ssh_user=root
openshift_deployment_type=origin
openshift_release=3.9

#define network 
osm_cluster_network_cidr=10.129.0.0/21
openshift_portal_net=10.130.0.0/21
osm_host_subnet_length=9
openshift_disable_check=disk_availability,docker_storage,memory_availability,docker_image_availability

#uncomment the following to enable htpasswd authentication; defaults to DenyAllPasswordIdentityProvider
openshift_master_identity_providers=[{'name':'htpasswd_auth','login':'true','challenge':'true','kind':'HTPasswdPasswordIdentityProvider','filename':'/etc/origin/master/htpasswd'}]

#docker options
openshift_docker_options="-l warn --ipv6=false --insecure-registry=0.0.0.0/0 --registry-mirror=https://docker.mirrors.ustc.edu.cn --log-opt max-size=1M --log-opt max-file=3"

#multiple masters co-located with etcd nodes
openshift_master_cluster_method=native
openshift_master_cluster_hostname=openshift.test.com
openshift_master_cluster_public_hostname=openshift.test.com

#host group for masters
[masters]
openshift-4.test
openshift-5.test
openshift-2.test

#host group for nodes, includes region info
[nodes]
openshift-4.test openshift_schedulable=True openshift_node_labels="{'region': 'infra'}"
openshift-5.test openshift_schedulable=True openshift_node_labels="{'region': 'infra'}"
openshift-2.test openshift_schedulable=True openshift_node_labels="{'region': 'infra'}"
openshift-3.test openshift_node_labels="{'region': 'test'}"

[etcd]
openshift-4.test
openshift-5.test
openshift-2.test

[lb]
openshift-1.test

解释：

此安装模式是高可用架构，即三个master，三个etcd，三个router，master和etcd是在同一台机器上，openshift_master_cluster_method=native就是这个意思。

openshift_master_cluster_hostname和openshift_master_cluster_public_hostname这两行的意思是定义访问域名。

openshift_docker_options是修改docker配置文件

openshift_master_identity_providers是定义集群访问模式,本文写的是htpasswd方式，也可以接入ldap等

openshift_node_labels是定义节点标签，对于三个master节点必须要定义为"{‘region’: ‘infra’}"

openshift_schedulable表示此节点可以被调度运行容器

lb是三个master节点的负载均衡，haproxy

osm_cluster_network_cidr是定义pod的网段

openshift_portal_net是定义service的网段

运行ansible playbooks安装集群

cd openshift-ansible
ansible-playbook playbooks/prerequisites.yml 
ansible-playbook playbooks/deploy_cluster.yml 

注：安装过程比较漫长，如果网络不太稳定的话中途可能会出现各种报错，不用担心，多跑几次就好了。

安装成功后，创建管理员账号

在其中一个master上进行以下操作，然后把/etc/origin/master/htpasswd内容复制到其他两个master上。

htpasswd -b /etc/origin/master/htpasswd admin admin
oc login -u system:admin
oc adm policy add-cluster-role-to-user cluster-admin admin

更多容器技术请关注公众号：