Gitlab CICD 与Kubernetes实践·部署GitLab
上一篇，简单的从????Gitlab CI/CD方法论中探索实践中大致了解Gitlab在CI/CD功能的基本介绍，现在我们通过在K8s集群内安装Gitlab、Gitlab Runner来为深入探索Gitlab持续集成做好前期准备，首先我们要在集群内安装Gitlab.
集群环境
K8s的集群版本为1.11.5，CNI使用的是Flannel, DNS为KubeDNS
☸️ ACK???? devops ~ ???? ???? k cluster-info
Kubernetes master is running at https://192.168.99.128:6443
Heapster is running at https://192.168.99.128:6443/api/v1/namespaces/kube-system/services/heapster/proxy
KubeDNS is running at https://192.168.99.128:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
monitoring-influxdb is running at https://192.168.99.128:6443/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
☸️ ACK???? devops ~ ???? ???? k get nodes
NAME STATUS ROLES AGE VERSION
cn-beijing.i-2ze01 Ready master 1y v1.11.5
cn-beijing.i-2ze02 Ready master 1y v1.11.5
cn-beijing.i-2ze03 Ready master 1y v1.11.5
cn-beijing.i-2ze04 Ready 1y v1.11.5
cn-beijing.i-2ze05 Ready 1y v1.11.5
通过学习Github上一个开源项目Dockerized GitLab[1]在kubernetes上进行GitLab-CE的安装部署，在部署Gitlab的之前，我们需要先部署其依赖的Redis和PostgreSQL,然后才能正常的运行gitlab. 这个开源项目中资源配置清单使用的是ReplicationController,这里我将修改使用Deployment控制器：
配置Redis服务
以为redis服务是提供给Gitlab服务使用，并不需要暴露在集群外部，因此我们在gitlab<->redis之间调用的时候，采用内部通信的方式,准备redis的配置清单redis.deployment.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: gitlab-redis # deployment的名字
namespace: devops # 部署在devops namespace里面
labels:
name: gitlab-redis
spec:
replicas: 1
template:
metadata:
name: gitlab-redis # 约定gitlab-redis的deployment的模板名
labels:
name: gitlab-redis
spec:
containers:
- name: redis # pod内redis的容器名
image: redis:5.0.9 #也可以使用sameersbn的redis镜像,此处我们使用官方的镜像
imagePullPolicy: IfNotPresent # 镜像的拉取策略
ports:
- name: redis
containerPort: 6379
volumeMounts:
- mountPath: /var/lib/redis # 需要持久化的数据目录
name: data
livenessProbe: # 进行存活性监测
exec:
command:
- redis-cli
- ping
initialDelaySeconds: 30 # 在启动存活性探测之前等待的秒数
timeoutSeconds: 5 # 探测的超时时长
readinessProbe:
exec:
command:
- redis-cli
- ping
initialDelaySeconds: 5
timeoutSeconds: 1
volumes:
- name: data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: gitlab-redis-svc
namespace: devops
labels:
name: gitlab-redis-svc
spec:
ports:
- name: redis
port: 6379
targetPort: redis
selector:
name: gitlab-redis
配置postgresql服务
与redis服务一样，我们准备postgresql服务的配置清单postgresql.deployment.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: postgresql
namespace: devops
labels:
name: postgresql
spec:
template:
metadata:
name: postgresql
labels:
name: postgresql
spec:
containers:
- name: postgresql
image: sameersbn/postgresql # latest
imagePullPolicy: IfNotPresent
env:
- name: DB_USER
value: gitlab
- name: DB_PASS
value: gitlab_postgresql
- name: DB_NAME
value: gitlab_production
- name: DB_EXTENSION
value: pg_trgm
ports:
- name: postgres
containerPort: 5432
volumeMounts:
- mountPath: /var/lib/postgresql
name: data
livenessProbe:
exec:
command:
- pg_isready
- -h
- localhost
- -U
- postgres
initialDelaySeconds: 30
timeoutSeconds: 5
readinessProbe:
exec:
command:
- pg_isready
- -h
- localhost
- -U
- postgres
initialDelaySeconds: 5
timeoutSeconds: 1
volumes:
- name: data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: postgresql
namespace: devops
labels:
name: postgresql
spec:
ports:
- name: postgres
port: 5432
targetPort: postgres
selector:
name: postgresql
配置Gitlab服务
gitlab服务的部署就相对复杂一些，要添加正确的redis和postgresql的链接信息，同时为了在集群外部访问gitlab,我们需要给gitlab配置一个ingress,我们使用的环境是ACK,所以最后的时候我们需要给gitlab的服务分配一个ACK上绑定的测试域名，假设我们这里ACK的测试域名为*.cloudnativecosystem.cn-beijing.alicontainer.com,准备gitlab服务的配置清单gitlab.deployment.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: gitlab
namespace: devops
labels:
name: gitlab
spec:
template:
metadata:
name: gitlab
labels:
name: gitlab
spec:
containers:
- name: gitlab
image: sameersbn/gitlab:11.8.1
imagePullPolicy: IfNotPresent
env:
- name: TZ # 容器时区
value: Asia/Shanghai
- name: GITLAB_TIMEZONE # 配置gitlab的时区
value: Beijing
- name: GITLAB_SECRETS_DB_KEY_BASE # 用于加密数据库中的CI机密变量以及导入凭据。如果丢失或旋转了此机密，则将无法使用现有的CI机密
value: PjqzXnqkv9rjKWnTqhmgKLhtbM3sCKVH9bhHrmKRpnHXttd3hRjF4zXNjxztKKsC # 可以使用`pwgen -Bsv1 64`生成随机的字符串给变量
- name: GITLAB_SECRETS_SECRET_KEY_BASE # 用于密码重置链接和其他“标准”身份验证功能。如果丢失或旋转了此机密，电子邮件中的密码重置令牌将重置。
value: 3Tgk4WgqcK4JFn3gwMjNcgzTwkfhpTrL4wvmwhcmTRcPwqHzT4pcmgfsTvfpzpLV
- name: GITLAB_SECRETS_OTP_KEY_BASE # 用于加密数据库中的2FA机密。如果您丢失或旋转了此机密，则您的所有用户都将无法使用2FA登录
value: CdM9VprWKpqsdmw4V3tmcFwkzNVmHV9Kc3pLR7WtpVgHtFKmfCkMfJMW9TNw7pf7
- name: GITLAB_ROOT_PASSWORD
value: P@ssw0rd
- name: GITLAB_ROOT_EMAIL
value: cloudnativecosystem@******.cn-beijing.alicontainer.com
- name: GITLAB_HOST
value: code.******.cn-beijing.alicontainer.com
- name: GITLAB_PORT
value: "80"
- name: GITLAB_SSH_PORT
value: "30003"
- name: GITLAB_NOTIFY_ON_BROKEN_BUILDS
value: "true"
- name: GITLAB_NOTIFY_PUSHER
value: "false"
- name: GITLAB_BACKUP_SCHEDULE # 更多变量信息参考https://github.com/sameersbn/docker-gitlab#quick-start
value: daily
- name: GITLAB_BACKUP_TIME
value: 01:00
- name: DB_TYPE
value: postgres
- name: DB_HOST
value: postgresql
- name: DB_PORT
value: "5432"
- name: DB_USER
value: gitlab
- name: DB_PASS
value: P@ssw0rd
- name: DB_NAME
value: gitlab_production
- name: REDIS_HOST
value: gitlab-redis-svc
- name: REDIS_PORT
value: "6379"
ports:
- name: http
containerPort: 80
- name: ssh
containerPort: 22
volumeMounts:
- mountPath: /home/git/data
name: data
livenessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 180
timeoutSeconds: 5
readinessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 5
timeoutSeconds: 1
volumes:
- name: data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: gitlab
namespace: devops
labels:
name: gitlab
spec:
ports:
- name: http
port: 80
targetPort: http
- name: ssh
port: 22
targetPort: 30003
selector:
name: gitlab
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/service-weight: ''
name: gitlab
namespace: devops
spec:
rules:
- host: code.******.cn-beijing.alicontainer.com
http:
paths:
- backend:
serviceName: gitlab
servicePort: 80
path: /
status:
loadBalancer: {}
在准备完成所有相关的配置清单文件之后，我们就可以直接创建服务了
☸️ ACK???? devops ~/v1.11.5/gitlab ???? ???? ls -al
total 24
drwxr-xr-x 7 marionxue staff 224 Oct 24 16:34 .
drwxr-xr-x 5 marionxue staff 160 Oct 24 20:22 ..
-rw-r--r-- 1 marionxue staff 3631 Oct 24 21:14 gitlab.deployment.yaml
-rw-r--r-- 1 marionxue staff 1460 Oct 24 21:12 postgresql.deployment.yaml
-rw-r--r-- 1 marionxue staff 1397 Oct 24 20:47 redis.deployment.yaml
☸️ ACK???? devops ~/v1.11.5/gitlab ???? ???? k apply -f .
deployment.apps/gitlab created
service/gitlab created
ingress.extensions/gitlab created
deployment.apps/postgresql created
service/postgresql created
deployment.apps/gitlab-redis created
service/gitlab-redis-svc created
☸️ ACK???? devops ~/v1.11.5/gitlab ???? ???? k get pods
NAME READY STATUS RESTARTS AGE
gitlab-7c565f7845-vxzrw 0/1 ContainerCreating 0 12s
gitlab-redis-5d86f5cf95-pr8dp 1/1 Running 0 11s
postgresql-64d79556cf-prqn5 1/1 Running 0 11s
在所有的pods的运行状态为Running的时候，就可以访问gitlab了，如果很久没有运行起来，可以通过kubectl describe/logs进行初步排查。
等待之余，gitlab已经运行起来，我们直接可以通过ingress上配置的路由进行访问了
访问Kubernetes上的GitLab(root/P@ssw0rd)
然后进行一个正常的clone上传和下载操作,首先创建一个代码仓库gitlab-ci-demo
创建一个名为 gitlab-ci-demo的仓库
☸️ ACK???? devops ~/v1.11.5/gitlab ???? ???? git clone http://code.c33a5017db3924e7a86deeeaca6a706b8.cn-beijing.alicontainer.com/root/gitlab-ci-demo.git
Cloning into 'gitlab-ci-demo'...
warning: You appear to have cloned an empty repository.
☸️ ACK???? devops ~/v1.11.5/gitlab ???? ???? cd gitlab-ci-demo
☸️ ACK???? devops ~/v1.11.5/gitlab/gitlab-ci-demo master ???? ???? echo "云原生生态圈" > ./README.md
☸️ ACK???? devops ~/v1.11.5/gitlab/gitlab-ci-demo master ???? ???? git add .
☸️ ACK???? devops ~/v1.11.5/gitlab/gitlab-ci-demo master ✚ ???? ???? git commit -am "Update README.MD"
[master (root-commit) cc89857] Update README.MD
1 file changed, 1 insertion(+)
create mode 100644 README.md
☸️ ACK???? devops ~/v1.11.5/gitlab/gitlab-ci-demo master ???? ???? git push origin master
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'http://code.c33a5017db3924e7a86deeeaca6a706b8.cn-beijing.alicontainer.com/root/gitlab-ci-demo.git/'
☸️ ACK???? devops ~/v1.11.5/gitlab/gitlab-ci-demo master ???? ???? git push origin master
Username for 'http://code.c33a5017db3924e7a86deeeaca6a706b8.cn-beijing.alicontainer.com': root
Password for 'http://root@code.c33a5017db3924e7a86deeeaca6a706b8.cn-beijing.alicontainer.com':
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 241 bytes | 241.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To http://code.c33a5017db3924e7a86deeeaca6a706b8.cn-beijing.alicontainer.com/root/gitlab-ci-demo.git
* [new branch] master -> master
至此，我们基本上完成了在k8s上运行gitlab的初步需求。下一步就可以准备Gitlab Runner在k8s上的运行和进行持续集成持续部署的实践了。
参考资料
[1]
Dockerized GitLab: https://github.com/sameersbn/docker-gitlab
-
???? 15 个 Kubectl 现有命令使用技巧 - 拿来即用
???? Jenkins在kubernetes上的初体验
???? 走进Network Namespace学会容器网络调试
???? 实践 | Kubernetes守护进程集之DaemonSet
???? 神奇！如何快速成为一名优秀的YAML工程师？
???? 最流行的五款Kubernetes交互式可视化工具
???? ab压力测试模拟实现kubernetes Pod水平自动伸缩
???? 用Prometheus对业务服务进行监控
???? Prometheus监控系列-监控篇
???? kubernetes炼气期之掌握Kubernetes的背景
???? Prometheus监控系列-部署篇
???? 2020年最高效的10款Kubernetes助力神器
???? 写给孩子看的Kubernetes动画指南
???? kubernetes的ingress控制器比较(traefik2.0.5安装指南)
???? kubernetes深度探究Node和Pod的亲和性和反亲和性
???? 在kuebernetes上通过nfs-server持久化postgresql
???? kubernetes监控架构核心组件Metrics-server
???? 基于BasicAuth认证的Traefik2.0.5
???? 在Kubernetes上部署Nginx Ingress controller
???? kubernetes安装方案大全
???? kubernetes最常用的资源对象Deployment
???? kubernetes炼气期之掌握kuebernetes背景
???? kubernetes炼气期之k8s平台快速搭建
???? 二进制部署Kubernetes集群1.9版本
你可能还喜欢
点击下方图片即可阅读