我利用DevOps给公司来一套组合拳会产生什么样的效果
一、什么是DevOps？
DevOps 是 Development 和 Operations 的组合词。它是一组过程、方法与系统的统称，用于促进开发（应用程序 / 软件工程）、技术运营和质量保障（QA）部门之间的沟通、协作与整合。
它是一种重视“软件开发人员（Dev）”和“IT 运维技术人员（Ops）”之间沟通合作的文化、运动或惯例。透过自动化“软件交付”和“架构变更”的流程，来使得构建、测试、发布软件能够更加地快捷、频繁和可靠，把敏捷开发部门和运维部门之间的围墙打通，形成闭环。
在 DevOps 流程下，运维人员会在项目开发期间就介入到开发过程中，了解开发人员使用的系统架构和技术路线，从而制定适当的运维方案。而开发人员也会在运维的初期参与到系统部署中，并提供系统部署的优化建议。
二、DevOps的优势
-
快速交付： DevOps强调自动化和持续集成/持续交付（CI/CD），使得开发团队能够更快速地交付新的功能和更新。这有助于更及时地响应用户需求和市场变化。
-
频繁的部署： CI/CD流程使得软件能够以更频繁的方式进行部署。小而频繁的部署有助于减小每次部署的风险，减少问题的影响范围，并能够更迅速地纠正错误。
-
更高的可靠性： DevOps强调自动化测试、自动化部署和持续监控，这有助于发现和修复问题。通过自动化的运维实践，系统更容易保持稳定和可靠。
-
卓越的团队协作： DevOps强调开发和运维团队之间的协作和沟通。通过共享目标、工具和责任，可以减少团队之间的隔阂，提高整个团队的效率。
-
更好的透明度： DevOps通过自动化和监控提供更好的透明度，让团队能够更清晰地了解应用的性能、健康状况和用户反馈。这有助于更及时地做出决策和调整策略。
-
降低成本： 通过自动化和流程的改进，DevOps可以降低部署和运维的成本。自动化减少了手动操作的需要，提高了效率，同时也减少了因人为错误而导致的问题。
-
持续反馈： DevOps强调持续反馈的重要性。通过自动化测试和监控，团队可以及时获得关于系统状态和质量的反馈，有助于快速识别和解决问题。
-
灵活性和敏捷性： DevOps通过自动化、可伸缩的架构和持续交付，使得系统更具有灵活性和敏捷性。团队能够更容易地适应新的需求和变化。
三、使用 jenkins+GitLab+ansible+shell 组合拳实现一键同时在上百台服务器实现代码同步
注意事项：
这套组合拳一键自动部署代码同步到上百台服务器，大量的节省了运维成本和时间成本，有更多的时间用来摸鱼，千万别被老板发现哦
小编完全根据公司的实际业务编写的，如果有正在学习的小伙伴也完全可以用来学习
jenkins完全使用Pipeline流水线式操作，不懂得小伙伴可以学习一下流水线语法
项目部署：
1、将ansible的主机清单文件放到jenkins，这里会涉及到清单文件的一个插件（当然不止这一个插件），小伙伴可自己网上搜索下载。
2、涉及到的jenkins变量参数
git分支参数
oss对象存储的路径
oss对象存储的包名
ssl使用的主机清单参数
初始化/更新选项参数
3、准备一个oss存储作为备份以及代码中转站
克隆代码将代码打包上传到oss云存储
3.1 这里考虑到是否为新服务器做初始化或者当前服务器的代码更新，分别用了两个脚本
init初始化脚本：在此脚本中安装，部署项目中涉及到的所有环境
#!/bin/bash
formatted_date=$(date "+%y-%m-%d-%H-%M-%S")
project_dir="/eyou"
script_dir="/root/ansible"
images=("eyou-php" "mysql" "nginx")
file_path="/opt/zhanqun.zip"
cron_text="* * * * * curl 127.0.0.1/index.php?m=control&c=Article&a=addArticleByChat"
#判断是否已安装docker
check_docker() {
echo "检查docker版本"
docker --version
if [ $? -ne 0 ];then
echo "docker 未安装,开始安装docker23"
[ -d "/opt/docker_pack" ] || mkdir -p "/opt/docker_pack/docker_offline"
# wget -O /opt/docker_pack/docker_offline.zip http://8.218.7.215/oss/zqinstall/docker_offline.zip
sh $script_dir/oss.sh get zqinstall/docker_offline.zip /opt/docker_pack/docker_offline.zip
yum install -y unzip
unzip -d /opt/docker_pack/docker_offline /opt/docker_pack/docker_offline.zip
yum localinstall /opt/docker_pack/docker_offline/docker_offline/*.rpm -y
systemctl restart docker
fi
docker-compose --version
if [ $? -ne 0 ];then
# wget -O /opt/docker_pack/docker-compose-linux-x86_64 http://8.218.7.215/oss/zqinstall/docker-compose-linux-x86_64
sh $script_dir/oss.sh get zqinstall/docker-compose-linux-x86_64 /opt/docker_pack/docker-compose-linux-x86_64
cp /opt/docker_pack/docker-compose-linux-x86_64 /usr/local/bin/docker-compose
chmod a+x /usr/local/bin/docker-compose
ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
fi
}
#判断eyou目录是否存在
check_dir() {
if [ -d ${project_dir} ]; then
echo "项目目录已存在"
cd ${project_dir}
docker-compose down
CODE=1
cd /root
rm -rf ${project_dir}_old*
mv ${project_dir} ${project_dir}_old_$formatted_date
else
echo "创建项目目录"
mkdir ${project_dir}
fi
}
#重建目录结构
remake_dir() {
echo "正在重建目录结构"
mkdir -p ${project_dir}/www/html
mkdir -p ${project_dir}/www/logs
mkdir -p ${project_dir}/www/conf.d
mkdir -p ${project_dir}/mysql/logs
chmod 777 -R ${project_dir}/mysql/logs
cat << 'EOF' > ${project_dir}/www/conf.d/default.conf
server_tokens off;
server {
listen 80 default_server;
keepalive_timeout 300;
server_name localhost;
root /usr/share/nginx/html;
location / {
index index.php index.html index.htm;
}
location ~ \.php$ {
fastcgi_pass php:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
fastcgi_connect_timeout 300s;
fastcgi_send_timeout 300s;
fastcgi_read_timeout 300s;
}
}
EOF
cat << EOF > ${project_dir}/mysql/my.cnf
[mysqld]
user=mysql
default-storage-engine=INNODB
character-set-server=utf8mb4
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
datadir = /var/lib/mysql
log-error = /var/log/mysql/error.log
bind-address = 0.0.0.0
port=3306
innodb_buffer_pool_size=256M
symbolic-links=0
performance_schema=0
skip-name-resolve=0
sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION
max_allowed_packet=64M
max_connections=200
[mysql]
default-character-set=utf8mb4
[client]
default-character-set=utf8mb4 # 设置mysql客户端默认字符集
EOF
cat << EOF > ${project_dir}/Dockerfile
# 使用官方 PHP 7.3 FPM 镜像作为基础镜像
FROM php:7.3-fpm
# 安装 PHP GD 扩展
RUN apt-get update && apt-get install -y libfreetype6-dev libjpeg62-turbo-dev libpng-dev libzip-dev libxml2-dev openssl libcurl4-openssl-dev libssl-dev
RUN docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/
RUN docker-php-ext-install -j$(nproc) gd mysqli pdo pdo_mysql bcmath fileinfo mbstring curl zip xml
CMD ["php-fpm"]
EOF
cat << EOF > ${project_dir}/docker-compose.yml
version: '3'
services:
mysql:
image: mysql:qky1.0
restart: always
container_name: mysql
networks:
- my-network
environment:
TZ: Asia/Shanghai
ports:
- 3306:3306
volumes:
- ${project_dir}/mysql/data:/var/lib/mysql
- ${project_dir}/mysql/my.cnf:/etc/mysql/my.cnf
- ${project_dir}/mysql/logs:/var/log/mysql
# - ${project_dir}/mysql/init_mysql_user.sh:/init-mysql-user.sh
- ${project_dir}/mysql/init.sql:/docker-entrypoint-initdb.d/init.sql
- ${project_dir}/mysql/eu.sql:/docker-entrypoint-initdb.d/eu.sql
command:
--max_connections=1000
--character-set-server=utf8mb4
--collation-server=utf8mb4_general_ci
--default-authentication-plugin=mysql_native_password
# entrypoint: /init-mysql-user.sh
nginx:
image: nginx:qky1.0
container_name: "compose-nginx"
restart: always
ports:
- "80:80"
- "443:443"
environment:
- TZ=Asia/Shanghai
depends_on:
- "php"
volumes:
- "${project_dir}/www/conf.d/:/etc/nginx/conf.d/"
- "${project_dir}/www/html:/usr/share/nginx/html"
- "${project_dir}/www/logs:/var/log/nginx"
networks:
- my-network
php:
image: eyou-php:qky1.0
# build:
# context: .
# dockerfile: Dockerfile
container_name: "compose-php"
restart: always
ports:
- "9000:9000"
environment:
- TZ=Asia/Shanghai
volumes:
- "${project_dir}/www/html:/usr/share/nginx/html"
networks:
- my-network
networks:
my-network:
driver: bridge
EOF
}
download_image() {
[ -d "/opt/docker_image" ] || mkdir -p "/opt/docker_image"
# 检查每个镜像是否存在
for image in "${images[@]}"; do
if ! docker images | grep -q "$image"; then
echo "镜像 $image 不存在，正在下载并导入..."
sh $script_dir/oss.sh get zqinstall/image/image-${image}.tar.gz /opt/docker_image/image-${image}.tar.gz
docker load -i /opt/docker_image/image-${image}.tar.gz
else
echo "镜像 $image 已存在"
fi
done
}
unzip_pkg() {
# 检查文件是否存在
rm -rf /opt/zhanqun.zip
if [ ! -f "/opt/zhanqun.zip" ]; then
echo "项目代码不存在,正在下载..."
sh $script_dir/oss.sh get zqclient1/allgroup/2023-12-20/generate/client99.zip /opt/zhanqun.zip
else
echo "文件已存在"
fi
unzip -q -d ${project_dir}/www/html/ /opt/zhanqun.zip
chmod 777 -R ${project_dir}/www/html
sh $script_dir/oss.sh get zqinstall/config/eu.sql ${project_dir}/mysql/eu.sql
}
change_conf() {
echo "修改项目config"
sed -i "s/'hostname' *=> *'[^']*'/'hostname' => 'mysql'/" ${project_dir}/www/html/application/database.php
}
add_cron() {
if grep -q "$cron_text" /var/spool/cron/root
then
:
else
echo "$cron_text" >> /var/spool/cron/root
fi
}
check_docker
check_dir
remake_dir
download_image
unzip_pkg
change_conf
cd ${project_dir}
echo "修改伪静态"
sed -i "s/.*index.html.*/\tindex index.php index.html index.htm;/" ${project}/www/conf.d/default.conf
echo "开始运行compose"
docker-compose up -dupdate脚本：用来在当前服务器更新代码，涉及到从oss拉取代码，这里编写了oss拉取/上传代码的脚本
#!/bin/bash
#
project_dir="/eyou"
script_dir="/root/ansible"
file_path="/opt/zhanqun.zip"
#停止compose
stop_compose() {
echo "正在停止compose-nginx..."
cd ${project_dir}
docker-compose down
echo "compose-nginx 已停止"
}
#下载项目代码
download_code() {
echo "正在替换项目代码..."
rm -rf /eyou/www/html_*
mv /eyou/www/html /eyou/www/html_bak
rm -rf /opt/zhanqun.zip
if [ ! -f "/opt/zhanqun.zip" ]
then
echo "下载项目代码..."
sh $script_dir/oss.sh get zqcode/zhanqun1.2.zip /opt/zhanqun.zip
else
echo "项目代码已存在"
fi
}
#解压替换项目代码
replace_code() {
unzip -q -d ${project_dir}/www/html/ /opt/zhanqun.zip
#保留旧的日志文件，防止覆盖
find ${project_dir}/www/html_bak/application/control/util/ -type f -name "*.log" -exec cp {} ${project_dir}/www/html/application/control/util/ \;
chmod 777 -R ${project_dir}/www/html
}
start_compose() {
cd ${project_dir}
docker-compose up -d
}
stop_compose
download_code
replace_code
start_composeoss脚本
#!/bin/bash
host="oss-cn-hongkong.aliyuncs.com"
bucket="zq-xg"
Id="LTAI5tNjacthSuvSsU8CeSEX"
Key="I31h0zsBnRPVqQ2cxL2ayiVsyPPy6M"
method=$1
source=$2
dest=$3
osshost=$bucket.$host
if test -z "$method"
then
method=GET
fi
if [ "${method}"x = "get"x ] || [ "${method}"x = "GET"x ]
then
method=GET
elif [ "${method}"x = "put"x ] || [ "${method}"x = "PUT"x ]
then
method=PUT
else
method=GET
fi
if test -z "$dest"
then
dest=$source
fi
echo "method:"$method
echo "source:"$source
echo "dest:"$dest
if test -z "$method" || test -z "$source" || test -z "$dest"
then
echo $0 put localfile objectname
echo $0 get objectname localfile
exit -1
fi
if [ "${method}"x = "PUT"x ]
then
resource="/${bucket}/${dest}"
contentType=`file -ib ${source} |awk -F ";" '{print $1}'`
dateValue="`TZ=GMT date +'%a, %d %b %Y %H:%M:%S GMT'`"
stringToSign="${method}\n\n${contentType}\n${dateValue}\n${resource}"
signature=`echo -en $stringToSign | openssl sha1 -hmac ${Key} -binary | base64`
echo $stringToSign
echo $signature
url=http://${osshost}/${dest}
echo "upload ${source} to ${url}"
curl -i -q -X PUT -T "${source}" \
-H "Host: ${osshost}" \
-H "Date: ${dateValue}" \
-H "Content-Type: ${contentType}" \
-H "Authorization: OSS ${Id}:${signature}" \
${url}
else
resource="/${bucket}/${source}"
contentType=""
dateValue="`TZ=GMT date +'%a, %d %b %Y %H:%M:%S GMT'`"
stringToSign="${method}\n\n${contentType}\n${dateValue}\n${resource}"
signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${Key} -binary | base64`
url=http://${osshost}/${source}
echo "download ${url} to ${dest}"
curl --create-dirs \
-H "Host: ${osshost}" \
-H "Date: ${dateValue}" \
-H "Content-Type: ${contentType}" \
-H "Authorization: OSS ${Id}:${signature}" \
${url} -o ${dest}
fi4、 考虑到要在十几台甚至上百台服务器同时执行此脚本，这里使用了ansible工具
通过playbook剧本执行init初始化脚本或者update脚本，来实现多台服务器的同时部署
init初始化剧本，考虑到新服务器域名需要证书，编写ssl脚本来给域名颁发证书
---
- name: do_init_install
hosts: cncn
remote_user: root
tasks:
- name: check_ansible_dir
stat:
path: /root/ansible
register: dir_status
- name: mkdir(ansible_dir_notexist)
file:
path: /root/ansible
state: directory
when: not dir_status.stat.exists
- name: check_install.sh
stat:
path: /root/ansible/install.sh
register: script_status
- name: check_ssl.sh
stat:
path: /root/ansible/ssl.sh
register: ssl_status
- name: check_oss.sh
stat:
path: /root/ansible/oss.sh
register: oss_status
- name: check_add_cron.sh
stat:
path: /root/ansible/add_cron.sh
register: add_cron_status
- name: check_iplist
stat:
path: /root/ansible/iplist
register: iplist_status
- name: cp_install.sh(install.sh_notexist)
copy:
src: /var/jenkins_home/zqscript/install.sh
dest: /root/ansible/install.sh
mode: '0755'
when: not script_status.stat.exists
- name: cp_ssl.sh(ssl.sh_notexist)
copy:
src: /var/jenkins_home/zqscript/ssl.sh
dest: /root/ansible/ssl.sh
mode: '0755'
when: not ssl_status.stat.exists
- name: cp_oss.sh(oss.sh_notexist)
copy:
src: /var/jenkins_home/zqscript/oss.sh
dest: /root/ansible/oss.sh
mode: '0755'
when: not oss_status.stat.exists
- name: cp_add_cron.sh(add_cron_notexist)
copy:
src: /var/jenkins_home/zqscript/add_cron.sh
dest: /root/ansible/add_cron.sh
mode: '0755'
when: not add_cron_status.stat.exists
- name: cp_iplist(iplist_notexist)
copy:
src: /var/jenkins_home/zqscript/iplist
dest: /root/ansible/iplist
mode: '0755'
when: not iplist_status.stat.exists
- name: run install.sh on dest
shell: /root/ansible/install.sh
register: script_output
- name: show script output
debug:
var: script_output.stdout_lines
- name: run ssl.sh on dest
shell: /root/ansible/ssl.sh
register: ssl_output
- name: show ssl output
debug:
var: ssl_output.stdout_lines
- name: run add_cron.sh on dest
shell: /root/ansible/add_cron.sh
register: add_cron_output
- name: show add_cron output
debug:
var: add_cron_output.stdout_linesssl证书脚本，将对应的ip和域名写到jenkins的iplist内即可使用此脚本
#!/bin/bash
# 域名
LOCAL_IP=`curl -sS ip.sb`
DOMAIN=`cat /root/ansible/iplist | grep $LOCAL_IP | awk -F " " '{print $1}'`
CONTAINER_NAME="compose-nginx"
CERT_PATH=`ls -l /etc/letsencrypt/live/ | grep $DOMAIN | awk -F " " '{print $9}' | tail -1`
CERT_DIR="/etc/letsencrypt/live/$CERT_PATH"
CON_CERT_DIR="/etc/nginx/conf.d"
EMAIL="[email protected]"
NGINX_CONF="/eyou/www/conf.d/default.conf"
# 检查 Certbot
if ! command -v certbot &> /dev/null; then
yum install epel-release -y
yum install certbot python2-certbot-nginx -y
fi
#停止nginx容器,暂时释放80端口，供Certbot使用80 验证域名
docker stop $CONTAINER_NAME
# 创建证书前判断目录
if [ -d ${CERT_DIR} ]; then
rm -rf ${CERT_DIR}
fi
# 使用 Certbot 创建证书
certbot certonly --standalone -d $DOMAIN --email $EMAIL --agree-tos --no-eff-email
# 再次定义变量
CERT_PATH=`ls -l /etc/letsencrypt/live/ | grep $DOMAIN | awk -F " " '{print $9}' | tail -1`
CERT_DIR="/etc/letsencrypt/live/$CERT_PATH"
# 复制证书到容器
cp $CERT_DIR/fullchain.pem /eyou/www/conf.d/
cp $CERT_DIR/privkey.pem /eyou/www/conf.d/
# 更新Nginx配置文件
#sed -i "2s/.*/\tlisten 443 ssl;/" $NGINX_CONF
#sed -i "3s/.*/\tserver_name $DOMAIN ;/" $NGINX_CONF
#if grep -q "ssl_certificate" $NGINX_CONF; then
# sed -i "s#ssl_certificate.*#ssl_certificate $CON_CERT_DIR/fullchain.pem;#" $NGINX_CONF
#else
# sed -i "4i ssl_certificate $CON_CERT_DIR/fullchain.pem;" $NGINX_CONF
#fi
#if grep -q "ssl_certificate_key" $NGINX_CONF; then
# sed -i "s#ssl_certificate_key.*#ssl_certificate $CON_CERT_DIR/privkey.pem;#" $NGINX_CONF
#else
# sed -i "5i ssl_certificate_key $CON_CERT_DIR/privkey.pem;" $NGINX_CONF
#fi
#count=$(grep -o "ssl_certificate" "$NGINX_CONF" | wc -l)
#if [ "$count" -gt 2 ]; then
# # 使用 tac 命令反转文件行，删除最后出现的 ssl_certificate 行，再反转回来
# tac "$NGINX_CONF" | sed '0,/^ssl_certificate/{//d;}' | tac > "$NGINX_CONF.tmp"
# mv "$NGINX_CONF.tmp" "$NGINX_CONF"
#fi
#这里不选择正则匹配了，而是直接写入
cat << 'EOF' > $NGINX_CONF
server_tokens off;
server {
listen 80;
server_name 正则匹配修改;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
server_name 正则匹配修改;
ssl_certificate /etc/nginx/conf.d/fullchain.pem;
ssl_certificate_key /etc/nginx/conf.d/privkey.pem;
keepalive_timeout 300;
root /usr/share/nginx/html;
location / {
if (!-e $request_filename) {
rewrite ^(.*)$ /index.php?s=/$1 last;
break;
}
index index.php index.htm index.html;
}
location ~ \.php$ {
fastcgi_pass php:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
fastcgi_connect_timeout 300s;
fastcgi_send_timeout 300s;
fastcgi_read_timeout 300s;
}
}
EOF
sed -i "s/server_name.*/server_name $DOMAIN;/g" $NGINX_CONF
# 重新启动容器使证书生效
docker restart $CONTAINER_NAME
update更新剧本
---
- name: do_code_update
hosts: cncn
remote_user: root
tasks:
- name: check_ansible_dir
stat:
path: /root/ansible
register: dir_status
- name: mkdir(ansible_dir_notexist)
file:
path: /root/ansible
state: directory
when: not dir_status.stat.exists
- name: check_update.sh
stat:
path: /root/ansible/update.sh
register: update_script_status
- name: check_oss.sh
stat:
path: /root/ansible/oss.sh
register: oss_status
- name: cp_update.sh(update.sh_notexist)
copy:
src: /var/jenkins_home/zqscript/update.sh
dest: /root/ansible/update.sh
mode: '0755'
when: not update_script_status.stat.exists
- name: cp_oss.sh(oss.sh_notexist)
copy:
src: /var/jenkins_home/zqscript/oss.sh
dest: /root/ansible/oss.sh
mode: '0755'
when: not oss_status.stat.exists
- name: run update.sh on dest
shell: /root/ansible/update.sh
register: update_script_output
- name: show script output
debug:
var: update_script_output.stdout_lines5、所有剧本已经准备完毕，使用jenkins调用ansible来执行脚本
5.1 将旧的脚本删除，将install_bak模版脚本拷贝成新的脚本，并且将新下载的oss名替换为jenkins赋予的新名称
5.2 更新ansible的主机清单和ssl域名列表，流水线调用ansiblePlaybook语法，触发ansible，执行脚本，实现一键式初始化或更新
6、更新代码构建jenkins
构建页面
构建步骤
总结：
以上就是整个组合拳的流程，当有新的服务器需要部署的时候只需要在主机清单文件添加对应的域名和ip地址，在jenkins的hosts文件将新服务器的IP添加进去，定义一个组，组内可以有上百上千台服务器，只需要填写组名，即可构建任务自动发布。
环境脚本可以根据自己实际服务器的环境来编写，大体流程为jenkins控制ansible执行shell脚本。
不要轻易尝试哦，不然会很清闲的。
------制作不易，请给小编一点支持，后续会更新更有意思的技术文章哦------