微服务CI/CD实践（二）服务器先决准备

微服务CI/CD实践系列：
 
微服务CI/CD实践（一）环境准备及虚拟机创建
微服务CI/CD实践（二）服务器先决准备
微服务CI/CD实践（三）gitlab部署
微服务CI/CD实践（四）nexus3部署
微服务CI/CD实践（五）数据库,redis,nacos等基础中间件部署
微服务CI/CD实践（六）Jenkins部署
微服务CI/CD实践（七）Server服务器环境初始化
微服务CI/CD实践（八）Jenkins + Dokcer 部署微服务后端项目
微服务CI/CD实践（九）Jenkins + Dokcer 部署微服务前端VUE项目
微服务CI/CD实践（十）Minio服务器部署及应用

上一章介绍了物理主机环境准备、基于VMware ESXi 创建虚拟机步骤以及基本的虚拟机规划，此章主要解决部署微服务前后端服务器所需要的软件安装及依赖配置。以下为各个服务器所需要的先决软件：

• gitlab-server： docker
• db-server: docker
• jenkins-server： jdk17(运行环境) jdk8(构建环境) nodejs18 npm pnpm git maven
• nacos-server/nexus3: jdk8 docker docker compose
• minio-server: docker docker compose
• docker-server: docker

一、docker-ce安装

Docker CE是免费的Docker产品，Docker CE包含了完整的Docker平台，非常适合开发人员和运维团队构建容器APP。在Centos下我们使用yum进行安装。

1.1 国内镜像源配置

docker-hub因为政策原因，对国内停止服务，且国内从外网拉取镜像会经常性网络超时，导致安装过程中断，因此我们直接使用国内镜像源提升下载速度。
step1 备份当前yum源

## 备份
sudo mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup

## 还原
sudo cp /etc/yum.repos.d/CentOS-Base.repo.backup /etc/yum.repos.d/CentOS-Base.repo

step2 下载国内yum源配置文件
根据项目实际情况切换合适源，通常首选阿里云的源。我自己在安装过程其中有一台虚机

##  更换为阿里源
sudo curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo

##  更换为163源
sudo curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.163.com/.help/CentOS7-Base-163.repo

##  更换为腾讯源
sudo curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.cloud.tencent.com/repo/centos7_base.repo

##  更换为华为源
sudo curl -o /etc/yum.repos.d/CentOS-Base.repo https://repo.huaweicloud.com/repository/conf/CentOS-7-reg.repo

step3 更新yum源缓存

# 3.清空并生成缓存
yum clean all
yum makecache

step 4 安装docker-ce

# step 1: 安装必要的一些系统工具
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
# Step 2: 添加软件源信息
sudo yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# Step 3
sudo sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
# Step 4: 更新并安装Docker-CE
sudo yum makecache fast
# 默认安装最新版本
sudo yum -y install docker-ce

# 安装指定版本的Docker-CE
# Step 1: 查找Docker-CE的版本:
# yum list docker-ce.x86_64 --showduplicates | sort -r
# sudo yum -y install docker-ce-[VERSION]

# Step 4: 开启Docker服务
sudo service docker start

step5 验证安装

[root@k8s-rancher-node02 docker-sh]# docker -v
Docker version 26.1.4, build 5650f9b

step6 配置镜像加速

/etc/docker/daemon.json
{
    "registry-mirrors": [
        "https://docker.rainbond.cc",
        "https://do.nark.eu.org",
        "https://dc.j8.work",
        "https://docker.m.daocloud.io",
        "https://dockerproxy.com",
        "https://docker.mirrors.ustc.edu.cn",
        "https://dockerproxy.com",
        "https://mirror.baidubce.com",
        "https://docker.nju.edu.cn",
        "https://mirror.iscas.ac.cn",
        "https://docker.nju.edu.cn"
    ]
}
# 生效配置
sudo systemctl daemon-reload
# 重启docker
sudo systemctl restart docker

# 验证是否生效，生效后执行下面命令会看到上述配置。
docker info

二、docker compose安装

step1 下载文档版本
因为是github官网资源，直接使用linux下载容易中断，可以使用浏览器或者其他下载工具下载到本地，使用ftp上传到linux服务器，我在实际操作就是本地完成下载再上传到服务器。

sudo curl -L "https://github.com/docker/compose/releases/download/v2.28.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

step2 配置软链

# 使用ftp上传方式才有下面mv步骤，直接使用下载命令的忽略
cd /usr/local/bin/
mv docker-compose-linux-x86_64 docker-compose

# 给可执行权限
sudo chmod +x /usr/local/bin/docker-compose

# 创建软链接以便于从任何位置调用docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

# 验证安装
[root@k8s-rancher-server ~]# docker-compose --version
Docker Compose version v2.28.1

三、jdk安装及环境变量配置

step1 下载安装包
jenkins 运行时环境 openjdk-17.0.2_linux-x64_bin.tar.gz
工程构建 jdk-8u202-linux-x64.tar.gz
官网下载缓慢还需要注册账户，因此直接使用国内加速源下载：
阿里云
华为云
下载完成使用ftp上传到服务器

step2 解压缩并配置环境变量

cd /usr/lib
tar -zxvf  jdk-8u261-linux-i586.tar.gz
tar -zxvf  openjdk-17.0.2_linux-x64_bin.tar.gz

# 更改路径
mv  jdk-8u261 ../jdk8
mv  openjdk-17.0.2_linux-x64_bin ../jdk17

# 配置环境变量
vim /etc/profile

# 在文件末尾追加如下代码
export JAVA_HOME_8=/usr/lib/jdk8
export JAVA_HOME_17=/usr/lib/jdk17
export PATH=$JAVA_HOME_17/bin:$PATH

# 更新配置
source /etc/profile
#查看是否安装成功 显示如下内容即ok
[root@k8s-rancher-master ~]# java -version
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
openjdk version "17.0.2" 2022-01-18
OpenJDK Runtime Environment (build 17.0.2+8-86)
OpenJDK 64-Bit Server VM (build 17.0.2+8-86, mixed mode, sharing)

四、nodejs安装

Jenkins构建vue项目需要在本地安装nodejs环境：

• nodejs: v18.20.4
• npm: 10.7.0
• pnpm: 9.7.0

4.1 前置依赖

centos7版本系统默认的GLIBC、gcc、make版本都是过低无法满足nodejs版本。低版本下安装完成执行命令会报相应版本错误

[root@master ~]# node -v
node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by node)
node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by node)
node: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by node)
node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by node)
node: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by node)
node: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by node)

4.1.1 升级make

# 升级 make(默认为3 升级为4)
cd /usr/lib
wget http://ftp.gnu.org/gnu/make/make-4.3.tar.gz
tar -xzvf make-4.3.tar.gz && cd make-4.3/
./configure  --prefix=/usr/local/make
make && make install
cd /usr/bin/ && mv make make.bak
ln -sv /usr/local/make/bin/make /usr/bin/make

# 验证安装
[root@k8s-rancher-master lib]# make -v
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

4.1.2 升级gcc

GLIBC的安装依赖gcc编译工具，centos7默认gcc是4.x，需要升级到8.x。

step1 安装SCL
使用SCL+Devtoolset方式安装

yum install centos-release-scl centos-release-scl-rh

/etc/yum.repos.d 下生成 2 个 repo 源文件： CentOS-SCLo-scl.repo 和 CentOS-SCLo-scl-rh.repo
step2 更新SCL源
直接使用默认源安装会报未知错误

[root@localhost yum.repos.d]# yum list all --enablerepo='centos-sclo-rh' | grep "devtoolset-"
Could not retrieve mirrorlist http://mirrorlist.centos.org?arch=x86_64&release=7&repo=sclo-rh error was
14: curl#6 - "Could not resolve host: mirrorlist.centos.org; 未知的错误"


 One of the configured repositories failed (未知),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled
            yum --disablerepo= ...

     4. Disable the repository permanently, so yum won't use it by default. Yum
        will then just ignore the repository until you permanently enable it
        again or use --enablerepo for temporary usage:

            yum-config-manager --disable <repoid>
        or
            subscription-manager repos --disable=<repoid>

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true

Cannot find a valid baseurl for repo: centos-sclo-rh/x86_64

手工修改源配置，访问阿里云镜像站 centos镜像站，找到对应版本，比如我们的是centos7，则选择7，如下图：

进入centos->7>sclo->x86_64页面

修改CentOS-SCLo-scl.repo源 [centos-sclo-sclo]配置
注释原有baseurl
新增baseurl=https://mirrors.aliyun.com/centos/7/sclo/x86_64/sclo/，该地址为上图sclo链接地址

[centos-sclo-sclo]
name=CentOS-7 - SCLo sclo
# baseurl=http://mirror.centos.org/centos/7/sclo/$basearch/sclo/
baseurl=https://mirrors.aliyun.com/centos/7/sclo/x86_64/sclo/
# mirrorlist=http://mirrorlist.centos.org?arch=$basearch&release=7&repo=sclo-sclo
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-SCLo

修改CentOS-SCLo-scl-rh.repo源 [centos-sclo-rh] 配置

[centos-sclo-rh]
name=CentOS-7 - SCLo rh
#baseurl=http://mirror.centos.org/centos/7/sclo/$basearch/rh/
baseurl=https://mirrors.aliyun.com/centos/7/sclo/x86_64/rh/
#mirrorlist=http://mirrorlist.centos.org?arch=$basearch&release=7&repo=sclo-rh
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-SCLo

step3 刷新缓存

$ yum clean all
$ yum makecache

step4 安装gcc

# 列出可选版本
[root@k8s-rancher-master yum.repos.d]# yum list all --enablerepo='centos-sclo-rh' | grep "devtoolset-"
Failed to set locale, defaulting to C
Repository base is listed more than once in the configuration
Repository updates is listed more than once in the configuration
Repository extras is listed more than once in the configuration
Repository centosplus is listed more than once in the configuration
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
devtoolset-8-binutils.x86_64              2.30-55.el7.2         @centos-sclo-rh 
devtoolset-8-gcc.x86_64                   8.3.1-3.2.el7         @centos-sclo-rh 
devtoolset-8-gcc-c++.x86_64               8.3.1-3.2.el7         @centos-sclo-rh 
devtoolset-8-libstdc++-devel.x86_64       8.3.1-3.2.el7         @centos-sclo-rh 
devtoolset-8-runtime.x86_64               8.1-1.el7             @centos-sclo-rh 
devtoolset-10.x86_64                      10.1-0.el7            centos-sclo-rh  
devtoolset-10-annobin.x86_64              9.23-4.el7.1          centos-sclo-rh 
// 省略。。。。。

# 安装gcc8
sudo yum install devtoolset-8-gcc*

# 查看版本
[root@k8s-rancher-master yum.repos.d]# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-8/root/usr --mandir=/opt/rh/devtoolset-8/root/usr/share/man --infodir=/opt/rh/devtoolset-8/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --with-default-libstdcxx-abi=gcc4-compatible --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-8.3.1-20190311/obj-x86_64-redhat-linux/isl-install --disable-libmpx --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC) 

# 配置
vi /etc/profile
# 在末尾追加如下配置
source /opt/rh/devtoolset-8/enable
# 更新配置
source /etc/profile

在安装gcc8时可能出现如下错误：

configure: error: Building GCC requires GMP 4.2+, MPFR 2.4.0+ and MPC 0.8.0+.

按照错误提示安装gcc需要的这三个依赖：GMP 4.2+, MPFR 2.4.0+ and MPC 0.8.0+。

cd /usr/lib
wget ftp://gcc.gnu.org/pub/gcc/infrastructure/gmp-6.1.0.tar.bz2
wget ftp://gcc.gnu.org/pub/gcc/infrastructure/mpfr-3.1.4.tar.bz2  
wget ftp://gcc.gnu.org/pub/gcc/infrastructure/mpc-1.0.3.tar.gz

# 安装gmp
tar -jxvf gmp-6.1.0.tar.bz2
cd gmp-6.1.0
./configure
make
sudo make install

# 安装mpfr
tar -jxvf mpfr-3.1.4.tar.bz2
cd mpfr-3.1.4
./configure
make
sudo make install

# 安装mpc
tar -zxvf mpc-1.0.3.tar.gz
cd mpc-1.0.3
./configure
make
sudo make install

4.1.3 更新libstdc++

gcc安装后不会自动更新libstdc++，需要手工更新版本

# 查找编译gcc时生成的最新动态库
[root@k8s-rancher-master lib64]# find / -name "libstdc++.so*"
/usr/lib/gcc/x86_64-redhat-linux/4.8.2/32/libstdc++.so
/usr/lib/gcc/x86_64-redhat-linux/4.8.2/libstdc++.so
/usr/lib/gcc-7.5.0/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6.0.24
/usr/lib/gcc-7.5.0/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6
/usr/lib/gcc-7.5.0/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so
/usr/lib/gcc-7.5.0/build/stage1-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6.0.24
/usr/lib/gcc-7.5.0/build/stage1-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6
/usr/lib/gcc-7.5.0/build/stage1-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so
/usr/lib/gcc-7.5.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6.0.24
/usr/lib/gcc-7.5.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6
/usr/lib/gcc-7.5.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so
 

# 将上面的最新动态库 libstdc++.so.6.0.30 复制到/usr/lib64目录下：
$cp /usr/lib/gcc-7.5.0/build/stage1-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6.0.24 /usr/lib64
 
cd /usr/lib64
# 把原来的命令做备份
cp libstdc++.so.6 libstdc++.so.6.bak
rm -f libstdc++.so.6

# 重新链接
ln -s libstdc++.so.6.0.26 libstdc++.so.6

4.1.4 更新glibc

wget http://ftp.gnu.org/gnu/glibc/glibc-2.28.tar.gz
tar xf glibc-2.28.tar.gz 
cd glibc-2.28/ && mkdir build  && cd build
 
$ ../configure --prefix=/usr --disable-profile --enable-add-ons --with-headers=/usr/include --with-binutils=/usr/bin
$ make
$ make install
 
# 验证更新
[root@k8s-rancher-master glibc-2.28]# strings /lib64/libc.so.6 | grep GLIBC
// 省略....
GLIBC_2.22
GLIBC_2.23
GLIBC_2.24
GLIBC_2.25
GLIBC_2.26
GLIBC_2.27
GLIBC_2.28
GLIBC_PRIVATE
// 省略....

4.1.5 python3更新

本地下载安装包并上传到服务器，这里使用华为镜像源下载

tar -xvzf  Python-3.8.10.tgz
cd Python-3.8.10
./configure --enable-optimizations
make altinstall
# 验证安装
[root@k8s-rancher-master bin]# python3.8 --version
Python 3.8.10

4.2 node安装

使用tar包安装，下载指定版本安装包并上传到服务器，这里直接使用官网地址。

# 解压安装包
tar -xvf node-v18.20.4-linux-x64.tar.xz
mkdir -p /usr/local/nodejs
mv node-v18.20.4-linux-x64/* /usr/local/nodejs/

# 建立node软链接
ln -s /usr/local/nodejs/bin/node /usr/local/bin
# 建立npm 软链接
ln -s /usr/local/nodejs/bin/npm /usr/local/bin

# 验证安装结果
[root@k8s-rancher-master nodejs]# node -v
v18.20.4
[root@k8s-rancher-master nodejs]# npm -v
10.7.0

# 设置国内淘宝镜像源
npm config set registry https://registry.npm.taobao.org
# 查看设置信息
[root@k8s-rancher-master nodejs]# npm config list
; "user" config from /root/.npmrc

registry = "http://registry.npm.taobao.org/" 

; node bin location = /usr/local/nodejs/bin/node
; node version = v18.20.4
; npm local prefix = /usr/local/nodejs
; npm version = 10.7.0
; cwd = /usr/local/nodejs
; HOME = /root
; Run `npm config ls -l` to show all defaults.

# 安装pnpm，使用npm安装
 npm install -g pnpm
 # 增加环境变量配置
 # 在 ~/.bashrc 或 ~/.bash_profile 文件中添加以下行
 vi  ~/.bashrc
 export PATH="/usr/local/nodejs/bin:$PATH"
 # 生效配置
source ~/.bashrc
# 验证结果
[root@k8s-rancher-master nodejs]# pnpm -v
9.7.0

五、git安装

使用yum安装

yum -y install git
[root@k8s-rancher-master nodejs]# git --version
git version 1.8.3.1

六、maven安装

本地下载安装包并上传到服务器，使用华为源下载

#step1 解压
tar zxf apache-maven-3.8.1-bin.tar.gz
mv apache-maven-3.8.1 /usr/local/maven3.8.1
#step3 修改环境变量， 在/etc/profile中添加以下几行
vi /etc/profile
export MAVEN_HOME=/usr/local/maven3.8.1
export PATH=$MAVEN_HOME/bin:$PATH
#生效配置
source /etc/profile

# 验证安装
[root@k8s-rancher-master local]# mvn -v
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
Maven home: /usr/local/maven3.8.1
Java version: 17.0.2, vendor: Oracle Corporation, runtime: /usr/lib/jdk17
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-1160.el7.x86_64", arch: "amd64", family: "unix"