sleetdream

Jenkins Docker HARBOR集成

一、系统环境

组件	版本
Centos	7.6.1810
Jenkins	2.319.1
Docker	20.10.12
Docker-Compose	1.29.2
HARBOR	2.4.1

二、学习资料

《每天5分钟玩转Docker容器技术》
HARBOR官网

三、Docker安装

1、Docker

官方文档

2、Docker-Compose

官方文档

四、HARBOR安装

1、下载安装包

下载地址
【注意】
1、HARBOR分1.0和2.0两个大版本，根据自己需要选择版本
2、安装包分离线和在线安装版，我们选择离线安装包

2、官方文档

安装配置

3、解压安装包

sudo tar -zxvf harbor-offline-installer-v2.4.1.tgz

4、配置文件

进入HARBOR安装目录，对harbor.yml文件进行配置，下面有"配置"注释的行需要修改

# Configuration file of Harbor

# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
# 配置 设定主机名称，可以ip或者域名
hostname: 192.168.x.x

# http related config
http:
  # port for http, default is 80. If https enabled, this port will redirect to https port
  # 配置 设定http端口号，此处仅启用http方式，如需https方式请查阅相关文档
  port: 8182

# https related config
# https:
  # https port for harbor, default is 443
  # port: 443
  # The path of cert and key files for nginx
  # certificate: /your/certificate/path
  # private_key: /your/private/key/path

# # Uncomment following will enable tls communication between all harbor components
# internal_tls:
#   # set enabled to true means internal tls is enabled
#   enabled: true
#   # put your cert and key files on dir
#   dir: /etc/harbor/tls/internal

# Uncomment external_url if you want to enable external proxy
# And when it enabled the hostname will no longer used
# external_url: https://reg.mydomain.com:8433

# The initial password of Harbor admin
# It only works in first time to install harbor
# Remember Change the admin password from UI after launching Harbor.
# 配置 harbor管理员admin的登录密码
harbor_admin_password: Harbor12345

# Harbor DB configuration
database:
  # The password for the root user of Harbor DB. Change this before any producti
on use.
  # 配置 harbor数据库密码
  password: root123
  # The maximum number of connections in the idle connection pool. If it <=0, no
 idle connections are retained.
  max_idle_conns: 100
  # The maximum number of open connections to the database. If it <= 0, then the
re is no limit on the number of open connections.
  # Note: the default number of connections is 1024 for postgres of harbor.
  max_open_conns: 900

# The default data volume
# 配置 HARBOR的DockerVolume位置
data_volume: ~/opt/harbor/data
......

5、安装

# 切换到harbor目录下
# 准备阶段
./prepare
# 安装阶段
./install.sh

【注意】HARBOR为Docker一键安装，安装成功后会启动相关镜像

6、登录

访问HARBOR地址，如：192.168.x.x:8182，端口号为HARBOR配置文件中指定

7、创建项目

在HARBOR中新建一个项目，如：demo-hello

8、Jenkins部署节点Docker中添加私有镜像库

# 添加私有镜像库
vim /usr/lib/systemd/system/docker.service

ExecStart后添加–insecure-registry 192.168.x.x:8182

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket containerd.service

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
l
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sockk
 --insecure-registry 192.168.x.x:8182

五、Jenkins Build Docker镜像

1、Dockerfile

# 定制的镜像都是基于 FROM 的镜像，这里的openjdk:8-jdk-alpine就是基础镜像
FROM openjdk:8-jdk-alpine
# 构建参数
ARG JAR_FILE
# 复制指令，将指定文件复制为app.jar
COPY ${JAR_FILE} app.jar
# 声明对外暴露的端口
EXPOSE 8214
# 配置容器启动时运行的命令
ENTRYPOINT ["java","-jar","/app.jar"]

将Dockerfile放到代码目录下，通常和pom.xml在同一目录下

2、Pipeline

pipeline {

    /*Jenkins节点，any代表所有*/
//    agent any
    agent {
        node {
            //在label为dev1的节点进行部署
            label 'dev1'
        }
    }

    /*环境变量，类似全局变量*/
    environment {
        BUILD_USER = "" //项目构建者
        GIT_COMMIT_MSG = "" //GIT提交信息
        GIT_COMMIT_ID = "" //GIT提交ID，可用于标识版本

        /*部署配置*/
        POM_PATH = "${env.WORKSPACE}/pom.xml" //配置文件路径
        POM_ARTIFACTID = "" //项目名称
        POM_VERSION = "" //项目版本
        POM_PROJECT_NAME = "" //项目原名：POM_ARTIFACTID+'-'+POM_VERSION+'.jar'，用于杀死上一版本。值为空则读取Pom.xml中上个版本的jar，值不为空则指定杀死进程。

        JAR_NAME = "" //jar包名称
        JAR_PATH = "${env.WORKSPACE}/target" //生成的jar包路径


        /*部署配置*/
        JAR_WORK_PATH = "~/jenkins/jar/" //运行jar的工作路径，统一管理，并需要提前创建好
        /*部署配置*/
        LOG_PATH = "~/jenkins/log" //日志路径
        /*部署配置*/
        DOCKER_PORT = "8214" //程序运行端口号
    }

//    /*Jenkins自动构建触发器*/
//    triggers{
//        //每5分钟判断一次代码是否有变化
//        pollSCM('H/5 * * * *')
//    }

    /*构建阶段*/
    stages {
        /*准备阶段：拉取代码、定义全局变量等*/
        stage('Preparation') {
            steps {
                //使用build user vars插件，获取构建执行者
                wrap([$class: 'BuildUser']) {
                    script {
                        BUILD_USER = "${env.BUILD_USER}" //将构建执行者注入到环境变量中，方便最后通知使用
                    }
                }

                /*部署配置*/
                /** 从Bitbucket上拉取分支
                 * @url git地址
                 * @branch 分支名称
                 * @credentialsId Jenkins凭证Id，用于远程访问
                 */
                git(url: 'https://[email protected]/sleetdream/demo-hello.git', branch: 'master', credentialsId: 'sleetdream')

                script {
                    //执行Git命令获取Git相关信息赋值给全局变量，returnStdout返回命令结果
                    GIT_COMMIT_MSG = sh(script: 'git log -1 --pretty=%B ${GIT_COMMIT}', returnStdout: true).trim()
                    GIT_COMMIT_ID = sh(script: 'git rev-parse --short HEAD', returnStdout: true ).trim()
                }

                /*读取pom.xml文件，设置全局变量*/
                readRom()
            }
        }

        /*构建阶段*/
        stage('Build'){
            steps{
                /**
                 * 执行maven打包
                 * -B --batch-mode 在非交互（批处理）模式下运行(该模式下,当Mven需要输入时,它不会停下来接受用户的输入,而是使用合理的默认值)
                 * 打包时跳过JUnit测试用例
                 * -DskipTests 不执行测试用例，但编译测试用例类生成相应的class文件至target/test-classes下
                 * -Dmaven.test.skip=true，不执行测试用例，也不编译测试用例类
                 **/
                sh 'mvn -B -DskipTests clean package'
            }
        }

        stage('Docker Build') {
            steps {
                withEnv(['JENKINS_NODE_COOKIE=background_job']) {
                    sh """
pid=\$(docker ps -a | grep ${POM_ARTIFACTID} | awk \'{print \$1 }\')
image=\$(docker images | grep ${POM_ARTIFACTID} | awk \'{ print \$3 }\')

if [ -z "\$pid" ]
then
    echo container ${POM_ARTIFACTID} is already stopped
else
    docker stop \$pid
    docker rm \$pid
fi

if [ -z "\$image" ]
then
    echo image ${POM_ARTIFACTID}:${POM_VERSION} is already stopped
else
    docker rmi \$image
fi


# 创建默认路径
mkdir -p ${JAR_WORK_PATH}
cp -f ${JAR_PATH}/${JAR_NAME} ${JAR_WORK_PATH}
cp -f ${env.WORKSPACE}/Dockerfile ${JAR_WORK_PATH}
cd ${JAR_WORK_PATH}
docker build --build-arg JAR_FILE=${POM_PROJECT_NAME} -t ${POM_ARTIFACTID}:${POM_VERSION} .
docker run --name=${POM_ARTIFACTID} -p ${DOCKER_PORT}:${DOCKER_PORT} -d -v ${LOG_PATH}:/log ${POM_ARTIFACTID}:${POM_VERSION} 
                """
                }
            }
        }
    }
}


/**
 * 读取配置文件，获取信息
 * @return
 */
def readRom(){
    def pom = readMavenPom file: "${POM_PATH}" //使用Jenkins插件pipeline-utility-steps读取pom.xml文件，使用方法详见https://www.jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#readmavenpom-read-a-maven-project-file

    //设置全局变量
    POM_VERSION = "${pom.version}"
    POM_PROJECT_NAME = "${pom.artifactId}" + '-' + "${pom.version}" + '.jar'
    POM_ARTIFACTID = "${pom.artifactId}"
    JAR_WORK_PATH += POM_ARTIFACTID + '/'
    JAR_NAME = POM_ARTIFACTID + '-' + POM_VERSION + '.jar'
}

六、镜像推送HARBOR

1、Pipeline

/*在Docker Build的stage后增加Harbor PUSH的stage*/
stage('Harbor PUSH'){
    steps {
        // 使用Jenkins凭证Harbor-dev，登录Harbor，并上传镜像
        withCredentials([usernamePassword(credentialsId: "Harbor-dev", passwordVariable: 'password', usernameVariable: 'username')]) {
            // 登录HARBOR
            sh "docker login -u ${username} -p ${password} http://192.168.x.x:8182/"
            // 打标签
            sh "docker tag ${POM_ARTIFACTID}:${POM_VERSION} 192.168.x.x:8182/demo-hello/${POM_ARTIFACTID}:${POM_VERSION}"
            // 上传镜像
            sh "docker push 192.168.x.x:8182/demo-hello/${POM_ARTIFACTID}:${POM_VERSION}"
            // 删除本地镜像
            sh "docker rmi 192.168.x.x:8182/demo-hello/${POM_ARTIFACTID}:${POM_VERSION}"
        }
    }
}

2、Jenkins中添加HARBOR凭证

用户名：admin
密码：Harbor12345

3、Jenkins执行构建

在HARBOR中查看Push的镜像

七、参考鸣谢

[Ubuntu 20.04搭建Harbor 2.2.1私有仓库笔记记录]
解决docker删除镜像时image is referenced in multiple repositories
Docker harbor私有仓库部署与管理
Harbor仓库搭建及简单使用
部署Harbor私有镜像仓库（无坑）！
Ubuntu16.04 安装 Docker 及 docker-compose
Docker 添加–insecure-registry 私有镜像仓库

八、常见问题

1、image is referenced in multiple repositories

# 执行如下命令发生错误
docker rmi ee3742b7b6e7

Docker删除镜像时发现，两个镜像tag共用一个镜像id。如：为一个镜像打了两个标签
此时按repository和tag删除镜像即可，如：

docker rmi demo-hello:latest

2、Error response from daemon: Get https://xxxx.xxxx.xxx/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

方法一

# 添加私有镜像库
vim /usr/lib/systemd/system/docker.service