Harbor 企业级docker镜像私服

Harbor 企业级docker镜像私服

一、介绍

​ Harbor是由VMware公司开源的企业级的Docker Registry管理项目，它包括权限管理(RBAC)、LDAP、日志审核、管理界面、自我注册、镜像复制和中文支持等功能。

​ 下载地址：https://github.com/goharbor/harbor/releases。

​

​ Harbor 共有六大组件，全部基于docker运行，六大组件包括：

​ proxy：nginx前端代理，用来分发前端页面ui访问和镜像上传和下载流量
​ ui：提供了一个web管理页面，还包括一个前端页面和后端API，底层使用mysql数据库
​ registry：镜像仓库，负责存储镜像文件，当镜像上传完毕后通过hook通知ui创建repository，registry的token认证通过ui组件完成adminserver：系统配置管理中心附带检查存储用量，ui和jobserver启动时候需要加载adminserver的配置
​ jobsevice：负责镜像复制工作的，他和registry通信，从一个registry pull镜像然后push到另一个registry，并记录job_log
​ log：日志汇总组件，通过docker的log-driver把日志汇总到一起

二、安装环境准备

​ Harbor 安装前需要安装 docker、docker-ce、docker-compose 环境（安装方式见相应的章节）。

​ 版本检测（本例安装的均为截止目前最新版本）：

[root@cnkanon ~]# docker -v
Docker version 18.09.8, build 0dd43dd87f
[root@cnkanon ~]# docker-compose -v
docker-compose version 1.24.0, build 0aa59064

三、安装 Harbor

​ 这里说明一下，网上很多文章说的都是v1.5及以下版本的安装方式，和v1.8及以上版本有很大区别，配置文件上就有所不同，v1.5及以下版本配置文件是harbor.cfg，而v1.8.1配置文件是harbor.yml，本例以v1.8.1版本为例安装。

​ Harbor 安装包有 online 和 offline 两种，建议直接下载 offline 安装包，免去后续安装下载的麻烦。

1、下载离线安装包并解压

​ 下载到 /usr/local 目录（后续要安装在 /usr/local/harbor 目录下），并解压：

# 进入 /usr/local 目录
[root@cnkanon ~]# cd /usr/local
# 下载 offline 安装包
[root@cnkanon local]# wget https://storage.googleapis.com/harbor-releases/release-1.8.0/harbor-offline-installer-v1.8.1.tgz
[root@cnkanon local]# ll
总用量 845648
-rw-r--r--. 1 root root 561149414 7月  21 11:31 harbor-offline-installer-v1.8.1.tgz
# 解压
[root@cnkanon harbor]# ll
总用量 551216
-rwxr-xr-x. 1 root root  564403568 6月  17 11:30 harbor.v1.8.1.tar.gz
-rwxr-xr-x. 1 root root       4509 7月  21 20:50 harbor.yml
-rwxr-xr-x. 1 root root       5088 6月  17 11:29 install.sh
-rwxr-xr-x. 1 root root      11347 6月  17 11:29 LICENSE
-rwxr-xr-x. 1 root root       1654 6月  17 11:29 prepare

2、安装资源及环境准备

​ 执行 prepare 脚本，进行相关资源检测和准备，运行完成后会生成 docker-compose.yml 文件和 common 文件夹：

[root@cnkanon harbor]# ./prepare
[root@cnkanon harbor]# prepare base dir is set to /home/system/soft/harbor
[root@cnkanon harbor]# Unable to find image 'goharbor/prepare:v1.8.1' locally
[root@cnkanon harbor]# v1.8.1: Pulling from goharbor/prepare
[root@cnkanon harbor]# Digest: sha256:49542e66f4969f23ef7e1c65119f7e5338da0ba5b9c56e6d1e0ff58d3bb8664f
[root@cnkanon harbor]# Status: Downloaded newer image for goharbor/prepare:v1.8.1
[root@cnkanon harbor]# Generated configuration file: /config/log/logrotate.conf
[root@cnkanon harbor]# Generated configuration file: /config/nginx/nginx.conf
[root@cnkanon harbor]# Generated configuration file: /config/core/env
[root@cnkanon harbor]# Generated configuration file: /config/core/app.conf
[root@cnkanon harbor]# Generated configuration file: /config/registry/config.yml
[root@cnkanon harbor]# Generated configuration file: /config/registryctl/env
[root@cnkanon harbor]# Generated configuration file: /config/db/env
[root@cnkanon harbor]# Generated configuration file: /config/jobservice/env
[root@cnkanon harbor]# Generated configuration file: /config/jobservice/config.yml
[root@cnkanon harbor]# Generated and saved secret to file: /secret/keys/secretkey
[root@cnkanon harbor]# Generated certificate, key file: /secret/core/private_key.pem, cert file: /secret/registry/root.crt
[root@cnkanon harbor]# Generated configuration file: /compose_location/docker-compose.yml
[root@cnkanon harbor]# Clean up the input dir

[root@cnkanon harbor]# ll
总用量 551216
[root@cnkanon harbor]# drwxr-xr-x. 3 root   root        20 7月  21 21:38 common
[root@cnkanon harbor]# -rw-r-----. 1 root   root      5183 7月  21 21:38 docker-compose.yml
[root@cnkanon harbor]# -rwxr-xr-x. 1 system dev  564403568 6月  17 11:30 harbor.v1.8.1.tar.gz
[root@cnkanon harbor]# -rwxr-xr-x. 1 system dev       4509 7月  21 20:50 harbor.yml
[root@cnkanon harbor]# -rwxr-xr-x. 1 system dev       5088 6月  17 11:29 install.sh
[root@cnkanon harbor]# -rwxr-xr-x. 1 system dev      11347 6月  17 11:29 LICENSE
[root@cnkanon harbor]# -rwxr-xr-x. 1 system dev       1654 6月  17 11:29 prepare

​ 修改 harbor.yml 配置文件如下：

# 设置 harbor 服务监听地址，由于需要对外提供服务，不可填写 127.0.0.1 或 localhost，填写本机IP地址
hostname = 192.168.56.3

# 设置 harbor 服务监听端口，实际上 harbor 镜像暴露的端口号
http:
  port: 80

# 设置 harbor 基于 https 监听的端口和证书、私钥信息（http 和 https 只需要设置一种即可）
#https:
#  port: 443
    # The path of cert and key files for nginx
#    certificate: /usr/local/harbor/cert/full_chain.perm
#    private_key: /usr/local/harbor/cert/private.key

# 设置管理界面初始访问密码
harbor_admin_password: 123456

3、安装

​ 一切就绪后，执行 install.sh 脚本安装：

[root@cnkanon harbor]# ./install.sh 
[root@cnkanon harbor]# [Step 0]: checking installation environment ...
[root@cnkanon harbor]# Note: docker version: 18.09.8
[root@cnkanon harbor]# Note: docker-compose version: 1.24.0
[root@cnkanon harbor]# [Step 1]: loading Harbor images ...
[root@cnkanon harbor]# Loaded image: goharbor/harbor-core:v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/harbor-registryctl:v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/redis-photon:v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/notary-server-photon:v0.6.1-v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/chartmuseum-photon:v0.8.1-v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/harbor-db:v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/harbor-jobservice:v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/nginx-photon:v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/registry-photon:v2.7.1-patch-2819-v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/harbor-migrator:v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/prepare:v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/harbor-portal:v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/harbor-log:v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/notary-signer-photon:v0.6.1-v1.8.1
[root@cnkanon harbor]# Loaded image: goharbor/clair-photon:v2.0.8-v1.8.1
[root@cnkanon harbor]# [Step 2]: preparing environment ...
[root@cnkanon harbor]# prepare base dir is set to /home/system/soft/harbor
[root@cnkanon harbor]# Clearing the configuration file: /config/log/logrotate.conf
[root@cnkanon harbor]# Clearing the configuration file: /config/nginx/nginx.conf
[root@cnkanon harbor]# Clearing the configuration file: /config/core/env
[root@cnkanon harbor]# Clearing the configuration file: /config/core/app.conf
[root@cnkanon harbor]# Clearing the configuration file: /config/registry/config.yml
[root@cnkanon harbor]# Clearing the configuration file: /config/registryctl/env
[root@cnkanon harbor]# Clearing the configuration file: /config/registryctl/config.yml
[root@cnkanon harbor]# Clearing the configuration file: /config/db/env
[root@cnkanon harbor]# Clearing the configuration file: /config/jobservice/env
[root@cnkanon harbor]# Clearing the configuration file: /config/jobservice/config.yml
[root@cnkanon harbor]# Generated configuration file: /config/log/logrotate.conf
[root@cnkanon harbor]# Generated configuration file: /config/nginx/nginx.conf
[root@cnkanon harbor]# Generated configuration file: /config/core/env
[root@cnkanon harbor]# Generated configuration file: /config/core/app.conf
[root@cnkanon harbor]# Generated configuration file: /config/registry/config.yml
[root@cnkanon harbor]# Generated configuration file: /config/registryctl/env
[root@cnkanon harbor]# Generated configuration file: /config/db/env
[root@cnkanon harbor]# Generated configuration file: /config/jobservice/env
[root@cnkanon harbor]# Generated configuration file: /config/jobservice/config.yml
[root@cnkanon harbor]# loaded secret from file: /secret/keys/secretkey
[root@cnkanon harbor]# Generated configuration file: /compose_location/docker-compose.yml
[root@cnkanon harbor]# Clean up the input dir
[root@cnkanon harbor]# [Step 3]: starting Harbor ...
[root@cnkanon harbor]# Creating network "harbor_harbor" with the default driver
[root@cnkanon harbor]# Creating harbor-log ... 
[root@cnkanon harbor]# Creating registryctl ... 
[root@cnkanon harbor]# Creating registry    ... 
[root@cnkanon harbor]# Creating harbor-db   ... 
[root@cnkanon harbor]# Creating redis       ... 
[root@cnkanon harbor]# Creating harbor-core ... 
[root@cnkanon harbor]# Creating harbor-portal ... 
[root@cnkanon harbor]# Creating harbor-jobservice ... 
[root@cnkanon harbor]# Creating nginx             ... 
[root@cnkanon harbor]# ? ----Harbor has been installed and started successfully.----
[root@cnkanon harbor]# Now you should be able to visit the admin portal at http://192.168.56.3. 
[root@cnkanon harbor]# For more details, please visit https://github.com/goharbor/harbor .

​ 安装完成后，会看到通过 docker-compose 启动了 9 个容器：

[root@cnkanon harbor]# docker-compose ps
      Name                     Command                  State                 Ports          
---------------------------------------------------------------------------------------------
harbor-core         /harbor/start.sh                 Up (healthy)                            
harbor-db           /entrypoint.sh postgres          Up (healthy)   5432/tcp                 
harbor-jobservice   /harbor/start.sh                 Up                                      
harbor-log          /bin/sh -c /usr/local/bin/ ...   Up (healthy)   127.0.0.1:1514->10514/tcp
harbor-portal       nginx -g daemon off;             Up (healthy)   80/tcp                   
nginx               nginx -g daemon off;             Up (healthy)   0.0.0.0:80->80/tcp       
redis               docker-entrypoint.sh redis ...   Up             6379/tcp                 
registry            /entrypoint.sh /etc/regist ...   Up (healthy)   5000/tcp                 
registryctl         /harbor/start.sh                 Up (healthy)                            

​ 在浏览器中输入 http://192.168.56.3 可以访问到 Harbor 运行界面，输入 admin/123456 后登录：

​ ***说明：***docker 在 v10.0 版本以上默认使用 https 与镜像仓库通信，

三、使用

1、登录 Harbor

​ 使用如下指令登录 harbor 镜像仓库：

[root@cnkanon harbor]# docker login -u admin 192.168.56.3
Error response from daemon: Get https://192.168.56.3/v2/: dial tcp 192.168.56.3:443: connect: connection refused

​ 如无意外会出现上述错误，原因是 docker 在 v10.0 版本以上默认使用 https 与镜像仓库通信，而全例中配置的 harbor 使用的是 http 协议，导致登录失败。

​ 此处需要对 docker 配置文件进行相应修改，使其支持 http 模式，网上很多文章中的修改方法针对 docker v18.0 版本并不能真正生效，经本人测试，需要做如下修改：

第一步：编辑 /etc/docker/daemon.json 文件，增加如下代码

[root@cnkanon harbor]# vi /etc/docker/daemon.json 
{
  "insecure-registries": ["192.168.56.3"]
}

第二步：编辑 /lib/systemd/system/docker.service 在 ExecStart 后增加 --insecure-registry=192.168.56.3

[root@cnkanon harbor]# vi /lib/systemd/system/docker.service
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecStart=/usr/bin/dockerd -H  fd:// --containerd=/run/containerd/containerd.sock --insecure-registry=192.168.56.3
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always

第三步：很关键，网上很多教程缺少此步骤，编辑 /etc/systemd/system/docker.service.d/10-machine.conf 文件，在 ExecStart 后增加 --insecure-registry=192.168.56.3

[root@cnkanon harbor]# vi /etc/systemd/system/docker.service.d/10-machine.conf
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H unix:///var/run/docker.sock --insecure-registry=192.168.56.3 --storage-driver overlay2 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=generic
Environment=

第四步：加载配置，重启 docker

[root@cnkanon harbor]# systemctl daemon-reload
[root@cnkanon harbor]# systemctl restart docker

第四步：验证，重启后查看 docker 状态，会发现在 CGroup 中，/usr/bin/dockerd 命令中增加了 --insecure-registry=192.168.56.3

[root@cnkanon harbor]# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─10-machine.conf
   Active: active (running) since 一 2019-07-22 09:33:39 CST; 4h 59min ago
     Docs: https://docs.docker.com
 Main PID: 19142 (dockerd)
    Tasks: 47
   Memory: 102.0M
   CGroup: /system.slice/docker.service
           ├─19142 /usr/bin/dockerd -H unix:///var/run/docker.sock --insecure-registry=192.168.56.3 --storage-driver overlay2 --tlsverify --tlsca...
           ├─19321 /usr/bin/docker-proxy -proto tcp -host-ip 127.0.0.1 -host-port 1514 -container-ip 172.18.0.5 -container-port 10514
           ├─19373 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 9000 -container-ip 172.17.0.2 -container-port 9000
           └─21025 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 172.18.0.2 -container-port 80

​ 再次登录 Harbor：

[root@cnkanon docker.service.d]# docker login -u admin 192.168.56.3
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

2、向 Harbor 推送镜像

（1）制作镜像

​ 本例中使用 springboot + maven 构建了一个工程，生成最终的文件名为 springboot-docker-ci.jar，在同一目录下编辑 Dockerfile，并执行 docker build 构建镜像。

[root@cnkanon demo]# ll
总用量 36036
-rw-r--r--. 1 system dev      232 7月   2 15:11 Dockerfile
-rw-r--r--. 1 system dev 36894026 7月   2 21:54 springboot-docker-ci.jar

# 编辑 Dockerfile
[root@cnkanon demo]# vi Dockerfile 
FROM java
MAINTAINER qinyong "[email protected]"
WORKDIR /root
# 从本地磁盘中安装jdk
COPY springboot-docker-ci.jar /root/springboot-docker-ci.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "/root/springboot-docker-ci.jar"]

# 构建镜像（最后的 . 表示当前路径，不能少）
[root@cnkanon demo]# docker build -f /root/demo/Dockerfile -t springboot-docker-ci:v1 .

（2）推送镜像

​ 登录 Harbor 后会看到项目中默认有个 library，而且是公开的项目，这里直接使用 admin/123456 账号登录并推送

[root@cnkanon demo]# docker push library/springboot-docker-ci:v1
The push refers to repository [docker.io/library/springboot-docker-ci]
Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 8.8.8.8:53: read udp 192.168.56.3:57410->8.8.8.8:53: i/o timeout

​ 此处会报错，原因是构建的 springboot-docker-ci:v1 镜像默认指向的是 registry-1.docker.io 的，此处需要将镜像 tag 到 Harbor：

[root@cnkanon docker.service.d]# docker tag springboot-docker-ci:v1 192.168.56.3/library/springboot-docker-ci:v1

​ 再次推送镜像到 Harbor：

[root@cnkanon docker.service.d]# docker push 192.168.56.3/library/springboot-docker-ci:v1 
The push refers to repository [192.168.56.3/library/springboot-docker-ci]
2ac436222394: Pushed 
35c20f26d188: Pushed 
c3fe59dd9556: Pushed 
6ed1a81ba5b6: Pushed 
a3483ce177ce: Pushed 
ce6c8756685b: Pushed 
30339f20ced0: Pushed 
0eb22bfb707d: Pushed 
a2ae92ffcd29: Pushed 
v1: digest: sha256:2925516b8269b7bb187e5bdb224afbc700dade1c7a51e21db1881072a7ea3caf size: 2212

​ 登录 Harbor 查看，会发现 library 项目中已经有了 springboot-docker-ci 镜像：

​