Docker-学习总结（集群管理-镜像仓库harbor）

harbor

• Harbor是VMware公司开源了企业级Registry项目, 其的目标是帮助用户迅速搭建一个企业级的Docker registry服务。它以Docker公司开源的registry为基础，额外提供了如下功能:
  • Cloud native registry:支持容器镜像和Helm Charts，为云原生环境提供服务
  • Role based access control：基于角色的访问控制
  • Policy based image replication：基于策略的镜像复制
  • Vulnerability Scanning：镜像的漏洞扫描
  • LDAP/AD support：AD/LDAP集成
  • Image deletion & garbage collection：镜像的删除和空间清理
  • Notary：可以保证镜像的真实性
  • Graphical user portal：友好的管理UI
  • Auditing：日志审计
  • RESTful API：提供RESTfull接口易于与外部系统集成
  • Easy deployment：部署简单

实践环境

主机名	IP地址	系统	作用
vmhost	192.168.27.7	rhel7.5	提供rhel的yum源及docker-ce-18.09.6-3.el7.x86_64安装包以及依赖关系的软件
node1	192.168.27.12	rhel7.5	docker管理节点，有docker-machine软件
repository	192.168.27.20	rhel7.5	纯净系统，准备安装镜像仓库harbor

安装软件

docker-ce软件及其依赖性软件：

• docker-ce-18.09.6-3.el7.x86_64.rpm
• docker-ce-cli-18.09.6-3.el7.x86_64.rpm
• containerd.io-1.2.5-3.1.el7.x86_64.rpm
• container-selinux-2.21-1.el7.noarch.rpm
  harbor软件包：
  harbor-offline-installer-v1.8.2.tgz

实践过程

安装harbor

1. 新创建的虚拟机repository配置docker环境，使用node1上的docker-machine，进行安装docker

[root@node1 opt]# vim /etc/hosts
……
192.168.27.20 repository
……
[root@node1 opt]# ssh-copy-id repository
#拷贝密钥给repository虚拟机
[root@repository ~]# vim /etc/os-release
……
ID="centos"
……
[root@node1 opt]# docker-machine create -d generic --engine-install-url http://192.168.27.7/docker/get-docker.s
Running pre-create checks...
Creating machine...
(repository) No SSH key specified. Assuming an existing key at the default location.
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with centos...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-mach
[root@node1 opt]# docker-machine ls
NAME         ACTIVE   DRIVER    STATE     URL                        SWARM   DOCKER     ERRORS
node2        -        generic   Running   tcp://192.168.27.12:2376           v18.09.6
node3        -        generic   Running   tcp://192.168.27.13:2376           v18.09.6
repository   -        generic   Running   tcp://192.168.27.20:2376           v18.09.6
#docker环境已安装完成

2. harbor安装

[root@repository ~]# cd /opt
[root@repository opt]# ls
containerd  harbor-offline-installer-v1.8.2.tgz
[root@repository opt]# tar zxf harbor-offline-installer-v1.8.2.tgz
[root@repository opt]# ls
containerd  harbor  harbor-offline-installer-v1.8.2.tgz
[root@repository opt]# cd harbor/
[root@repository harbor]# ls
harbor.v1.8.2.tar.gz  harbor.yml  install.sh  LICENSE  prepare
#

3. harbor.yml文件内容修改

[root@repository harbor]# vim harbor.yml

# Configuration file of Harbor

# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: reg.mydocker.com
#设置仓库
# http related config
#http:
  # port for http, default is 80. If https enabled, this port will redirect to https port
#  port: 80

#常规http访问模式

#   # https port for harbor, default is 443
   port: 443
#   # The path of cert and key files for nginx
   certificate: /etc/docker/certs/reg.mydocker.com.crt
   private_key: /etc/docker/certs/reg.mydocker.com.key

#https加密访问，与http只能开启一个模块，另https需要开启认证证书和密钥这两行,证书和密钥还没有生成，但是存放路径先填写好

# Uncomment external_url if you want to enable external proxy
# And when it enabled the hostname will no longer used
# external_url: https://reg.mydomain.com:8433

# The initial password of Harbor admin
# It only works in first time to install harbor
# Remember Change the admin password from UI after launching Harbor.
harbor_admin_password: redhat

# Harbor DB configuration
database:
  # The password for the root user of Harbor DB. Change this before any production use.
  password: redhat
#harbor使用的数据库容器管理员密码
# The default data volume
data_volume: /data
#默认数据存放卷

4. 因为仓库选择使用了https认证，需要生成相应密钥和证书

[root@repository certs]# pwd
/etc/docker/certs
[root@repository certs]# openssl req -newkey rsa:4096 -nodes -sha256 -keyout reg.mydocker.com.key -x509 -days 365 -out reg.mydocker.com.crt
……
省略了过程，7项信息填写（国家、省、城市、机构、部门、服务器名称、邮箱）
注意此项服务器名称：Common Name (eg, your name or your server's hostname) []:reg.mydocker.com
……
[root@repository certs]# ls
reg.mydocker.com.crt  reg.mydocker.com.key

5. 安装harbor，先运行下prepare将修改的配置文件准备好，再进行install.sh 自动安装

[root@repository harbor]# ./install.sh

[Step 0]: checking installation environment ...

Note: docker version: 18.09.6
✖ Need to install docker-compose(1.18.0+) by yourself first and run this script again.
#额，需要安装docker-compose，很明显因为使用harbor是多容器，yml文件安装，所以需要编排软件

6. 安装docker-compose

[root@repository opt]# ls
containerd  docker-compose-Linux-x86_64  harbor  harbor-offline-installer-v1.8.2.tgz
[root@repository opt]# mv docker-compose-Linux-x86_64 /usr/local/bin/docker-compose
[root@repository opt]# chmod +x /usr/local/bin/docker-compose

7. 再次进行harbor安装

[root@repository harbor]# ./install.sh

[Step 0]: checking installation environment ...

Note: docker version: 18.09.6

Note: docker-compose version: 1.21.2

[Step 1]: loading Harbor images ...
Loaded image: goharbor/prepare:v1.8.2
Loaded image: goharbor/registry-photon:v2.7.1-patch-2819-v1.8.2
Loaded image: goharbor/notary-signer-photon:v0.6.1-v1.8.2
Loaded image: goharbor/chartmuseum-photon:v0.9.0-v1.8.2
Loaded image: goharbor/harbor-log:v1.8.2
Loaded image: goharbor/harbor-jobservice:v1.8.2
Loaded image: goharbor/redis-photon:v1.8.2
Loaded image: goharbor/clair-photon:v2.0.8-v1.8.2
Loaded image: goharbor/harbor-portal:v1.8.2
Loaded image: goharbor/harbor-core:v1.8.2
Loaded image: goharbor/nginx-photon:v1.8.2
Loaded image: goharbor/notary-server-photon:v0.6.1-v1.8.2
Loaded image: goharbor/harbor-db:v1.8.2
Loaded image: goharbor/harbor-registryctl:v1.8.2
Loaded image: goharbor/harbor-migrator:v1.8.2


[Step 2]: preparing environment ...
prepare base dir is set to /opt/harbor
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /secret/keys/secretkey
Generated certificate, key file: /secret/core/private_key.pem, cert file: /secret/registry/root.crt
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir



[Step 3]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... done
Creating redis       ... done
Creating harbor-db   ... done
Creating registry    ... done
Creating registryctl ... done
Creating harbor-core ... done
Creating harbor-portal     ... done
Creating harbor-jobservice ... done
Creating nginx             ... done

✔ ----Harbor has been installed and started successfully.----

Now you should be able to visit the admin portal at https://reg.mydocker.com.
For more details, please visit https://github.com/goharbor/harbor .
#安装成功
[root@repository harbor]# docker-compose ps
      Name                     Command                       State                         Ports
--------------------------------------------------------------------------------------------------------------
harbor-core         /harbor/start.sh                 Up (health: starting)
harbor-db           /entrypoint.sh postgres          Up (health: starting)   5432/tcp
harbor-jobservice   /harbor/start.sh                 Up
harbor-log          /bin/sh -c /usr/local/bin/ ...   Up (health: starting)   127.0.0.1:1514->10514/tcp
harbor-portal       nginx -g daemon off;             Up (health: starting)   80/tcp
nginx               nginx -g daemon off;             Up (health: starting)   0.0.0.0:443->443/tcp,
                                                                             0.0.0.0:80->80/tcp
redis               docker-entrypoint.sh redis ...   Up                      6379/tcp
registry            /entrypoint.sh /etc/regist ...   Up (health: starting)   5000/tcp
registryctl         /harbor/start.sh                 Up (health: starting)
#查看服务是否启动正常

8. 浏览器访问
   
   
9. 添加一个私有仓库和公开仓库进行对比
   

私有仓库与公开仓库对比实践

仓库	节点（其他主机）	push镜像	pull镜像
私有仓库	如果还https需要添加ca证书	需要login认证	需要login认证
公开仓库	如果还https需要添加ca证书	需要login认证	无需login认证

1. 测试节点node1，添加证书
   因为仓库是https

[root@node1 reg.mydocker.com]# mkdir -p /etc/docker/certs.d/reg.mydocker.com
[root@node1 reg.mydocker.com]# scp repository:/etc/docker/certs/reg.mydocker.com.crt ca.crt
reg.mydocker.com.crt                                                        100% 2057   460.3KB/s   00:00

2. 没有login的情况下上传私有仓库和公开仓库

[root@node1 ~]# docker images
REPOSITORY                 TAG                 IMAGE ID            CREATED             SIZE
nginx                      latest              e548f1a579cf        24 months ago       109MB
dockersamples/visualizer   latest              17e55a9b2354        2 years ago         148MB
game2048                   latest              19299002fdbe        3 years ago         55.5MB
haproxy                    latest              fbd1f55f79b3        3 years ago         139MB
[root@node1 ~]# docker tag nginx:latest reg.mydocker.com/library/ngnix:latest
#修改公开上传标签
[root@node1 ~]# docker tag nginx:latest reg.mydocker.com/private/ngnix:latest
#修改私有上传标签
[root@node1 ~]# docker push reg.mydocker.com/library/ngnix
The push refers to repository [reg.mydocker.com/library/ngnix]
e89b70d28795: Preparing
832a3ae4ac84: Preparing
014cf8bfcb2d: Preparing
denied: requested access to the resource is denied
#共有仓库访问拒绝了
[root@node1 ~]# docker push reg.mydocker.com/private/ngnix
The push refers to repository [reg.mydocker.com/private/ngnix]
e89b70d28795: Preparing
832a3ae4ac84: Preparing
014cf8bfcb2d: Preparing
denied: requested access to the resource is denied
#私有仓库也拒绝了

3. node1节点上使用admin用户登录仓库

[root@node1 ~]# docker login reg.mydocker.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

#登录成功
[root@node1 ~]# cat .docker/config.json
{
	"auths": {
		"reg.mydocker.com": {
			"auth": "YWRtaW46YWRtaW4="
		}
	},
	"HttpHeaders": {
		"User-Agent": "Docker-Client/18.09.6 (linux)"
	}
}
#登录信息保存地址

4. 再次测试上传，login后都成功了

[root@node1 ~]#docker push reg.mydocker.com/library/ngnix
The push refers to repository [reg.mydocker.com/library/ngnix]
e89b70d28795: Pushed
832a3ae4ac84: Pushed
014cf8bfcb2d: Pushed
latest: digest: sha256:600bff7fb36d7992512f8c07abd50aac08db8f17c94e3c83e47d53435a1a6f7c size: 948
[root@node1 ~]# docker push reg.mydocker.com/private/ngnix
The push refers to repository [reg.mydocker.com/private/ngnix]
e89b70d28795: Mounted from library/ngnix
832a3ae4ac84: Mounted from library/ngnix
014cf8bfcb2d: Mounted from library/ngnix
latest: digest: sha256:600bff7fb36d7992512f8c07abd50aac08db8f17c94e3c83e47d53435a1a6f7c size: 948
[root@node1 ~]#
#都成功了

5. 测试下载，先测试登录下下载问题

[root@node1 ~]# docker rmi reg.mydocker.com/private/ngnix
Untagged: reg.mydocker.com/private/ngnix:latest
Untagged: reg.mydocker.com/private/ngnix@sha256:600bff7fb36d7992512f8c07abd50aac08db8f17c94e3c83e47d53435a1a6f7c
[root@node1 ~]# docker rmi reg.mydocker.com/library/ngnix
Untagged: reg.mydocker.com/library/ngnix:latest
Untagged: reg.mydocker.com/library/ngnix@sha256:600bff7fb36d7992512f8c07abd50aac08db8f17c94e3c83e47d53435a1a6f7c、
#先删除镜像

[root@node1 ~]# docker pull reg.mydocker.com/private/ngnix
Using default tag: latest
latest: Pulling from private/ngnix
Digest: sha256:600bff7fb36d7992512f8c07abd50aac08db8f17c94e3c83e47d53435a1a6f7c
Status: Downloaded newer image for reg.mydocker.com/private/ngnix:latest
[root@node1 ~]# docker pull reg.mydocker.com/library/ngnix
Using default tag: latest
latest: Pulling from library/ngnix
Digest: sha256:600bff7fb36d7992512f8c07abd50aac08db8f17c94e3c83e47d53435a1a6f7c
Status: Downloaded newer image for reg.mydocker.com/library/ngnix:latest
#下载都成功了

6. 删除镜像，退出登录，测试可否下载

[root@node1 ~]# docker rmi reg.mydocker.com/private/ngnix
Untagged: reg.mydocker.com/private/ngnix:latest
Untagged: reg.mydocker.com/private/ngnix@sha256:600bff7fb36d7992512f8c07abd50aac08db8f17c94e3c83e47d53435a1a6f7c
[root@node1 ~]# docker rmi reg.mydocker.com/library/ngnix
Untagged: reg.mydocker.com/library/ngnix:latest
Untagged: reg.mydocker.com/library/ngnix@sha256:600bff7fb36d7992512f8c07abd50aac08db8f17c94e3c83e47d53435a1a6f7c、
#删除镜像

[root@node1 ~]# docker logout reg.mydocker.com
Removing login credentials for reg.mydocker.com
#退出登录
[root@node1 ~]# docker pull reg.mydocker.com/library/ngnix
Using default tag: latest
latest: Pulling from library/ngnix
Digest: sha256:600bff7fb36d7992512f8c07abd50aac08db8f17c94e3c83e47d53435a1a6f7c
Status: Downloaded newer image for reg.mydocker.com/library/ngnix:latest
#共有仓库可以下载
[root@node1 ~]# docker pull reg.mydocker.com/private/ngnix
Using default tag: latest
Error response from daemon: pull access denied for reg.mydocker.com/private/ngnix, repository does not exist or may require 'docker login'
#私有仓库不可以