使用 harbor 搭建 docker 私有镜像仓库
harbor 是由
vmware 中国团队开发并开源的企业级 docker registry 服务，是对官方开源的 Docker Registry 的扩展，增加了一些企业需要的功能，如安全、复制和管理。harbor 主要用于搭建私有 registry，提供了企业需要的安全和控制功能。同时，它也帮助减少带宽使用量，这对提供生产能力和性能有帮助。
harbor 的强大之处在于，同时提供了 registry 的存储功能、认证功能、web ui 浏览和管理功能，且搭建相对简单。
本文搭建环境：
$ cat /etc/redhat-release
CentOS Linux release 7.1.1503 (Core)
$ uname -srvmpio
Linux 3.10.0-229.14.1.el7.x86_64 #1 SMP Tue Sep 15 15:05:51 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$ ip addr show | grep eth0 | grep inet
inet 172.20.30.35/24 brd 172.20.30.255 scope global dynamic eth0
搭建目标是：
能通过 http://172.20.30.35:10080 打开 web ui；
能通过
172.20.30.35:10080 push/pull docker 镜像。
而 harbor 的默认使用的端口是 80，我们要改变这个端口到 10080，因为目标主机的 80 端口在外部无法访问。
Harbor 的安装方法有两种：
- 从源码安装 -- 这通过一个完全的构建过程，且需要你的主机能连接外网
- 预构建安装包 -- 这比较节省时间（不需要构建），同时允许在一台没有联网的主机上搭建
这篇文章同时介绍这两种安装方式。
搭建必备条件
harbor 是以若干个 docker 容器化应用部署的，所以可以安装在任何支持 docker 的 linux 机器上。目标主机需要安装 python 2.7或以上, docker 1.10或以上，docker-compose 1.6.0 或以上。
从源码安装
1、获取源码
$ git clone https://github.com/vmware/harbor
然后
$ cd harbor/Deploy
注意：除非明确说明，否则后续所有操作都在这个目录中进行。
2、配置
主要配置下面几个文件（改动的行用 ** 标记）：
2.1
$ vim templates/ui/app.conf
appname = registry
runmode = dev
[lang]
types = en-US|zh-CN
names = en-US|zh-CN
[dev]
**httpport = 10080**
[mail]
host = $email_server
port = $email_server_port
username = $email_username
password = $email_password
from = $email_from
ssl = $email_ssl
2.2
$ vim templates/registry/config.yml
version: 0.1
log:
level: debug
fields:
service: registry
storage:
cache:
layerinfo: inmemory
filesystem:
**rootdirectory: /var/lib/registry**
maintenance:
uploadpurging:
enabled: false
delete:
enabled: true
http:
addr: :5000
secret: placeholder
debug:
addr: localhost:5001
auth:
token:
issuer: registry-token-issuer
**realm: $ui_url:10080/service/token**
rootcertbundle: /etc/registry/root.crt
service: token-service
notifications:
endpoints:
- name: harbor
disabled: false
url: http://ui/service/notifications
timeout: 500ms
threshold: 5
backoff: 1s
2.3
$ vim harbor.cfg
## Configuration file of Harbor
#The IP address or hostname to access admin UI and registry service.
#DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
**hostname = 172.20.30.35**
#The protocol for accessing the UI and token/notification service, by default it is http.
#It can be set to https if ssl is enabled on nginx.
ui_url_protocol = http
#Email account settings for sending out password resetting emails.
email_server = smtp.mydomain.com
email_server_port = 25
email_username = [email protected]
email_password = abc
email_from = admin
email_ssl = false
##The password of Harbor admin, change this before any production use.
**harbor_admin_password = 1**
##By default the auth mode is db_auth, i.e. the credentials are stored in a local database.
#Set it to ldap_auth if you want to verify a user's credentials against an LDAP server.
auth_mode = db_auth
#The url for an ldap endpoint.
ldap_url = ldaps://ldap.mydomain.com
#The basedn template to look up a user in LDAP and verify the user's password.
#For AD server, uses this template:
#ldap_basedn = CN=%s,OU=Dept1,DC=mydomain,DC=com
ldap_basedn = uid=%s,ou=people,dc=mydomain,dc=com
#The password for the root user of mysql db, change this before any production use.
db_password = root123
#Turn on or off the self-registration feature
**self_registration = off**
#Determine whether the UI should use compressed js files.
#For production, set it to on. For development, set it to off.
use_compressed_js = on
#Maximum number of job workers in job service
max_job_workers = 3
#Determine whether the job service should verify the ssl cert when it connects to a remote registry.
#Set this flag to off when the remote registry uses a self-signed or untrusted certificate.
**verify_remote_cert = off**
#Determine whether or not to generate certificate for the registry's token.
#If the value is on, the prepare script creates new root cert and private key
#for generating token to access the registry. If the value is off, a key/certificate must
#be supplied for token generation.
**customize_crt = off**
#Information of your organization for certificate
crt_country = CN
crt_state = State
crt_location = CN
crt_organization = organization
crt_organizationalunit = organizational unit
crt_commonname = example.com
crt_email = [email protected]
#####
2.4
$ vim docker-compose.yml
version: '2'
services:
log:
build: ./log/
restart: always
volumes:
- /var/log/harbor/:/var/log/docker/
ports:
- 1514:514
registry:
image: library/registry:2.4.0
restart: always
volumes:
**- /var/lib/registry:/storage**
- ./config/registry/:/etc/registry/
environment:
- GODEBUG=netdns=cgo
ports:
- 5001:5001
command:
["serve", "/etc/registry/config.yml"]
depends_on:
- log
logging:
driver: "syslog"
options:
syslog-address: "tcp://127.0.0.1:1514"
tag: "registry"
mysql:
build: ./db/
restart: always
volumes:
- /data/database:/var/lib/mysql
env_file:
- ./config/db/env
depends_on:
- log
logging:
driver: "syslog"
options:
syslog-address: "tcp://127.0.0.1:1514"
tag: "mysql"
ui:
build:
context: ../
dockerfile: Dockerfile.ui
env_file:
- ./config/ui/env
restart: always
volumes:
- ./config/ui/app.conf:/etc/ui/app.conf
- ./config/ui/private_key.pem:/etc/ui/private_key.pem
depends_on:
- log
logging:
driver: "syslog"
options:
syslog-address: "tcp://127.0.0.1:1514"
tag: "ui"
jobservice:
build:
context: ../
dockerfile: Dockerfile.job
env_file:
- ./config/jobservice/env
restart: always
volumes:
- /data/job_logs:/var/log/jobs
- ./config/jobservice/app.conf:/etc/jobservice/app.conf
depends_on:
- ui
logging:
driver: "syslog"
options:
syslog-address: "tcp://127.0.0.1:1514"
tag: "jobservice"
proxy:
image: library/nginx:1.9
restart: always
volumes:
- ./config/nginx:/etc/nginx
ports:
**- 10080:10080**
- 443:443
depends_on:
- mysql
- registry
- ui
- log
logging:
driver: "syslog"
options:
syslog-address: "tcp://127.0.0.1:1514"
tag: "proxy"
2.5
$ vim config/jobservice/app.conf
appname = jobservice
runmode = dev
[dev]
**httpport = 10080**
2.6
$ vim config/nginx/nginx.conf
worker_processes auto;
events {
worker_connections 1024;
use epoll;
multi_accept on;
}
http {
tcp_nodelay on;
# this is necessary for us to be able to disable request buffering in all cases
proxy_http_version 1.1;
upstream registry {
server registry:5000;
}
upstream ui {
**server ui:10080;**
}
server {
**listen 10080;**
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
location / {
proxy_pass http://ui/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /v1/ {
return 404;
}
location /v2/ {
proxy_pass http://registry/v2/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
location /service/ {
proxy_pass http://ui/service/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
}
}
2.7
$ vim config/ui/app.conf
appname = registry
runmode = dev
[lang]
types = en-US|zh-CN
names = en-US|zh-CN
[dev]
**httpport = 10080**
[mail]
host = smtp.mydomain.com
port = 25
username = [email protected]
password = abc
from = admin
ssl = false
2.8
$ vim /etc/sysconfig/docker
# /etc/sysconfig/docker
# Modify these options if you want to change the way the docker daemon runs
OPTIONS='--selinux-enabled --log-driver=journald'
DOCKER_CERT_PATH=/etc/docker
**DOCKER_OPTS="--insecure-registry 172.20.30.35:10080"**
# If you want to add your own registry to be used for docker search and docker
# pull use the ADD_REGISTRY option to list a set of registries, each prepended
# with --add-registry flag. The first registry added will be the first registry
# searched.
#ADD_REGISTRY='--add-registry registry.access.redhat.com'
# If you want to block registries from being used, uncomment the BLOCK_REGISTRY
# option and give it a set of registries, each prepended with --block-registry
# flag. For example adding docker.io will stop users from downloading images
# from docker.io
# BLOCK_REGISTRY='--block-registry'
# If you have a registry secured with https but do not have proper certs
# distributed, you can tell docker to not look for full authorization by
# adding the registry to the INSECURE_REGISTRY line and uncommenting it.
INSECURE_REGISTRY='--insecure-registry 172.20.30.35:10080'
# On an SELinux system, if you remove the --selinux-enabled option, you
# also need to turn on the docker_transition_unconfined boolean.
# setsebool -P docker_transition_unconfined 1
# Location used for temporary files, such as those created by
# docker load and build operations. Default is /var/lib/docker/tmp
# Can be overriden by setting the following environment variable.
# DOCKER_TMPDIR=/var/tmp
# Controls the /etc/cron.daily/docker-logrotate cron job status.
# To disable, uncomment the line below.
# LOGROTATE=false
#
# docker-latest daemon can be used by starting the docker-latest unitfile.
# To use docker-latest client, uncomment below line
#DOCKERBINARY=/usr/bin/docker-latest
2.9 重启 docker-engine
$ service docker restart
或
$ systemctl restart docker.service
3、构建并启动 harbor
$ ./prepare
....
The configuration files are ready, please use docker-compose to start the service.
$ docker-compose up
Starting deploy_log_1
Starting deploy_ui_1
Starting deploy_mysql_1
Starting deploy_registry_1
Starting deploy_jobservice_1
Starting deploy_proxy_1
Attaching to deploy_log_1, deploy_ui_1, deploy_registry_1, deploy_mysql_1, deploy_jobservice_1, deploy_proxy_1
ui_1 | 2016-07-26T13:12:15Z [INFO] Config path: /etc/ui/app.conf
ui_1 | 2016-07-26T13:12:15Z [DEBUG] [base.go:54]: db url: mysql:3306, db user: root
ui_1 | 2016-07-26T13:12:16Z [ERROR] [base.go:66]: failed to connect to db, retry after 2 seconds :dial tcp 172.18.0.5:3306: getsockopt: connection refused
registry_1 | time="2016-07-26T13:12:15.557929687Z" level=info msg="configuring endpoint harbor (http://ui/service/notifications), timeout=500ms, headers=map[]" go.version=go1.6.1 instance.id=d69175ba-c563-4a9a-bbcb-fc90271935e8 service=registry version=v2.4.0
registry_1 | time="2016-07-26T13:12:15.563872153Z" level=info msg="redis not configured" go.version=go1.6.1 instance.id=d69175ba-c563-4a9a-bbcb-fc90271935e8 service=registry version=v2.4.0
registry_1 | time="2016-07-26T13:12:15.572819857Z" level=info msg="debug server listening localhost:5001"
registry_1 | time="2016-07-26T13:12:15.611411032Z" level=info msg="using inmemory blob descriptor cache" go.version=go1.6.1 instance.id=d69175ba-c563-4a9a-bbcb-fc90271935e8 service=registry version=v2.4.0
registry_1 | time="2016-07-26T13:12:15.611763465Z" level=debug msg="configured \"token\" access controller" go.version=go1.6.1 instance.id=d69175ba-c563-4a9a-bbcb-fc90271935e8 service=registry version=v2.4.0
registry_1 | time="2016-07-26T13:12:15.611835323Z" level=info msg="listening on [::]:5000" go.version=go1.6.1 instance.id=d69175ba-c563-4a9a-bbcb-fc90271935e8 service=registry version=v2.4.0
mysql_1 | 2016-07-26 13:12:16 0 [Note] mysqld (mysqld 5.6.31) starting as process 1 ...
mysql_1 | 2016-07-26 13:12:16 1 [Note] Plugin 'FEDERATED' is disabled.
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: Using atomics to ref count buffer pool pages
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: The InnoDB memory heap is disabled
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: Memory barrier is not used
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: Compressed tables use zlib 1.2.8
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: Using Linux native AIO
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: Using CPU crc32 instructions
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: Initializing buffer pool, size = 128.0M
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: Completed initialization of buffer pool
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: Highest supported file format is Barracuda.
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: 128 rollback segment(s) are active.
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: Waiting for purge to start
mysql_1 | 2016-07-26 13:12:16 1 [Note] InnoDB: 5.6.31 started; log sequence number 1736276
mysql_1 | 2016-07-26 13:12:16 1 [Note] Server hostname (bind-address): '*'; port: 3306
mysql_1 | 2016-07-26 13:12:16 1 [Note] IPv6 is available.
mysql_1 | 2016-07-26 13:12:16 1 [Note] - '::' resolves to '::';
mysql_1 | 2016-07-26 13:12:16 1 [Note] Server socket created on IP: '::'.
mysql_1 | 2016-07-26 13:12:16 1 [Warning] 'proxies_priv' entry '@ root@22b6d68df32a' ignored in --skip-name-resolve mode.
mysql_1 | 2016-07-26 13:12:17 1 [Note] Event Scheduler: Loaded 0 events
mysql_1 | 2016-07-26 13:12:17 1 [Note] mysqld: ready for connections.
mysql_1 | Version: '5.6.31' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
jobservice_1 | 2016-07-26T13:12:16Z [INFO] Config path: /etc/jobservice/app.conf
jobservice_1 | 2016-07-26T13:12:16Z [DEBUG] [config.go:89]: config: maxJobWorkers: 3
jobservice_1 | 2016-07-26T13:12:16Z [DEBUG] [config.go:90]: config: localUIURL: http://ui
jobservice_1 | 2016-07-26T13:12:16Z [DEBUG] [config.go:91]: config: localRegURL: http://registry:5000
jobservice_1 | 2016-07-26T13:12:16Z [DEBUG] [config.go:92]: config: verifyRemoteCert: off
jobservice_1 | 2016-07-26T13:12:16Z [DEBUG] [config.go:93]: config: logDir: /var/log/jobs
jobservice_1 | 2016-07-26T13:12:16Z [DEBUG] [config.go:94]: config: uiSecret: ******
jobservice_1 | 2016-07-26T13:12:16Z [DEBUG] [base.go:54]: db url: mysql:3306, db user: root
jobservice_1 | 2016-07-26T13:12:16Z [ERROR] [base.go:66]: failed to connect to db, retry after 2 seconds :dial tcp 172.18.0.5:3306: getsockopt: connection refused
ui_1 | 2016-07-26T13:12:18Z [INFO] User id: 1 already has its encrypted password.
ui_1 | 2016/07/26 13:12:18 [asm_amd64.s:1998][I] http server Running on :10080
jobservice_1 | 2016-07-26T13:12:18Z [DEBUG] [workerpool.go:123]: worker 0 started
jobservice_1 | 2016-07-26T13:12:18Z [DEBUG] [workerpool.go:123]: worker 1 started
jobservice_1 | 2016-07-26T13:12:18Z [DEBUG] [workerpool.go:123]: worker 2 started
jobservice_1 | 2016-07-26T13:12:18Z [DEBUG] [main.go:36]: Trying to resume halted jobs...
jobservice_1 | 2016/07/26 13:12:18 [asm_amd64.s:1998][I] http server Running on :10080
输出类似上述信息，就说明启动成功。且 http://172.20.30.35:10080 能够访问。
如果有错误，可以从输出日志中看看是哪出错了。
查看一下其它相关信息
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7f893cfcead0 library/nginx:1.9 "nginx -g 'daemon off" About an hour ago Up About a minute 0.0.0.0:443->443/tcp, 80/tcp, 0.0.0.0:10080->10080/tcp deploy_proxy_1
e70967b2b72f deploy_jobservice "/go/bin/harbor_jobse" About an hour ago Up About a minute deploy_jobservice_1
975e74c43e47 deploy_mysql "docker-entrypoint.sh" About an hour ago Up About a minute 3306/tcp deploy_mysql_1
e4b364187a43 library/registry:2.4.0 "/bin/registry serve " About an hour ago Up About a minute 5000/tcp, 0.0.0.0:5001->5001/tcp deploy_registry_1
e91872374e25 deploy_ui "/go/bin/harbor_ui" About an hour ago Up About a minute 80/tcp deploy_ui_1
04d2ddf3cd63 deploy_log "/bin/sh -c 'cron && " About an hour ago Up About a minute 0.0.0.0:1514->514/tcp deploy_log_1$ netstat -nlpt | grep docker-proxy
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp6 0 0 :::5001 :::* LISTEN 13272/docker-proxy
tcp6 0 0 :::1514 :::* LISTEN 13177/docker-proxy
tcp6 0 0 :::443 :::* LISTEN 13442/docker-proxy
tcp6 0 0 :::10080 :::* LISTEN 13435/docker-proxy
到这里，源码搭建就算完成了。
预构建安装
预构建安装 跟 源码 安装几乎一样。
预构建安装 支持
无外网连接安装，其实就是先在能连外网的机器上安装一遍，再用 docker save 将镜像导出来，再把导出来的镜像用 scp 或其它工具传给目标主机，再在目标主机上用 docker load 导进去。之后步骤就是上面讲的改配置等等。
使用
1、在你的机器上修改文件 /etc/sysconfig/docker ，添加（或修改）字段：
2、重启 docker-engine
OPTIONS='--selinux-enabled --log-driver=journald'
DOCKER_CERT_PATH=/etc/docker
ADD_REGISTRY='--add-registry 172.20.30.35:10080'
DOCKER_OPTS="--insecure-registry 172.20.30.35:10080"
INSECURE_REGISTRY='--insecure-registry 172.20.30.35:10080'2、重启 docker-engine
$ service docker restart
或
$ systemctl restart docker.service3、登录
$ docker login 172.20.30.35:100804、push 镜像到仓库
4.1, 先tag
$ docker tag your-docker-image 172.20.30.35:10080/your-repository-name/your-docker-image4.2, push 到 172.20.30.35:10080 上去
$ docker push 172.20.30.35:10080/your-repository-name/your-docker-image5、从仓库中 pull 镜像
$ docker pull your-repository-name/your-docker-image$ docker pull 172.20.30.35:10080/your-repository-name/your-docker-image
参考： http://1.chaoxu.sinaapp.com/archives/3969