docker镜像仓库harbor之清理镜像

一、背景

相比于nexus,harbor的一大优点是方便及时清理无用的docker镜像。本文就harbor怎么设置清理,梳理一下具体的操作办法。

  • harbor 版本是 v2.9.0

二、目标

随着我们推送至仓库的镜像越来越多,带来的一个最大运维问题就是存储空间的浪费,需要我们能够把旧的镜像清理出去。
这就要求我们按推送的时间倒序排列,保留最新推送的多少个,从而删除较早推送的记录。

三、设置

docker镜像仓库harbor之清理镜像_第1张图片

  • 已使用存储空间为2.54GB
  • library/vue-cli镜像已推送的总数为4个
  • library/node镜像已推送的总数为3个

所以,我们测试规则中的保留数不宜过多,暂定为3个。
预期效果是:library/vue-cli的最早的一个镜像将被清理掉,而 library/node 因为没有超过阈值,所以不会变化。

docker镜像仓库harbor之清理镜像_第2张图片

1、策略

docker镜像仓库harbor之清理镜像_第3张图片
docker镜像仓库harbor之清理镜像_第4张图片
策略支持多个规则,不过我们不需要那么多。我这里选择的是第一个默认规则,保留最近推送的3个。
然后执行的频率是每周,这个频率看每个人的使用场景。
docker镜像仓库harbor之清理镜像_第5张图片

2、模拟运行

docker镜像仓库harbor之清理镜像_第6张图片
查看清理的明细日志:

2023-10-10T01:02:19Z [INFO] [/pkg/retention/job.go:86]: Run retention process.
Repository: library/vue-cli
Rule Algorithm: or
Dry Run: true
2023-10-10T01:02:21Z [INFO] [/pkg/retention/job.go:101]: Load 4 candidates from repository library/vue-cli
2023-10-10T01:02:21Z [INFO] [/pkg/retention/job.go:212]:
| Digest | Tag | Kind | Labels | PushedTime | PulledTime | CreatedTime | Retention |
|-------------------------------------------------------------------------|-------------------------|-------|--------|---------------------|---------------------|---------------------|-----------|
| sha256:9ebe43b637dbda81e0513ef21b950a4fcf3319482fe6f25e16653b41406370ef | 3.3.0_build202309150905 | image | | 2023/10/09 01:21:56 | 2023/10/09 09:40:53 | 2023/10/09 01:21:56 | RETAIN |
| sha256:dce192888562d266e6959776f1fd772e89f1f654044fbd445e4475d5ba4d7f7e | 3.3.0_build202303140655 | image | | 2023/10/09 01:20:50 | | 2023/10/09 01:20:49 | RETAIN |
| sha256:165463708f354f8d96ba083b7fbbb1f25b9c8533cbc01117e7648c195733cc8d | 3.3.0_build202109241209 | image | | 2023/10/09 01:20:11 | | 2023/10/09 01:20:11 | RETAIN |
| sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4 | 3.3.0_build202108200549 | image | | 2023/10/09 01:20:01 | | 2023/10/09 01:20:01 | DEL |

日志是按创建时间排序排列,可以看出,标签为3.3.0_build202108200549的镜像将被移除,它的推送时间是2023/10/09 01:20:01,为最早推送的一个,因此符合我们对清理的预期。

  • 作为对比,另外一个镜像的清理日志就不会出现DEL的操作。(因为library/node总共只有3个,而清理的规则也恰好是3个,意味着没有超过清理的阈值)

2023-10-10T00:59:41Z [INFO] [/pkg/retention/job.go:86]: Run retention process.
Repository: library/node
Rule Algorithm: or
Dry Run: true
2023-10-10T00:59:43Z [INFO] [/pkg/retention/job.go:101]: Load 3 candidates from repository library/node
2023-10-10T00:59:43Z [INFO] [/pkg/retention/job.go:212]:
| Digest | Tag | Kind | Labels | PushedTime | PulledTime | CreatedTime | Retention |
|-------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|-------|--------|---------------------|---------------------|---------------------|-----------|
| sha256:1b7e5ae382ed76a8e39a4afaa968f4b68f00501db7e8b91e5870117b881207c9 | 3.3.0_build202310090525 | image | | 2023/10/09 05:30:15 | 2023/10/10 00:41:02 | 2023/10/09 05:30:14 | RETAIN |
| sha256:9ebe43b637dbda81e0513ef21b950a4fcf3319482fe6f25e16653b41406370ef | 3.3.0_build202310090127,3.3.0_build202309210924,3.3.0_build202309150958,3.3.0_build202309150956 | image | | 2023/10/09 01:27:10 | 2023/10/09 01:30:42 | 2023/10/09 01:25:00 | RETAIN |
| sha256:59167f718a1f6daecb900ffbb4f5b09f0dca709c5d7e180f33bbaeac076cdb7b | 16.20.2-bullseye | image | | 2023/10/09 01:24:56 | | 2023/10/09 01:24:56 | RETAIN |

3、立即执行

docker镜像仓库harbor之清理镜像_第7张图片
在你使用模拟运行确认无误后,一定会要点击“立即运行”。
但是运行完成后,持续时间是4秒,可harbor的存储空间并无变化。(仍旧是2.54GB)

四、清理服务

docker镜像仓库harbor之清理镜像_第8张图片
点击“立即清理垃圾”,看到下面的任务记录。
docker镜像仓库harbor之清理镜像_第9张图片

2023-10-10T01:42:45Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:599]: ID-3 MediaType-application/vnd.docker.container.image.v1+json ManifestMediaType-application/vnd.docker.distribution.manifest.v2+json RepositoryName-library/vue-cli Digest-sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4 CreationTime-2023-10-10 01:38:52
2023-10-10T01:42:45Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:341]: [05491b08-f601-4ee7-a5bf-c32a76cd96ca][3/11] delete the manifest with registry v2 API: library/vue-cli, application/vnd.docker.distribution.manifest.v2+json, sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4
2023-10-10T01:42:45Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:370]: [05491b08-f601-4ee7-a5bf-c32a76cd96ca][3/11] delete manifest from storage: sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4
2023-10-10T01:42:45Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:398]: [05491b08-f601-4ee7-a5bf-c32a76cd96ca][3/11] delete artifact blob record from database: 3, library/vue-cli, sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4
2023-10-10T01:42:45Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:406]: [05491b08-f601-4ee7-a5bf-c32a76cd96ca][3/11] delete artifact trash record from database: 3, library/vue-cli, sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4

docker镜像仓库harbor之清理镜像_第10张图片

五、查看清理后

观察镜像library/vue-cli 的数量,可以看出,最早的标签已被清理。
docker镜像仓库harbor之清理镜像_第11张图片

六、保留最近多少天被拉取过的artifact

由于xxx/api-platform镜像只推送过一次镜像,为了演示“立即运行”的清理,我使用另外一个规则,“保留最近多少天被拉取过的artifact”。(因为xxx/api-platform镜像好久没被拉取过,所以预期将被清理)
docker镜像仓库harbor之清理镜像_第12张图片
从上图也可以看出,它将保留0个,也即全部清理。

2023-10-10T01:57:03Z [INFO] [/pkg/retention/job.go:86]: Run retention process.
Repository: xxx/api-platform
Rule Algorithm: or
Dry Run: false
2023-10-10T01:57:05Z [INFO] [/pkg/retention/job.go:101]: Load 1 candidates from repository xxx/api-platform
2023-10-10T01:57:05Z [INFO] [/pkg/retention/job.go:212]:
| Digest | Tag | Kind | Labels | PushedTime | PulledTime | CreatedTime | Retention |
|-------------------------------------------------------------------------|-------|-------|--------|---------------------|------------|---------------------|-----------|
| sha256:f7f0721d6ce20278942c97b59c361a0256fc8330a1c21fe0044bc85e6ce91d58 | 1.0.2 | image | | 2023/10/09 01:26:25 | | 2023/10/09 01:26:25 | DEL |

docker镜像仓库harbor之清理镜像_第13张图片

七、总结

这里通过清理的对比实验,可以看出“策略”–》“立即运行”,也可以清理掉无用的构建物;那么外面单独的菜单–“清理服务”是起何作用?

清理服务,从日志内容可以看到,它清理的是垃圾:“xx个 blob(s) 和 xx个 manifest(s) 将被删除”,另外它还可以清理harbor本身的日志记录。

2023-10-10T02:13:03Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:162]: Garbage Collection parameters: [delete_untagged: false, dry_run: true, time_window: 2, workers: 1]
2023-10-10T02:13:03Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:172]: start to run gc in job.
2023-10-10T02:13:03Z [WARNING] [/jobservice/job/impl/gc/garbage_collection.go:213]: no removed artifacts.
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:d5db42f314af162c5dc93a6c89a5bfeda6307f9acf3c2290d41c32668855016d
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:076545cf68e620c922789f9321e2de7f8eb88b850906d53194eeb5c77f1ff23d
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:330d755f124c8b8e56a11ad2b053fb4eee18a0399b6ec8acc777bb712ba12b30
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:eac1b95df832dc9f172fd1f07e7cb50c1929b118a4249ddd02c6318a677b506a
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:47aa3ed2034c4f27622b989b26c06087de17067268a19a1b3642a7e2686cd1a3
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:0a34e63c37606f57dbba92a722b19bfb743d619a48ea93de80548ca5198ad22d
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:e29a0832e742d9ab0040e9bab8f502b7890d9d17bc6a140432deee5659710e27
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:f7f0721d6ce20278942c97b59c361a0256fc8330a1c21fe0044bc85e6ce91d58
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:5bd4f271ae1c8e7db16cdd6948b5c2736852247ff16414102dd568e5f11f94b5
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:17497ec24c6e61e3f16afadd5e376e9bf737edb2175e62864d26806bf328a496
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:273]: 9 blobs and 1 manifests eligible for deletion
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:274]: The GC could free up 373 MB space, the size is a rough estimation.
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:200]: success to run gc in job.

docker镜像仓库harbor之清理镜像_第14张图片

使用建议

先对各个项目的策略进行配置,配合使用清理服务。

你可能感兴趣的:(docker,容器)