GitLab基础：新版本下使用gitlab-rake进行备份和恢复

从GitLab12.1开始，可以直接使用rake进行备份，如果之前的项目中使用的gitlab-rake进行备份和恢复的方式是否可用这个观点进行了如下实验，通过本文进行简单记录。

备份源环境准备

容器化方式，直接使用docker-compose，启动如下服务

[root@host131 gitlab]# cat docker-compose.yml 
version: '2'    
services:
  # Version Control service: Gitlab
  gitlab:
    image: gitlab/gitlab-ce:12.10.5-ce.0
    ports:
      - "32001:80"
      - "30022:22"
      - "30443:443"
    volumes:
      - ./log/:/var/log/gitlab
      - ./data/:/var/opt/gitlab
      - ./conf/:/etc/gitlab
    restart: "no"
[root@host131 gitlab]# 

修改root用户，并创建如下项目

同步数据

使用rsync将各数据卷拷贝至备份目标机器，首先同步数据目录

[root@host131 ~]# rsync -r gitlab/data 192.168.163.132:/root/gitlab/data
[email protected]'s password: 
skipping non-regular file "data/gitaly/gitaly.socket"
skipping non-regular file "data/gitaly/internal_sockets/internal.sock"
skipping non-regular file "data/gitaly/internal_sockets/ruby.0"
skipping non-regular file "data/gitaly/internal_sockets/ruby.1"
skipping non-regular file "data/gitlab-rails/sockets/gitlab.socket"
skipping non-regular file "data/gitlab-workhorse/socket"
skipping non-regular file "data/grafana/conf"
skipping non-regular file "data/grafana/public"
skipping non-regular file "data/nginx/logs"
skipping non-regular file "data/postgresql/.s.PGSQL.5432"
skipping non-regular file "data/redis/redis.socket"
[root@host131 ~]#

注意事前需要在host132备份目标机器创建对应根目录

[root@host132 ~]# pwd
/root
[root@host132 ~]# ls
anaconda-ks.cfg
[root@host132 ~]# mkdir gitlab
[root@host132 ~]# ls gitlab/
[root@host132 ~]#

不然同步数据的时候会出现类似如下错误提示信息

[root@host131 ~]# rsync -r gitlab/data 192.168.163.132:/root/gitlab/data
[email protected]'s password: 
rsync: mkdir "/root/gitlab/data" failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(657) [Receiver=3.1.2]
[root@host131 ~]# 

• 同步配置数据

[root@host131 ~]# rsync -r gitlab/conf 192.168.163.132:/root/gitlab/conf
[email protected]'s password: 
[root@host131 ~]# 

• 同步日志数据

[root@host131 ~]# rsync -r gitlab/log 192.168.163.132:/root/gitlab/log
[email protected]'s password: 
skipping non-regular file "log/gitlab-rails/sidekiq.log"
[root@host131 ~]# 

PS： 其实也不用这么费劲，直接打个打的tar扔过去就可以了

启动备份目标侧GitLab服务

使用docker-compose启动之后，界面显示如下，看起来和没有数据的方式一样

使用gitlab-rake备份数据

[root@host131 ~]# docker exec -it gitlab_gitlab_1 sh
# gitlab-rake gitlab:backup:create
2020-08-11 23:26:51 +0000 -- Dumping database ... 
Dumping PostgreSQL database gitlabhq_production ... [DONE]
2020-08-11 23:26:56 +0000 -- done
2020-08-11 23:26:56 +0000 -- Dumping repositories ...
 * root/testproject (@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b) ... [SKIPPED]
[SKIPPED] Wiki
2020-08-11 23:26:57 +0000 -- done
2020-08-11 23:26:57 +0000 -- Dumping uploads ... 
2020-08-11 23:26:57 +0000 -- done
2020-08-11 23:26:57 +0000 -- Dumping builds ... 
2020-08-11 23:26:57 +0000 -- done
2020-08-11 23:26:57 +0000 -- Dumping artifacts ... 
2020-08-11 23:26:57 +0000 -- done
2020-08-11 23:26:57 +0000 -- Dumping pages ... 
2020-08-11 23:26:57 +0000 -- done
2020-08-11 23:26:57 +0000 -- Dumping lfs objects ... 
2020-08-11 23:26:57 +0000 -- done
2020-08-11 23:26:57 +0000 -- Dumping container registry images ... 
2020-08-11 23:26:57 +0000 -- [DISABLED]
Creating backup archive: 1597188417_2020_08_11_12.10.5_gitlab_backup.tar ... done
Uploading backup archive to remote storage  ... skipped
Deleting tmp directories ... done
done
done
done
done
done
done
done
Deleting old backups ... skipping
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data 
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
Backup task is done.
# 
# cd /var/opt/gitlab/backups
# ls
1597188417_2020_08_11_12.10.5_gitlab_backup.tar
# du -k *.tar
172	1597188417_2020_08_11_12.10.5_gitlab_backup.tar
# 

可以看到数据较少，只有170K，而且提示了一个Warning信息，说一些包含敏感数据的文件gitlab.rb和gitlab-secrets.json文件不再备份之中，需要手动备份和恢复。

执行恢复

使用restore执行恢复，提示如下错误信息：

[root@host132 backups]# docker exec -it gitlab_gitlab_1 sh
# cd /var/opt/gitlab/backups
# ls
1597188417_2020_08_11_12.10.5_gitlab_backup.tar
# gitlab-rake gitlab:backup:restore BACKUP=1597188417
The backup file 1597188417_gitlab_backup.tar does not exist!
#

这是因为后续tar文件名的格式发生了变化，里面加入了GitLab的版本号，所以之前的命令早不到用于恢复的备份文件，修改文件名如下：

# cp 1597188417_2020_08_11_12.10.5_gitlab_backup.tar 1597188417_gitlab_backup.tar
# 

PS：更为简单的方式是使用如下命令

执行命令：gitlab-rake gitlab:backup:restore BACKUP=1597188417_2020_08_11_12.10.5

在恢复之前需要首先停止服务

# gitlab-ctl stop
ok: down: alertmanager: 0s, normally up
ok: down: gitaly: 0s, normally up
ok: down: gitlab-exporter: 1s, normally up
ok: down: gitlab-workhorse: 0s, normally up
ok: down: grafana: 1s, normally up
ok: down: logrotate: 0s, normally up
ok: down: nginx: 1s, normally up
ok: down: postgres-exporter: 0s, normally up
ok: down: postgresql: 0s, normally up
ok: down: prometheus: 1s, normally up
ok: down: redis: 0s, normally up
ok: down: redis-exporter: 1s, normally up
ok: down: sidekiq: 0s, normally up
ok: down: sshd: 0s, normally up
ok: down: unicorn: 0s, normally up
# 

导入的时候需要注意权限是否正确否则容易出现如下错误

# gitlab-rake gitlab:backup:restore BACKUP=1597188417
Unpacking backup ... tar: 1597188417_gitlab_backup.tar: Cannot open: Permission denied
tar: Error is not recoverable: exiting now
unpacking backup failed
#

这是因为rsync同步时的文件权限受umask影响的缺省设定，修改即可

# ls -lrt
total 344
-rw------- 1 root root 174080 Aug 12 01:23 1597188417_2020_08_11_12.10.5_gitlab_backup.tar
-rw------- 1 root root 174080 Aug 12 01:26 1597188417_gitlab_backup.tar
# chmod 644 *
#

执行备份恢复出现如下错误：

# gitlab-rake gitlab:backup:restore BACKUP=1597188417
Unpacking backup ... done
Before restoring the database, we will remove all existing
tables to avoid future upgrade problems. Be aware that if you have
custom tables in the GitLab database these tables and all data will be
removed.

Do you want to continue (yes/no)? yes
Removing all tables. Press `Ctrl-C` within 5 seconds to abort
2020-08-12 01:28:07 +0000 -- Cleaning the database ... 
rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:27:in `block (3 levels) in '
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:64:in `block (3 levels) in '
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `
'
Tasks: TOP => gitlab:db:drop_tables
(See full trace by running task with --trace)
# 

提示信息显示无法连接数据库，理论上来说把PostgreSQL启动起来可以继续，这里干脆全部启动

# gitlab-ctl start
ok: run: alertmanager: (pid 3009) 1s
ok: run: gitaly: (pid 3017) 0s
ok: run: gitlab-exporter: (pid 3030) 1s
ok: run: gitlab-workhorse: (pid 3033) 0s
ok: run: grafana: (pid 3042) 1s
ok: run: logrotate: (pid 3050) 0s
ok: run: nginx: (pid 3056) 0s
ok: run: postgres-exporter: (pid 3062) 1s
ok: run: postgresql: (pid 3078) 0s
ok: run: prometheus: (pid 3088) 1s
ok: run: redis: (pid 3096) 0s
ok: run: redis-exporter: (pid 3101) 1s
ok: run: sidekiq: (pid 3106) 0s
ok: run: sshd: (pid 3112) 1s
ok: run: unicorn: (pid 3114) 0s
# 
# gitlab-rake gitlab:backup:restore BACKUP=1597188417
Unpacking backup ... done
Before restoring the database, we will remove all existing
tables to avoid future upgrade problems. Be aware that if you have
custom tables in the GitLab database these tables and all data will be
removed.

Do you want to continue (yes/no)? yes
Removing all tables. Press `Ctrl-C` within 5 seconds to abort
2020-08-12 02:27:25 +0000 -- Cleaning the database ... 
2020-08-12 02:27:34 +0000 -- done
2020-08-12 02:27:34 +0000 -- Restoring database ... 
Restoring PostgreSQL database gitlabhq_production ... SET
SET
SET
SET
SET
 set_config 
------------
 
(1 row)

SET
SET
SET
SET
ERROR:  relation "public.u2f_registrations" does not exist
ERROR:  relation "public.timelogs" does not exist
...省略
ALTER TABLE
[DONE]
2020-08-12 02:28:19 +0000 -- done
2020-08-12 02:28:19 +0000 -- Restoring repositories ...
 * root/testproject ... [DONE]
2020-08-12 02:28:20 +0000 -- done
2020-08-12 02:28:20 +0000 -- Restoring uploads ... 
2020-08-12 02:28:20 +0000 -- done
2020-08-12 02:28:20 +0000 -- Restoring builds ... 
2020-08-12 02:28:20 +0000 -- done
2020-08-12 02:28:20 +0000 -- Restoring artifacts ... 
2020-08-12 02:28:20 +0000 -- done
2020-08-12 02:28:20 +0000 -- Restoring pages ... 
2020-08-12 02:28:20 +0000 -- done
2020-08-12 02:28:20 +0000 -- Restoring lfs objects ... 
2020-08-12 02:28:20 +0000 -- done
This task will now rebuild the authorized_keys file.
You will lose any data stored in the authorized_keys file.
Do you want to continue (yes/no)? yes

Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data 
and are not included in this backup. You will need to restore these files manually.
Restore task is done.
# 

整个过程中数据库数据导入最开始提示了relation不不存在的错误和需要手工处理gitlab.rb等文件的Warning，其余均显示正常完成。首先在目标备份机上的登录，可以看到已经不再提示需要进行密码修改了，输入备份侧的用户名和密码信息点击登录按钮

可以看到刚刚在host131的GitLab服务上创建的工程项目信息也已经成功恢复了