GitLab备份与恢复/迁移方案；

---

一、GitLab备份

1.确定备份目录
gitlab 默认的备份目录为/var/opt/gitlab/backups
可通过配置gitlab.rb配置文件进行修改，如：

[root@localhost ~]# vim /etc/gitlab/gitlab.rb
#若要修改备份文件的存储目录话，打开下面选项的注释并修改为自己的备份路径即可；
#gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"

#gitlab备份所保留的时长，默认为七天
#Limit backup lifetime to 7 days - 604800 seconds
#gitlab_rails['backup_keep_time'] = 604800

2.创建备份

[root@localhost ~]# gitlab-backup create
#或者使用
[root@localhost ~]# gitlab-rake gitlab:backup:create
2023-03-15 23:01:47 -0400 -- Dumping database ... 
Dumping PostgreSQL database gitlabhq_production ... [DONE]
2023-03-15 23:01:51 -0400 -- done
2023-03-15 23:01:51 -0400 -- Dumping repositories ..
...
Creating backup archive: 1678935707_2023_03_15_14.9.5_gitlab_backup.tar ... done
Uploading backup archive to remote storage  ... skipped
Deleting tmp directories ... done
done
done
done
done
done
done
done
done
done
Deleting old backups ... skipping
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data 
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
Backup task is done.

3、查看备份的文件
备份文件的格式为：
EPOCH_YYYY_MM_DD_GitLab version_gitlab_backup.tar

[root@localhost ~]# ll /var/opt/gitlab/backups/
总用量 360
-rw-------. 1 git git 368640 3月  15 23:01 1678935707_2023_03_15_14.9.5_gitlab_backup.tar

因为备份的是一个tar包，可以解压出来看下里面都有什么内容

[root@localhost ~]# cd /var/opt/gitlab/backups/
[root@localhost backups]# ll 
总用量 360
-rw-------. 1 git git 368640 3月  15 23:01 1678935707_2023_03_15_14.9.5_gitlab_backup.tar
[root@localhost backups]# tar -xf 1678935707_2023_03_15_14.9.5_gitlab_backup.tar 
[root@localhost backups]# ll 
总用量 392
-rw-------. 1 git  git  368640 3月  15 23:01 1678935707_2023_03_15_14.9.5_gitlab_backup.tar
-rw-------. 1 git  git     147 3月  15 23:01 artifacts.tar.gz 
-rw-r--r--. 1 git  git     190 3月  15 23:01 backup_information.yml 
-rw-------. 1 git  git     148 3月  15 23:01 builds.tar.gz 
drwxr-xr-x. 2 root root     29 3月  16 10:15 db #数据库备份：主要为PostgreSQL数据库内容
-rw-------. 1 git  git     147 3月  15 23:01 lfs.tar.gz 
-rw-------. 1 git  git     147 3月  15 23:01 packages.tar.gz 
-rw-------. 1 git  git     155 3月  15 23:01 pages.tar.gz
drwx------. 3 git  git      21 3月  15 23:01 repositories #git仓库的备份
-rw-------. 1 git  git     148 3月  15 23:01 terraform_state.tar.gz
-rw-------. 1 git  git     148 3月  15 23:01 uploads.tar.gz #附件数据的备份

4.除了自动备份gitlab的数据外，还需要手动备份如下必要文件
提示：如果是gitlab迁移的话，一定要进行下面的手动备份

/etc/gitlab/gitlab.rc
/etc/gitlab/gitlab-secrets.json

或备份整个/etc/gitlab目录也是可以的；

GitLab定时备份

[root@iZbp1awnpoj2h25jtjo8nxZ ~]# crontab -e
#每天凌晨两点执行gitlab备份
0 2 * * * /opt/gitlab/bin/gitlab-backup create CRON=1

CRON=1：环境设置将告诉备份脚本禁止所有进度输出. 这样就不会收到带有作业输出的冗余CRON电子邮件。

二、GitLab备份恢复

gitlab恢复数据前提：

• 执行恢复时需要手动恢复/etc/gitlab/gitlab-secrets.json文件，此文件中包含数据库加密密钥，CI/CD变量以及双因子认证等变量信息，如果在GitLab中使用到此部分内容，必须进行此文件的手动恢复。
• GitLab的版本必须要与备份数据的版本一致！！！

如下我有两个项目仓库，test-bak仓库里有个dev分支下面含有两个文件


此时我不小心将这个test-bak仓库删除了，需要进行恢复
确保删除前进行了数据备份


恢复步骤如下：
1：确保GitLab服务的启动可正常访问，并且版本必须和备份的数据版本一致；
2：在恢复备份副本(备份文件)之前，首先确保本分副本位于/var/opt/gitlab/backups目录中；
3：停止GitLab的数据库相关进程；

[root@localhost backups]# gitlab-ctl stop unicorn
[root@localhost backups]# gitlab-ctl stop sidekiq
ok: down: sidekiq: 0s, normally up
[root@localhost backups]# gitlab-ctl stop puma
ok: down: puma: 1s, normally up

4：验证GitLab服务的状态；

[root@localhost backups]# gitlab-ctl status
run: alertmanager: (pid 215917) 83981s; run: log: (pid 15049) 202063s
run: gitaly: (pid 215926) 83980s; run: log: (pid 14326) 202224s
run: gitlab-exporter: (pid 215486) 84087s; run: log: (pid 14940) 202089s
run: gitlab-kas: (pid 215488) 84087s; run: log: (pid 14646) 202204s
run: gitlab-workhorse: (pid 215498) 84087s; run: log: (pid 14810) 202111s
run: grafana: (pid 215506) 84086s; run: log: (pid 15347) 201991s
run: logrotate: (pid 349753) 1282s; run: log: (pid 14230) 202236s
run: nginx: (pid 215522) 84086s; run: log: (pid 174881) 107021s
run: node-exporter: (pid 215532) 84085s; run: log: (pid 14899) 202095s
run: postgres-exporter: (pid 215538) 84086s; run: log: (pid 15185) 202053s
run: postgresql: (pid 215546) 84085s; run: log: (pid 14502) 202212s
run: prometheus: (pid 215555) 84085s; run: log: (pid 15003) 202076s
down: puma: 18s, normally up; run: log: (pid 140781) 127605s
run: redis: (pid 215576) 84084s; run: log: (pid 14276) 202231s
run: redis-exporter: (pid 215582) 84084s; run: log: (pid 14968) 202084s
down: sidekiq: 239s, normally up; run: log: (pid 14757) 202123s

5：现在，使用备份副本的时间戳恢复备份；
恢复命令如下

gitlab-backup restore BACKUP=1678977671_2023_03_16_14.9.5
或
gitlab-rake gitlab:backup:restore BACKUP=1678977671_2023_03_16_14.9.5

BACKUP=备份归档文件时间戳 ： 使用指定的备份归档文件进行恢复

[root@localhost backups]# gitlab-backup restore BACKUP=1678977671_2023_03_16_14.9.5
Unpacking backup ... done
2023-03-16 10:58:44 -0400 -- Restoring database ... 
...
Do you want to continue (yes/no)? yes  #输入yes
...
Removing all tables. Press `Ctrl-C` within 5 seconds to abort
2023-03-16 15:00:50 UTC -- Cleaning the database ... 
2023-03-16 15:00:53 UTC -- done
Restoring PostgreSQL database gitlabhq_production ... ERROR:  must be owner of extension pg_trgm
ERROR:  must be owner of extension btree_gist
ERROR:  must be owner of extension btree_gist
ERROR:  must be owner of extension pg_trgm
...
2023-03-16 11:01:06 -0400 -- done
2023-03-16 11:01:06 -0400 -- Restoring uploads ... 
2023-03-16 11:01:06 -0400 -- done
2023-03-16 11:01:06 -0400 -- Restoring builds ... 
2023-03-16 11:01:06 -0400 -- done
2023-03-16 11:01:06 -0400 -- Restoring artifacts ... 
2023-03-16 11:01:06 -0400 -- done
2023-03-16 11:01:06 -0400 -- Restoring pages ... 
2023-03-16 11:01:06 -0400 -- done
2023-03-16 11:01:06 -0400 -- Restoring lfs objects ... 
2023-03-16 11:01:06 -0400 -- done
2023-03-16 11:01:06 -0400 -- Restoring terraform states ... 
2023-03-16 11:01:06 -0400 -- done
2023-03-16 11:01:06 -0400 -- Restoring packages ... 
2023-03-16 11:01:06 -0400 -- done
Do you want to continue (yes/no)? yes #输入yes即可

Deleting backups/tmp ... done
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data 
and are not included in this backup. You will need to restore these files manually.
Restore task is done.

在上面恢复的过程中可以看到有三处ERROR，这时不要慌张，急忙去打断恢复，亲测这三处ERROR不影响后续的恢复数据！只是权限的问题，下面会补充解决方案；

注：可以在执行恢复命令时添加force=yes参数，这个参数的意思是在恢复过程中不再进行交互式询问（不用再输入yes了）；
命令如下：

gitlab-backup restore force=yes BACKUP=1678977671_2023_03_16_14.9.5
或
gitlab-rake gitlab:backup:restore force=yes BACKUP=1678977671_2023_03_16_14.9.5

6：重新加载重新启动GitLab组件：

[root@localhost backups]# gitlab-ctl reconfigure
[root@localhost backups]# gitlab-ctl restart
ok: run: alertmanager: (pid 352840) 1s
ok: run: gitaly: (pid 352850) 0s
ok: run: gitlab-exporter: (pid 352864) 0s
ok: run: gitlab-kas: (pid 352866) 0s
ok: run: gitlab-workhorse: (pid 352872) 1s
ok: run: grafana: (pid 352884) 1s
ok: run: logrotate: (pid 352893) 0s
ok: run: nginx: (pid 352899) 0s
ok: run: node-exporter: (pid 352901) 0s
ok: run: postgres-exporter: (pid 352910) 0s
ok: run: postgresql: (pid 352921) 0s
ok: run: prometheus: (pid 352926) 0s
ok: run: puma: (pid 352928) 1s
ok: run: redis: (pid 352946) 1s
ok: run: redis-exporter: (pid 352952) 0s
ok: run: sidekiq: (pid 352958) 0s

7：通过清理数据库来检查GitLab组件是否都正常

[root@localhost backups]# gitlab-rake gitlab:check SANITIZE=true
//-SANITIZE = true标志会删除所有电子邮件地址，因为它们是保密的，可以删除CI变量和访问令牌，因为它们可以在生产实例中使用；

8：验证数据是否恢复

数据已全部恢复；

三、GitLab恢复数据时遇到的错误提示

如下

解决方案：
1.修改PostgreSQL配置

[root@localhost backups]# vim /var/opt/gitlab/postgresql/data/postgresql.conf
#修改监听地址
listen_addresses = '*'

[root@localhost backups]# vim /var/opt/gitlab/postgresql/data/pg_hba.conf
#在最下面新增这两行
local   all         all                               trust
host    all         all                               127.0.0.1/32 trust

2.重启gitlab

[root@localhost backups]# gitlab-ctl restart

3.修改PostgreSQL的gitlab账号权限为超级用户

[root@localhost backups]# gitlab-psql 
psql (12.7)
Type "help" for help.

gitlabhq_production=# ALTER USER gitlab WITH SUPERUSER;
ALTER ROLE
gitlabhq_production=# \q

然后再进行恢复数据的时候就不会有这个错误的提示了；