Gitlab Admin管理页面提示500内部错误(500 Internal error)的解决办法

Gitlab Admin管理页面提示500内部错误(500 Internal error)的解决办法

参考

  • Doctor Rake tasks
  • Reset runner registration tokens
  • Issue-26171

现象

很多Admin区域的配置在修改后保存时,页面报错500 Internal Error

解决办法

首先,遍历数据库中所有可能的加密值,验证它们是否可使用 gitlab-secrets.json解密。

root@VM-0-15-ubuntu:~# sudo gitlab-rake gitlab:doctor:secrets
I, [2021-08-11T22:08:52.783505 #28545]  INFO -- : Checking encrypted values in the database
I, [2021-08-11T22:09:07.804776 #28545]  INFO -- : - ApplicationSetting failures: 1
I, [2021-08-11T22:09:07.945049 #28545]  INFO -- : - User failures: 0
I, [2021-08-11T22:09:07.960477 #28545]  INFO -- : - GeoNode failures: 0
I, [2021-08-11T22:09:07.975234 #28545]  INFO -- : - Ci::InstanceVariable failures: 0
I, [2021-08-11T22:09:07.991697 #28545]  INFO -- : - Clusters::Platforms::Kubernetes failures: 0
I, [2021-08-11T22:09:08.005759 #28545]  INFO -- : - Snippet failures: 0
I, [2021-08-11T22:09:08.009996 #28545]  INFO -- : - PersonalSnippet failures: 0
I, [2021-08-11T22:09:08.013390 #28545]  INFO -- : - ProjectSnippet failures: 0
I, [2021-08-11T22:09:08.028282 #28545]  INFO -- : - Ci::PipelineScheduleVariable failures: 0
I, [2021-08-11T22:09:08.042971 #28545]  INFO -- : - Ci::GroupVariable failures: 0
I, [2021-08-11T22:09:08.057457 #28545]  INFO -- : - Clusters::Applications::Helm failures: 0
I, [2021-08-11T22:09:08.079662 #28545]  INFO -- : - Clusters::Applications::Prometheus failures: 0
I, [2021-08-11T22:09:08.095962 #28545]  INFO -- : - Ci::Variable failures: 0
I, [2021-08-11T22:09:08.111584 #28545]  INFO -- : - AlertManagement::HttpIntegration failures: 0
I, [2021-08-11T22:09:08.126730 #28545]  INFO -- : - Alerting::ProjectAlertingSetting failures: 0
I, [2021-08-11T22:09:08.142377 #28545]  INFO -- : - Atlassian::Identity failures: 0
I, [2021-08-11T22:09:08.158268 #28545]  INFO -- : - BulkImports::Configuration failures: 0
I, [2021-08-11T22:09:08.174411 #28545]  INFO -- : - Ci::JobVariable failures: 0
I, [2021-08-11T22:09:08.190214 #28545]  INFO -- : - Ci::PipelineVariable failures: 0
I, [2021-08-11T22:09:08.192590 #28545]  INFO -- : - Clusters::Integrations::Prometheus failures: 0
I, [2021-08-11T22:09:08.207469 #28545]  INFO -- : - Clusters::KubernetesNamespace failures: 0
I, [2021-08-11T22:09:08.223416 #28545]  INFO -- : - Clusters::Providers::Aws failures: 0
I, [2021-08-11T22:09:08.241831 #28545]  INFO -- : - Clusters::Providers::Gcp failures: 0
I, [2021-08-11T22:09:08.262736 #28545]  INFO -- : - ErrorTracking::ProjectErrorTrackingSetting failures: 0
I, [2021-08-11T22:09:08.280151 #28545]  INFO -- : - GrafanaIntegration failures: 0
I, [2021-08-11T22:09:08.297219 #28545]  INFO -- : - WebHook failures: 0
I, [2021-08-11T22:09:08.301177 #28545]  INFO -- : - ProjectHook failures: 0
I, [2021-08-11T22:09:08.304130 #28545]  INFO -- : - ServiceHook failures: 0
I, [2021-08-11T22:09:08.307443 #28545]  INFO -- : - SystemHook failures: 0
I, [2021-08-11T22:09:08.311309 #28545]  INFO -- : - GroupHook failures: 0
I, [2021-08-11T22:09:08.327254 #28545]  INFO -- : - IncidentManagement::ProjectIncidentManagementSetting failures: 0
I, [2021-08-11T22:09:08.341923 #28545]  INFO -- : - JiraConnectInstallation failures: 0
I, [2021-08-11T22:09:08.358597 #28545]  INFO -- : - Packages::Debian::GroupDistribution failures: 0
I, [2021-08-11T22:09:08.373612 #28545]  INFO -- : - Packages::Debian::ProjectDistribution failures: 0
I, [2021-08-11T22:09:08.390188 #28545]  INFO -- : - PagesDomain failures: 0
I, [2021-08-11T22:09:08.405048 #28545]  INFO -- : - PagesDomainAcmeOrder failures: 0
I, [2021-08-11T22:09:08.419118 #28545]  INFO -- : - ProjectImportData failures: 0
I, [2021-08-11T22:09:08.434796 #28545]  INFO -- : - IssueTrackerData failures: 0
I, [2021-08-11T22:09:08.451478 #28545]  INFO -- : - JiraTrackerData failures: 0
I, [2021-08-11T22:09:08.471227 #28545]  INFO -- : - OpenProjectTrackerData failures: 0
I, [2021-08-11T22:09:08.486142 #28545]  INFO -- : - RemoteMirror failures: 0
I, [2021-08-11T22:09:08.501184 #28545]  INFO -- : - Serverless::DomainCluster failures: 0
I, [2021-08-11T22:09:08.504077 #28545]  INFO -- : - Gitlab::BackgroundMigration::BackfillJiraTrackerDeploymentType2::JiraTrackerDataTemp failures: 0
I, [2021-08-11T22:09:08.507262 #28545]  INFO -- : - Gitlab::BackgroundMigration::MigrateIssueTrackersSensitiveData::IssueTrackerData failures: 0
I, [2021-08-11T22:09:08.509604 #28545]  INFO -- : - Gitlab::BackgroundMigration::MigrateIssueTrackersSensitiveData::JiraTrackerData failures: 0
I, [2021-08-11T22:09:08.524464 #28545]  INFO -- : - Dast::SiteProfileSecretVariable failures: 0
I, [2021-08-11T22:09:08.539281 #28545]  INFO -- : - StatusPage::ProjectSetting failures: 0
I, [2021-08-11T22:09:08.924548 #28545]  INFO -- : - Group failures: 0
I, [2021-08-11T22:09:09.333398 #28545]  INFO -- : - Project failures: 0
I, [2021-08-11T22:09:09.353966 #28545]  INFO -- : - Ci::Runner failures: 0
I, [2021-08-11T22:09:09.378477 #28545]  INFO -- : - Ci::Build failures: 0
I, [2021-08-11T22:09:09.397156 #28545]  INFO -- : - DeployToken failures: 0
I, [2021-08-11T22:09:09.413991 #28545]  INFO -- : - Clusters::AgentToken failures: 0
I, [2021-08-11T22:09:09.429953 #28545]  INFO -- : - Operations::FeatureFlagsClient failures: 0
I, [2021-08-11T22:09:09.447277 #28545]  INFO -- : - ScimOauthAccessToken failures: 0
I, [2021-08-11T22:09:09.448701 #28545]  INFO -- : Total: 1 row(s) affected
I, [2021-08-11T22:09:09.448768 #28545]  INFO -- : Done!

如果有无法解密的值,可以按照以下步骤重置它们:

root@VM-0-15-ubuntu:~# sudo gitlab-rails dbconsole
Password for user gitlab: 
psql (12.6)
Type "help" for help.

gitlabhq_production=> UPDATE projects SET runners_token = null, runners_token_encrypted = null;
UPDATE 2
gitlabhq_production=> 
gitlabhq_production=> UPDATE namespaces SET runners_token = null, runners_token_encrypted = null;
UPDATE 6
gitlabhq_production=> 
gitlabhq_production=> UPDATE application_settings SET runners_registration_token_encrypted = null;
UPDATE 1
gitlabhq_production=> 
gitlabhq_production=> UPDATE application_settings SET encrypted_ci_jwt_signing_key = null;
UPDATE 1
gitlabhq_production=> 
gitlabhq_production=> UPDATE ci_runners SET token = null, token_encrypted = null;
UPDATE 0
gitlabhq_production=> 
gitlabhq_production=> 
gitlabhq_production=> exit 
could not save history to file "/var/opt/gitlab/.psql_history": Permission denied
root@VM-0-15-ubuntu:~# sudo gitlab-rake gitlab:doctor:secrets
I, [2021-08-11T22:15:16.600635 #31117]  INFO -- : Checking encrypted values in the database
I, [2021-08-11T22:15:31.465633 #31117]  INFO -- : - ApplicationSetting failures: 0
I, [2021-08-11T22:15:31.601481 #31117]  INFO -- : - User failures: 0
I, [2021-08-11T22:15:31.617889 #31117]  INFO -- : - GeoNode failures: 0
I, [2021-08-11T22:15:31.633402 #31117]  INFO -- : - Ci::InstanceVariable failures: 0
I, [2021-08-11T22:15:31.648806 #31117]  INFO -- : - Clusters::Platforms::Kubernetes failures: 0
I, [2021-08-11T22:15:31.663590 #31117]  INFO -- : - Snippet failures: 0
I, [2021-08-11T22:15:31.666048 #31117]  INFO -- : - PersonalSnippet failures: 0
I, [2021-08-11T22:15:31.668886 #31117]  INFO -- : - ProjectSnippet failures: 0
I, [2021-08-11T22:15:31.683639 #31117]  INFO -- : - Ci::PipelineScheduleVariable failures: 0
I, [2021-08-11T22:15:31.698858 #31117]  INFO -- : - Ci::GroupVariable failures: 0
I, [2021-08-11T22:15:31.712966 #31117]  INFO -- : - Clusters::Applications::Helm failures: 0
I, [2021-08-11T22:15:31.727849 #31117]  INFO -- : - Clusters::Applications::Prometheus failures: 0
I, [2021-08-11T22:15:31.743283 #31117]  INFO -- : - Ci::Variable failures: 0
I, [2021-08-11T22:15:31.757994 #31117]  INFO -- : - AlertManagement::HttpIntegration failures: 0
I, [2021-08-11T22:15:31.772461 #31117]  INFO -- : - Alerting::ProjectAlertingSetting failures: 0
I, [2021-08-11T22:15:31.787215 #31117]  INFO -- : - Atlassian::Identity failures: 0
I, [2021-08-11T22:15:31.800757 #31117]  INFO -- : - BulkImports::Configuration failures: 0
I, [2021-08-11T22:15:31.814656 #31117]  INFO -- : - Ci::JobVariable failures: 0
I, [2021-08-11T22:15:31.829898 #31117]  INFO -- : - Ci::PipelineVariable failures: 0
I, [2021-08-11T22:15:31.832273 #31117]  INFO -- : - Clusters::Integrations::Prometheus failures: 0
I, [2021-08-11T22:15:31.846476 #31117]  INFO -- : - Clusters::KubernetesNamespace failures: 0
I, [2021-08-11T22:15:31.863140 #31117]  INFO -- : - Clusters::Providers::Aws failures: 0
I, [2021-08-11T22:15:31.879091 #31117]  INFO -- : - Clusters::Providers::Gcp failures: 0
I, [2021-08-11T22:15:31.893133 #31117]  INFO -- : - ErrorTracking::ProjectErrorTrackingSetting failures: 0
I, [2021-08-11T22:15:31.907856 #31117]  INFO -- : - GrafanaIntegration failures: 0
I, [2021-08-11T22:15:31.924289 #31117]  INFO -- : - WebHook failures: 0
I, [2021-08-11T22:15:31.927353 #31117]  INFO -- : - ProjectHook failures: 0
I, [2021-08-11T22:15:31.930184 #31117]  INFO -- : - ServiceHook failures: 0
I, [2021-08-11T22:15:31.933803 #31117]  INFO -- : - SystemHook failures: 0
I, [2021-08-11T22:15:31.936799 #31117]  INFO -- : - GroupHook failures: 0
I, [2021-08-11T22:15:31.954429 #31117]  INFO -- : - IncidentManagement::ProjectIncidentManagementSetting failures: 0
I, [2021-08-11T22:15:31.971990 #31117]  INFO -- : - JiraConnectInstallation failures: 0
I, [2021-08-11T22:15:31.986940 #31117]  INFO -- : - Packages::Debian::GroupDistribution failures: 0
I, [2021-08-11T22:15:32.002329 #31117]  INFO -- : - Packages::Debian::ProjectDistribution failures: 0
I, [2021-08-11T22:15:32.017951 #31117]  INFO -- : - PagesDomain failures: 0
I, [2021-08-11T22:15:32.033981 #31117]  INFO -- : - PagesDomainAcmeOrder failures: 0
I, [2021-08-11T22:15:32.049288 #31117]  INFO -- : - ProjectImportData failures: 0
I, [2021-08-11T22:15:32.067100 #31117]  INFO -- : - IssueTrackerData failures: 0
I, [2021-08-11T22:15:32.083323 #31117]  INFO -- : - JiraTrackerData failures: 0
I, [2021-08-11T22:15:32.098613 #31117]  INFO -- : - OpenProjectTrackerData failures: 0
I, [2021-08-11T22:15:32.113851 #31117]  INFO -- : - RemoteMirror failures: 0
I, [2021-08-11T22:15:32.128925 #31117]  INFO -- : - Serverless::DomainCluster failures: 0
I, [2021-08-11T22:15:32.131172 #31117]  INFO -- : - Gitlab::BackgroundMigration::BackfillJiraTrackerDeploymentType2::JiraTrackerDataTemp failures: 0
I, [2021-08-11T22:15:32.133837 #31117]  INFO -- : - Gitlab::BackgroundMigration::MigrateIssueTrackersSensitiveData::IssueTrackerData failures: 0
I, [2021-08-11T22:15:32.136193 #31117]  INFO -- : - Gitlab::BackgroundMigration::MigrateIssueTrackersSensitiveData::JiraTrackerData failures: 0
I, [2021-08-11T22:15:32.152574 #31117]  INFO -- : - Dast::SiteProfileSecretVariable failures: 0
I, [2021-08-11T22:15:32.168309 #31117]  INFO -- : - StatusPage::ProjectSetting failures: 0
I, [2021-08-11T22:15:32.536621 #31117]  INFO -- : - Group failures: 0
I, [2021-08-11T22:15:32.946062 #31117]  INFO -- : - Project failures: 0
I, [2021-08-11T22:15:32.965917 #31117]  INFO -- : - Ci::Runner failures: 0
I, [2021-08-11T22:15:32.991323 #31117]  INFO -- : - Ci::Build failures: 0
I, [2021-08-11T22:15:33.010162 #31117]  INFO -- : - DeployToken failures: 0
I, [2021-08-11T22:15:33.027674 #31117]  INFO -- : - Clusters::AgentToken failures: 0
I, [2021-08-11T22:15:33.043918 #31117]  INFO -- : - Operations::FeatureFlagsClient failures: 0
I, [2021-08-11T22:15:33.063264 #31117]  INFO -- : - ScimOauthAccessToken failures: 0
I, [2021-08-11T22:15:33.063377 #31117]  INFO -- : Total: 0 row(s) affected
I, [2021-08-11T22:15:33.063397 #31117]  INFO -- : Done!

你可能感兴趣的:(GitLab,gitlab,ci/cd)