devstack服务器断电重启后,openstack有些服务无法启动。
source /opt/stack/devstack/openrc admin admin
nova service-list --binary nova-compute
/opt/stack/devstack/tools/discover_hosts.sh
执行:
systemctl restart devstack@*
重启服务后,nova服务up状态。但是cinder-volume服务开始up状态,过一会儿是down状态。
查看cinder服务状态:
cinder service-list
开始cinder服务up,
查询日志:
sudo journalctl -a -u [email protected] | grep "ERROR"
显示:
Aug 25 12:04:05 computer cinder-volume[5124]: ERROR cinder.utils [None req-5f09d1d8-9f85-4f77-82f0-eff577d4de0e None None] Volume driver LVMVolumeDriver not initialized
Aug 25 12:04:05 computer cinder-volume[5124]: ERROR cinder.volume.manager [None req-5f09d1d8-9f85-4f77-82f0-eff577d4de0e None None] Cannot complete RPC initialization because driver isn't initialized properly.: cinder.exception.DriverNotInitialized: Volume driver not ready.
Aug 25 12:04:15 computer cinder-volume[5124]: ERROR cinder.service [-] Manager for service cinder-volume computer@lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
过一会儿再执行:
cinder service-list
输出:
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
| Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
| cinder-scheduler | controller | nova | enabled | up | 2020-08-25T06:19:08.000000 | - |
| cinder-volume | controller@lvmdriver-1 | nova | enabled | down | 2020-08-25T06:19:00.000000 | - |
| cinder-volume | computer@lvmdriver-1 | nova | enabled | down | 2020-08-25T06:19:06.000000 | - |
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
开始以为是时间同步问题,但是检查时间后,发现不是时间同步问题。
controller 下执行:
vgs
无输出。
computer下,
执行:
vgs
也无输出。
分别再controller和computer下执行:
sudo losetup /dev/loop0 /opt/stack/data/stack-volumes-default-backing-file
输出:
losetup: /opt/stack/data/stack-volumes-default-backing-file: failed to set up loop device: Device or resource busy
重启服务器:
重复上述步骤
再执行(#将/opt/stack/data/stack-volumes-lvmdriver-1-backing-file绑在第一个空闲的loop):
sudo losetup -f /opt/stack/data/stack-volumes-lvmdriver-1-backing-file
再分别执行
vgs
显示:
VG #PV #LV #SN Attr VSize VFree
stack-volumes-lvmdriver-1 1 1 0 wz--n- <24.00g <1.15g
再次重启cinder服务:
systemctl restart devstack@c*
cinder service-list
显示:
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
| Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
| cinder-scheduler | controller | nova | enabled | up | 2020-08-25T06:19:08.000000 | - |
| cinder-volume | controller@lvmdriver-1 | nova | enabled | up | 2020-08-25T06:19:00.000000 | - |
| cinder-volume | computer@lvmdriver-1 | nova | enabled | up | 2020-08-25T06:19:06.000000 | - |
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
问题解决。
但是问题产生具体原因 待查。
参考文档:
《cinder-volume down》
删除多余的cinder volume:
cinder-manage service remove cinder-volume computer@lvmdriver-2