Openstack虚拟机关机后无法启动(start).md
Openstack虚拟机关机后无法启动(start)
文章目录
- Openstack虚拟机关机后无法启动(start)
- 分析解决
- 通过日志直接分析问题
- 排查flag，先看他是什么东西
- 再看异常
- 查看系统日志
#现象
执行启动虚拟机(nova start)之后，虚拟机并未启动，到该计算节点母机查看
/var/log/nova/nova-compute.log日志发现如下报错
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2044, in reboot
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher block_device_info)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2156, in _hard_reboot
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher vifs_already_plugged=True)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4429, in _create_domain_and_network
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher power_on=power_on)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4360, in _create_domain
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher LOG.error(err)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4350, in _create_domain
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher domain.createWithFlags(launch_flags)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher result = proxy_call(self._autowrap, f, *args, **kwargs)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher rv = execute(f, *args, **kwargs)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher six.reraise(c, e, tb)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher rv = meth(*args, **kwargs)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib64/python2.7/site-packages/libvirt.py", line 996, in createWithFlags
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher libvirtError: unsupported configuration: Unable to find security driver for label selinux
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher
2018-09-30 02:32:14.535 4537 INFO nova.compute.resource_tracker [req-efb15755-2bc0-45d2-946c-ba01bf451836 - - - - -] Auditing locally available compute resources for node openstack-com04
分析解决
通过日志直接分析问题
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher libvirtError: unsupported configuration: Unable to find security driver for label selinux
这两行获取到俩个信息
withflags failed说明flag可能有问题- 第二行不支持的安全驱动配置
排查flag，先看他是什么东西
根据上面日志信息
vim /usr/lib64/python2.7/site-packages/libvirt.py +996
"""Launch a defined domain. If the call succeeds the domain moves from the
defined to the running domains pools.
If the VIR_DOMAIN_START_PAUSED flag is set, or if the guest domain
has a managed save image that requested paused state (see
virDomainManagedSave()) the guest domain will be started, but its
CPUs will remain paused. The CPUs can later be manually started
using virDomainResume(). In all other cases, the guest domain will
be running.
If the VIR_DOMAIN_START_AUTODESTROY flag is set, the guest
domain will be automatically destroyed when the virConnectPtr
object is finally released. This will also happen if the
client application crashes / loses its connection to the
libvirtd daemon. Any domains marked for auto destroy will
block attempts at migration, save-to-file, or snapshots.
If the VIR_DOMAIN_START_BYPASS_CACHE flag is set, and there is a
managed save file for this domain (created by virDomainManagedSave()),
then libvirt will attempt to bypass the file system cache while restoring
the file, or fail if it cannot do so for the given system; this can allow
less pressure on file system cache, but also risks slowing loads from NFS.
If the VIR_DOMAIN_START_FORCE_BOOT flag is set, then any managed save
file for this domain is discarded, and the domain boots from scratch. """
ret = libvirtmod.virDomainCreateWithFlags(self._o, flags)
if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
通过注释猜测到flag是一个虚拟机状态标识符，如果是-1则掷出异常，-1一般是无效值，但这个虚拟机原来状态是好的，由此推测状态被破坏
再看异常
2018-09-30 02:31:27.083 4537 TRACE oslo_messaging.rpc.dispatcher libvirtError: unsupported configuration: Unable to find security driver for label selinux
先看下libvirt中关于security driver配置
grep security_driver /etc/libvirt/qemu.conf
# security_driver = [ "selinux", "apparmor" ]
# value of security_driver cannot contain "dac". The value "none" is
# a special value; security_driver can be set to that value in
#security_driver = "selinux"
可以看到都是注释的，再看下虚拟机xml文件里面有没有包含sec(安全)与lable的字段,虚拟机配置文件在
| 平台 | 虚拟机配置文件路径 |
|---|---|
| kvm | /etc/libvirt/qemu/ |
| openstack | /var/lib/nova/instances/ |
查看并对比配置文件之后未发现与其他有正常服务器有不同之处，没有发现
查看系统日志
Sep 30 02:28:38 openstack-com04 journal: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_lxc.so not accessible
Sep 30 02:28:38 openstack-com04 systemd: Started Virtualization daemon.
Sep 30 02:29:18 openstack-com04 journal: unsupported configuration: Unable to find security driver for label selinux
Sep 30 02:29:18 openstack-com04 journal: Unable to restore from managed state /var/lib/libvirt/qemu/save/instance-0000001f.save. Maybe the file is corrupted?
Sep 30 02:31:26 openstack-com04 journal: unsupported configuration: Unable to find security driver for label selinux
Sep 30 02:31:26 openstack-com04 journal: Unable to restore from managed state /var/lib/libvirt/qemu/save/instance-0000001f.save. Maybe the file is corrupted?
Sep 30 02:32:53 openstack-com04 kernel: device tapb4172400-9b entered promiscuous mode
kvm中.save后缀的文件是保存虚拟机状态的文件，从日志可以看出，无法从这个状态文件恢复信息，可以移除掉这个文件直接重启虚拟机即可
mv /var/lib/libvirt/qemu/save/instance-0000001f.save /tmp/201809