openstack创建实例的时候出现报错:实例 “test-1” 执行所请求操作失败,实例处于错误状态。: 请稍后再试 [错误: Build of instance 6ec6e8b1-9300-4be4-95fe-20434ea041f5 aborted: Volume e3dcad3d-dd22-4947-99d8-b8d0ad4b1025 did not finish being created even after we waited 3 seconds or 2 attempts. And its status is error.].
第一反应以为是nova的创建实例超时时间过短导致,但是报错的时间并没有超过默认的配置。以防万一,先更改nova的配置:
[root@cinder ~]#vi /etc/nova/nova.conf
block_device_allocate_retries=300
block_device_allocate_retries_interval=5
block_device_creation_timeout=600
重启nova服务。果然还是有报错。查看nova-compute的日志:
[root@cinder ~]#tail -100f /var/log/nova/nova-compute.log
2019-08-02 10:44:46.592 21266 INFO nova.virt.block_device [req-fbf4e8e9-776b-4d41-8010-a71fbeb242d9 107aa93638c04ab5b76f70ac8f5ac0bf 9cf598aaadf543389ab3
488b0510eb56 - default default] [instance: 7c469771-57e3-4ea7-a27e-5ddb1f48fd6a] Booting with volume-backed-image a4f43e2a-bc8f-40dd-bf32-a8ec504061fa at
/dev/vda
2019-08-02 10:44:50.983 21266 WARNING nova.compute.manager [req-fbf4e8e9-776b-4d41-8010-a71fbeb242d9 107aa93638c04ab5b76f70ac8f5ac0bf 9cf598aaadf543389ab
3488b0510eb56 - default default] Volume id: c61e0f6a-8327-4ec2-adcc-c8ccc90abfc2 finished being created but its status is error.
2019-08-02 10:44:50.984 21266 ERROR nova.compute.manager [req-fbf4e8e9-776b-4d41-8010-a71fbeb242d9 107aa93638c04ab5b76f70ac8f5ac0bf 9cf598aaadf543389ab34
88b0510eb56 - default default] [instance: 7c469771-57e3-4ea7-a27e-5ddb1f48fd6a] Instance failed block device setup: VolumeNotCreated: Volume c61e0f6a-832
7-4ec2-adcc-c8ccc90abfc2 did not finish being created even after we waited 3 seconds or 2 attempts. And its status is error.
发现实例状态错误,没有完成创建。这一报错实际在页面已经显示出来了。我们继续查看cinder节点的日志:
tail -100f /var/log/cinder/volume.log
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager [req-31aad7e7-406e-4b5e-a82b-c49be4c19e84 - - - - -] Failed to initialize driver.: ProcessExecu
tionError: Unexpected error while running command.
Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings -o name cinder-volumes
Exit code: 5
Stdout: u’’
Stderr: u’File descriptor 20 (/dev/urandom) leaked on vgs invocation. Parent PID 11967: /usr/bin/python2\n WARNING: Device for PV 2i8zCF-bKBd-IMYO-OkCq-
z9IJ-r2b4-EuUTdn not found or rejected by a filter.\n Volume group “cinder-volumes” not found\n Cannot process volume group cinder-volumes\n’
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager Traceback (most recent call last):
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager File “/usr/lib/python2.7/site-packages/cinder/volume/manager.py”, line 433, in init_host
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager self.driver.check_for_setup_error()
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager File “/usr/lib/python2.7/site-packages/cinder/volume/drivers/lvm.py”, line 297, in check_for_
setup_error
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager self.configuration.lvm_suppress_fd_warnings))
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager File “/usr/lib/python2.7/site-packages/cinder/brick/local_dev/lvm.py”, line 106, in init
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager if self._vg_exists() is False:
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager File “/usr/lib/python2.7/site-packages/cinder/brick/local_dev/lvm.py”, line 139, in _vg_exist
s
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager run_as_root=True)
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager File “/usr/lib/python2.7/site-packages/os_brick/executor.py”, line 52, in _execute
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager result = self.__execute(*args, **kwargs)
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager File “/usr/lib/python2.7/site-packages/cinder/utils.py”, line 128, in execute
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs)
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager File “/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py”, line 424, in execut
e
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager cmd=sanitized_cmd)
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager ProcessExecutionError: Unexpected error while running command.
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings -o name c
inder-volumes
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager Exit code: 5
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager Stdout: u’’
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager Stderr: u’File descriptor 20 (/dev/urandom) leaked on vgs invocation. Parent PID 11967: /usr/bi
n/python2\n WARNING: Device for PV 2i8zCF-bKBd-IMYO-OkCq-z9IJ-r2b4-EuUTdn not found or rejected by a filter.\n Volume group “cinder-volumes” not found
n Cannot process volume group cinder-volumes\n’
2019-08-02 09:54:11.957 11959 ERROR cinder.volume.manager
2019-08-02 09:54:12.080 11959 INFO cinder.volume.manager [req-31aad7e7-406e-4b5e-a82b-c49be4c19e84 - - - - -] Initializing RPC dependent components of vo
lume driver LVMVolumeDriver (3.0.0)
2019-08-02 09:54:12.081 11959 ERROR cinder.utils [req-31aad7e7-406e-4b5e-a82b-c49be4c19e84 - - - - -] Volume driver LVMVolumeDriver not initialized
2019-08-02 09:54:12.081 11959 ERROR cinder.volume.manager [req-31aad7e7-406e-4b5e-a82b-c49be4c19e84 - - - - -] Cannot complete RPC initialization because
driver isn’t initialized properly.: DriverNotInitialized: \u5377\u9a71\u52a8\u672a\u51c6\u5907\u597d\u3002
2019-08-02 09:54:22.083 11959 ERROR cinder.service [-] Manager for service cinder-volume cinder@lvm is reporting problems, not sending heartbeat. Service
will appear “down”.
可以看到主要错误是Volume group “cinder-volumes” not found
原来是找不到名称为cinder-volumes的卷组。通过命令
df -h 和 fdisk -l 查看我们的挂载点和磁盘分区信息。发现没有这个卷组。笔者这里的服务器只有一个磁盘,已经全部使用了。测试更改cinder配置中volume_group的名称为磁盘已创建卷组的名称。例如:
我的服务器分区为:
磁盘 /dev/mapper/VolGroup-lv_home:3382 GB, 33822867456 字节,6606028863 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理):512 字节 / 512 字节
I/O 大小(最小/最佳):512 字节 / 512 字节
磁盘 /dev/mapper/VolGroup-lv_root:53.7 GB, 53687091200 字节,104857600 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理):512 字节 / 512 字节
I/O 大小(最小/最佳):512 字节 / 512 字节
vi /etc/cinder/cinder.conf
[lvm]
volume_group = VolGroup
重启cinder服务
发现仍然报错,继续查看cinder日志。发现错误内容变了:
2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager [req-71b1fecf-c29f-4852-8323-ccac1cd44587 - - - - -] Failed to initialize driver.: ProcessExecu
tionError: Unexpected error while running command.
Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvcreate -T -L 0.0g VolGroup/VolGroup-pool
Exit code: 3
Stdout: u’’
Stderr: u"File descriptor 20 (/dev/urandom) leaked on lvcreate invocation. Parent PID 12542: /usr/bin/python2\n --size may not be zero.\n Run lvcreate --help' for more information.\n" 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager Traceback (most recent call last): 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 433, in init_host 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager self.driver.check_for_setup_error() 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/lvm.py", line 297, in check_for_ setup_error 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager self.configuration.lvm_suppress_fd_warnings)) 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/brick/local_dev/lvm.py", line 116, in __init__ 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager self.create_thin_pool(pool_name) 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/brick/local_dev/lvm.py", line 528, in create_th in_pool 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager run_as_root=True) 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/os_brick/executor.py", line 52, in _execute 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager result = self.__execute(*args, **kwargs) 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 128, in execute 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs) 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 424, in execut e 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager cmd=sanitized_cmd) 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager ProcessExecutionError: Unexpected error while running command. 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvcreate -T -L 0.0g VolGro up/VolGroup-pool 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager Exit code: 3 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager Stdout: u'' 2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager Stderr: u"File descriptor 20 (/dev/urandom) leaked on lvcreate invocation. Parent PID 12542: /u sr/bin/python2\n --size may not be zero.\n Run
lvcreate --help’ for more information.\n"
2019-08-02 13:49:48.326 12510 ERROR cinder.volume.manager
2019-08-02 13:49:48.436 12510 INFO cinder.volume.manager [req-71b1fecf-c29f-4852-8323-ccac1cd44587 - - - - -] Initializing RPC dependent components of vo
lume driver LVMVolumeDriver (3.0.0)
2019-08-02 13:49:48.436 12510 ERROR cinder.utils [req-71b1fecf-c29f-4852-8323-ccac1cd44587 - - - - -] Volume driver LVMVolumeDriver not initialized
2019-08-02 13:49:48.437 12510 ERROR cinder.volume.manager [req-71b1fecf-c29f-4852-8323-ccac1cd44587 - - - - -] Cannot complete RPC initialization because
driver isn’t initialized properly.: DriverNotInitialized: \u5377\u9a71\u52a8\u672a\u51c6\u5907\u597d\u3002
2019-08-02 13:49:58.438 12510 ERROR cinder.service [-] Manager for service cinder-volume cinder@lvm is reporting problems, not sending heartbeat. Service
will appear “down”.
无法初始化驱动。这里笔者的方法比较简单粗暴。备份home分区中的数据到别的地方。先卸载挂载的home。删除逻辑卷,使用truncate创建一个虚拟文件,通过这个虚拟文件来创建卷组,具体步骤如下:
[root@cinder ~]#umount /home
[root@cinder ~]#lvremove /dev/mapper/VolGroup-lv_home
[root@cinder ~]#truncate -s 7000G virtual_volume
[root@cinder ~]#losetup -f virtual_volume /dev/loop0
[root@cinder ~]#losetup /dev/loop0 virtual_volume
创建cinder卷组
[root@cinder ~]#vgcreate cinder-volumes /dev/loop0
修改filter过滤器
[root@cinder ~]#vi /etc/lvm/lvm.conf
filter = [ “a|loop0|”, “r|.*|” ]
修改cinder配置文件,cinder-volume名称
[root@cinder ~]#vi /etc/cinder/cinder.conf
[lvm]
volume_group = cinder-volumes
重启服务
[root@cinder ~]#systemctl restart openstack-cinder-volume.service target.service