cinder volume 无法删除：ImageBusy error raised while deleting rbd volume.

cinder 错误日志：

ImageBusy error raised while deleting rbd volume. This may have been caused by a 
connection from a client that has crashed and, if so, may be resolved by retrying the 
delete after 30 seconds has elapsed.

到 ceph 查看 volume cbb25a66-6743-41f1-b60c-f1c54959770f 的 watchter 。发现 172.16.80.15 有 client 连接 给 volume

[root@ctrl01]# rbd status volumes/volume-cbb25a66-6743-41f1-b60c-f1c54959770f
Watchers:
    watcher=172.16.80.15:0/2240494846 client.3739308 cookie=139889011630080

到 172.16.80.15 查看连接的进程。确认在 OpenStack uuid 为110fb7b7-9c16-4384-bd7c-d52e2231d5d2 且在 172.16.80.15 节点domain 为 instance-0000095e 的实例在连接该 volume。

查看实例 110fb7b7-9c16-4384-bd7c-d52e2231d5d2 详情。发现该实例不存在，确认 instance-0000095e 为实例残留。

[root@ctrl02 ~]# nova show 110fb7b7-9c16-4384-bd7c-d52e2231d5d2
ERROR (CommandError): No server with a name or ID of '110fb7b7-9c16-4384-bd7c-d52e2231d5d2' exists.

清除 instance-0000095e

[root@com04 ~]# virsh destroy instance-0000095e
Domain instance-0000095e destroyed

[root@com04 ~]# virsh undefine instance-0000095e
Domain instance-0000095e has been undefined

删除 volume

[root@ctrl02 ~]# cinder delete cbb25a66-6743-41f1-b60c-f1c54959770f
Request to delete volume cbb25a66-6743-41f1-b60c-f1c54959770f has been accepted.