[优化篇]OpenStack的Cinder后端存储技术——NFS

题记

OpenStack的Cinder可以为虚拟机提供后台块存储资源，来扩展虚拟机的存储资源，而且Cinder的后端存储可以支持非常多的存储技术，包括NFS、GlusterFS、Ceph、以及不同存储厂商的硬件驱动，默认帮助文档上支持的是LVM，本文主要介绍基于NFS的后台存储。

环境

Ubuntu 14.04 LTS、OpenStack for icehouse 
使用VM Ware搭建的三个节点的GRE网络架构的虚拟机，另外添加了一个存储节点。

操作步骤

我是将原来的LVM方式修改为NFS。

存储节点

1、在存储节点安装rpcbind和nfs-kernel-server软件包 
2、创建一个路径，共享出去例如/data/nfs

mkdir -p /data/nfs

3、修改权限，建议使用cinder用户和组

root@cinder:~# ll /data/
total 12
drwxr-xr-x  3 root   root   4096 Jul 17 10:32 ./
drwxr-xr-x 23 root   root   4096 Jul 17 10:32 ../
drwxr-xr-x  2 cinder cinder 4096 Jul 17 15:40 nfs/

4、添加/etc/exportfs配置，共享出去

root@cinder:~# cat /etc/exports 
# /etc/exports: the access control list for filesystems which may be exported
#       to NFS clients.  See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
#
/data/nfs *(rw,sync,no_root_squash)

5、重启rpcbind和nfs-kernel-server服务 
6、创建/etc/cinder/nfs_shares文件，并修改其组和权限

-rw-r-----  1 root   cinder    24 Jul 17 10:37 nfs_shares

7、修改/etc/cinder/cinder.conf配置文件，添加如下内容[DEFAULT]

#添加nfs驱动
volume_driver=cinder.volume.drivers.nfs.NfsDriver
#创建nfs共享存储文件，可以自定义，就是步骤6创建的
nfs_shares_config=/etc/cinder/nfs_shares
#指定挂载点
nfs_mount_point_base=$state_path/mnt

8、重启cinder-volume服务 
9、查看mount挂载信息 
我们可以看到重启完cinder-volume服务之后，可以看到系统会自动mount上相关路径

192.168.3.185:/data/nfs on /var/lib/cinder/mnt/74877087a01856b116a3558d2981626e type nfs4 (rw,relatime,vers=4.1,rsize=262144,wsize=262144,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.3.185,local_lock=none,addr=192.168.3.185)

控制节点 
更新/etc/cinder/cinder.conf的nfs驱动

volume_driver=cinder.volume.drivers.nfs.NfsDriver

计算节点 
1、nova.conf更新cinder相关配置

volume_api_class=nova.volume.cinder.API

最后，建议在所有计算和控制节点安装rpcbind、nfs-common、nfs-kernel-server软件包，我在实施过程中遇到了一个问题，我可以创建好的我云硬盘，但是就是挂接不到我的虚拟机上。 
最后查看计算节点/var/log/nova/nova-compute.log有以下错误

2015-07-17 15:46:53.149 44244 TRACE oslo.messaging.rpc.dispatcher Stderr: u'mount: wrong fs type, bad option, bad superblock on 192.168.3.185:/data/nfs,\n       missing codepage or helper program, or other error\n       (for several filesystems (e.g. nfs, cifs) you might\n       need a /sbin/mount. helper program)\n       In some cases useful info is found in syslog - try\n       dmesg | tail  or so\n\n'
2015-07-17 15:46:53.149 44244 TRACE oslo.messaging.rpc.dispatcher
2015-07-17 15:46:53.151 44244 ERROR oslo.messaging._drivers.common [-] Returning exception Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf mount -t nfs 192.168.3.185:/data/nfs /var/lib/nova/mnt/74877087a01856b116a3558d2981626e
Exit code: 32
Stdout: u''
Stderr: u'mount: wrong fs type, bad option, bad superblock on 192.168.3.185:/data/nfs,\n       missing codepage or helper program, or other error\n       (for several filesystems (e.g. nfs, cifs) you might\n       need a /sbin/mount. helper program)\n       In some cases useful info is found in syslog - try\n       dmesg | tail  or so\n\n' to caller
@                                                                                                                                                                                                         @                   

最后在计算节点安装上述nfs安装包，问题解决。

验证

1、我们使用horizon创建一个1GB的云硬盘，我们可以看到cinder的挂载点

root@cinder:~# ll /var/lib/cinder/mnt/74877087a01856b116a3558d2981626e/
total 8
drwxr-xr-x 2 cinder cinder       4096 Jul 17 15:40 ./
drwxr-xr-x 3 cinder cinder       4096 Jul 17 10:39 ../
-rw-rw-rw- 1    108 ntp    1073741824 Jul 17 10:40 volume-d46c486d-71fe-4863-95b5-864621ca9353

2、其实该文件存储在共享文件夹中/data/nfs

root@cinder:~# ll /data/nfs
total 8
drwxr-xr-x 2 cinder cinder       4096 Jul 17 15:40 ./
drwxr-xr-x 3 root   root         4096 Jul 17 10:32 ../
-rw-rw-rw- 1    108 ntp    1073741824 Jul 17 10:40 volume-d46c486d-71fe-4863-95b5-864621ca9353

3、查看cinder列表

root@controller:~# cinder list
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
|                  ID                  | Status | Display Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
| d46c486d-71fe-4863-95b5-864621ca9353 | in-use |      a       |  1   |     None    |  false   | 614a2641-1e8d-4442-9704-6ab62e3f39d5 |
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+