iSCSI使用TCP/IP协议对存储使用的SCSI指令进行封装，使得可通过TCP/IP网络访问持久化存储。本文在CentOS7上对iSCSI的配置和使用进行介绍。
1. 实验环境
-
虚拟化软件：
- Virtual Box 5.6.2
- Vagrant 2.0.2
-
实验虚机：
- iscsi-disks: 192.168.56.20 （iSCSI target，提供存储设备），默认配置1个cpu，1G内存。
- iscsi-host: 192.168.56.21 （iSCSI initiator，访问iSCSI设备的主机），默认配置1个cpu，1G内存。
安装和管理网络：192.168.56.0/24，该网络为VirtualBox的Host-Only网络，支持物理机和VirtualBox虚机间的互相访问。
2. 克隆项目并启动上述虚拟机
本文中的实验仅涉及一个主机节点和一个存储节点。
$ git clone https://github.com/lprincewhn/iscsi.git
$ cd iscsi
$ vagrant up
虚拟机启动完毕后可使用以下用户登陆：
- root/vagrant
- vagrant/vagrant
3. 创建虚拟磁盘
使用root用户登陆iscsi-disks，使用losetup来创建实验用的存储设备。
Step 1 创建大文件用于支持虚拟存储设备
[root@iscsi-disks ~]# dd if=/dev/zero of=/lun1.img bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 2.29056 s, 469 MB/s
为了避免Selinux权限问题，建议不要将文件创建在/root目录下。
Step 2 创建loop设备
[root@iscsi-disks ~]# losetup -f
/dev/loop0
[root@iscsi-disks ~]# losetup /dev/loop0 /lun1.img
[root@iscsi-disks ~]# losetup
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE
/dev/loop0 0 0 0 0 /lun1.img
Step 3 安装iSCSI target软件包并启动服务
[root@iscsi-disks ~]# yum -y install targetcli
[root@iscsi-disks ~]# systemctl enable target && systemctl start target
Step 4 在targetcli命令行中创建存储设备
[root@iscsi-disks ~]# targetcli
Warning: Could not load preferences file /root/.targetcli/prefs.bin.
targetcli shell version 2.1.fb46
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
/> /backstores/block create ib_vol1 /dev/loop0 # 创建Block
Created block storage object ib_vol1 using /dev/loop0.
/> /iscsi create iqn.2016-06.com.iscsi-disks:iscsi-disks # 定义存储节点
Created target iqn.2016-06.com.iscsi-disks:iscsi-disks.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
/> /iscsi/iqn.2016-06.com.iscsi-disks:iscsi-disks/tpg1/acls create iqn.2016-06.com.iscsi-host:iscsi-host # 将访问主机加入设备ACL，这样主机才能访问到这个设备
Created Node ACL for iqn.2016-06.com.iscsi-host:iscsi-host
/> /iscsi/iqn.2016-06.com.iscsi-disks:iscsi-disks/tpg1/luns create /backstores/block/ib_vol1 # 使用Block定义存储节点上的lun
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2016-06.com.iscsi-host:iscsi-host
/> ls # 查看上述创建的内容及关系
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 1]
| | o- ib_vol1 .................... [/dev/loop0 (1.0GiB) write-thru activated]
| | o- alua ............................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ................... [ALUA state: Active/optimized]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 1]
| o- iqn.2016-06.com.iscsi-disks:iscsi-disks ....................... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
| o- acls ...................................................... [ACLs: 1]
| | o- iqn.2016-06.com.iscsi-host:iscsi-host ............ [Mapped LUNs: 1]
| | o- mapped_lun0 ........................... [lun0 block/ib_vol1 (rw)]
| o- luns ...................................................... [LUNs: 1]
| | o- lun0 .............. [block/ib_vol1 (/dev/loop0) (default_tg_pt_gp)]
| o- portals ................................................ [Portals: 1]
| o- 0.0.0.0:3260 ................................................. [OK]
o- loopback ..................................................... [Targets: 0]
注：同一台主机可以创建多个target，每个target包含自己的lun和主机，实现主机组和lun的绑定
如下图中定义了iscsi-disks和linuxha两个target，其中iscsi-disks中的lun0(block/ib_vol1)只允许iscsi-host访问，linuxha中的lun0(block/ha_vol1)只运行ha-host1访问，
o- iscsi ........................................................ [Targets: 2]
| o- iqn.2016-06.com.iscsi-disks:iscsi-disks ....................... [TPGs: 1]
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| | o- acls ...................................................... [ACLs: 1]
| | | o- iqn.2016-06.com.iscsi-host:iscsi-host ............ [Mapped LUNs: 1]
| | | o- mapped_lun0 ........................... [lun0 block/ib_vol1 (rw)]
| | o- luns ...................................................... [LUNs: 1]
| | | o- lun0 .............. [block/ib_vol1 (/dev/loop0) (default_tg_pt_gp)]
| | o- portals ................................................ [Portals: 1]
| | o- 0.0.0.0:3260 ................................................. [OK]
| o- iqn.2016-06.com.iscsi-disks:linuxha ........................... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
| o- acls ...................................................... [ACLs: 1]
| | o- iqn.2016-06.com.ha-host1:ha-host1 ................ [Mapped LUNs: 1]
| | o- mapped_lun0 ........................... [lun0 block/ha_vol1 (rw)]
| o- luns ...................................................... [LUNs: 1]
| | o- lun0 .............. [block/ha_vol1 (/dev/loop1) (default_tg_pt_gp)]
| o- portals ................................................ [Portals: 1]
| o- 0.0.0.0:3260 ................................................. [OK]
Step 5 放通防火墙
如果启用了防火墙，则需要放通以下端口：
# iptables -I INPUT 1 -p tcp --dport 3260 -j ACCEPT
# service iptables save #该命令需要安装iptables-services软件包
4. 让主机发现存储设备
Step 1 安装iSCSI initiator软件包
[root@iscsi-host ~]# yum -y install iscsi-initiator-utils
Step 2 修改配置文件/etc/iscsi/initiatorname.iscsi
在其中定义主机Initiator的名字：
InitiatorName=iqn.2016-06.com.iscsi-host:iscsi-host
此处定义的Initiator名字要和在存储节点中定义ACL时使用的主机名字一致。
Step 3 通过IP发现存储节点
[root@iscsi-host ~]# iscsiadm -m discovery -t sendtargets -p 192.168.56.20
192.168.56.20:3260,1 iqn.2016-06.com.iscsi-disks:iscsi-disks
[root@iscsi-host ~]# iscsiadm -m node -o show
# BEGIN RECORD 6.2.0.874-2
node.name = iqn.2016-06.com.iscsi-disks:iscsi-disks
node.tpgt = 1
node.startup = automatic
node.leading_login = No
...
# END RECORD
Step 4 从主机发起iscsi登陆
- 方法一：使用指令登陆
[root@iscsi-host ~]# iscsiadm -m node --login
Logging in to [iface: default, target: iqn.2016-06.com.iscsi-disks:iscsi-disks, portal: 192.168.56.20,3260] (multiple)
Login to [iface: default, target: iqn.2016-06.com.iscsi-disks:iscsi-disks, portal: 192.168.56.20,3260] successful.
[root@iscsi-host ~]# iscsiadm -m session -o show
tcp: [1] 192.168.56.20:3260,1 iqn.2016-06.com.iscsi-disks:iscsi-disks (non-flash)
如果之前发现了多个target，此处可用--targetname指定登陆的target。
- 方法二：iscsi.service会进行自动登陆，直接启动该服务即可
[root@iscsi-host ~]# systemctl start iscsi && systemctl enable iscsi
登陆后可使用fdisk发现新存储设备：
[root@iscsi-host ~]# fdisk -l
...
Disk /dev/sdb: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 126976 bytes
Step 5 使用新存储设备
新的存储设备可以当成普通硬盘使用，如创建文件系统并挂载到主机目录中：
[root@iscsi-host ~]# mkfs.xfs /dev/sdb
meta-data=/dev/sdb isize=512 agcount=4, agsize=65536 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=0, sparse=0
data = bsize=4096 blocks=262144, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
[root@iscsi-host ~]# mount /dev/sdb /mnt
[root@iscsi-host ~]# cd /mnt
[root@iscsi-host mnt]# ls
[root@iscsi-host mnt]# touch abcd
[root@iscsi-host mnt]# ls
abcd
5. iscsi服务和iscsid服务的关系
安装iscsi-initiator-utils后，系统新增了两个iscsi相关的服务，分别是iscsi.service和iscsid.service。
真正用于iscsi登陆的服务是iscsi.service，他启动时会同时启动iscsid.service，并且登陆完成后iscsi.service的进程将自动退出，由iscsid.service继续监控iscsi设备的状态。
[root@iscsi-host ~]# systemctl status iscsi
¡ñ iscsi.service - Login and scanning of iSCSI devices
Loaded: loaded (/usr/lib/systemd/system/iscsi.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2018-04-17 07:57:07 UTC; 3min 21s ago
Docs: man:iscsid(8)
man:iscsiadm(8)
Process: 1179 ExecStart=/sbin/iscsiadm -m node --loginall=automatic (code=exited, status=0/SUCCESS)
Process: 1176 ExecStart=/usr/libexec/iscsi-mark-root-nodes (code=exited, status=0/SUCCESS)
Main PID: 1179 (code=exited, status=0/SUCCESS)
[root@iscsi-host ~]# systemctl status iscsid
¡ñ iscsid.service - Open-iSCSI
Loaded: loaded (/usr/lib/systemd/system/iscsid.service; disabled; vendor preset: disabled)
Active: active (running) since Tue 2018-04-17 07:57:07 UTC; 36s ago
Docs: man:iscsid(8)
man:iscsiadm(8)
Process: 1168 ExecStop=/sbin/iscsiadm -k 0 2 (code=exited, status=0/SUCCESS)
Process: 1181 ExecStart=/usr/sbin/iscsid (code=exited, status=0/SUCCESS)
Main PID: 1183 (iscsid)
CGroup: /system.slice/iscsid.service
©À©¤1182 /usr/sbin/iscsid
©¸©¤1183 /usr/sbin/iscsid
Apr 17 07:57:07 iscsi-host systemd[1]: Starting Open-iSCSI...
Apr 17 07:57:07 iscsi-host systemd[1]: Failed to read PID from file /var/run/...nt
Apr 17 07:57:07 iscsi-host iscsid[1182]: iSCSI daemon with pid=1183 started!
Apr 17 07:57:07 iscsi-host systemd[1]: Started Open-iSCSI.
Apr 17 07:57:08 iscsi-host iscsid[1182]: Could not set session4 priority. REA...d.
Apr 17 07:57:08 iscsi-host iscsid[1182]: Connection4:0 to [target: iqn.2016-0...ow
Hint: Some lines were ellipsized, use -l to show in full.