iSCSI使用TCP/IP協(xié)議對存儲使用的SCSI指令進(jìn)行封裝，使得可通過TCP/IP網(wǎng)絡(luò)訪問持久化存儲。本文在CentOS7上對iSCSI的配置和使用進(jìn)行介紹。
1. 實驗環(huán)境
-
虛擬化軟件：
-
實驗虛機(jī)：
- iscsi-disks: 192.168.56.20 （iSCSI target，提供存儲設(shè)備），默認(rèn)配置1個cpu，1G內(nèi)存。
- iscsi-host: 192.168.56.21 （iSCSI initiator，訪問iSCSI設(shè)備的主機(jī)），默認(rèn)配置1個cpu，1G內(nèi)存。
安裝和管理網(wǎng)絡(luò)：192.168.56.0/24，該網(wǎng)絡(luò)為VirtualBox的Host-Only網(wǎng)絡(luò)，支持物理機(jī)和VirtualBox虛機(jī)間的互相訪問。
2. 克隆項目并啟動上述虛擬機(jī)
本文中的實驗僅涉及一個主機(jī)節(jié)點和一個存儲節(jié)點。
$ git clone https://github.com/lprincewhn/iscsi.git
$ cd iscsi
$ vagrant up
虛擬機(jī)啟動完畢后可使用以下用戶登陸：
- root/vagrant
- vagrant/vagrant
3. 創(chuàng)建虛擬磁盤
使用root用戶登陸iscsi-disks，使用losetup來創(chuàng)建實驗用的存儲設(shè)備。
Step 1 創(chuàng)建大文件用于支持虛擬存儲設(shè)備
[root@iscsi-disks ~]# dd if=/dev/zero of=/lun1.img bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 2.29056 s, 469 MB/s
為了避免Selinux權(quán)限問題，建議不要將文件創(chuàng)建在/root目錄下。
Step 2 創(chuàng)建loop設(shè)備
[root@iscsi-disks ~]# losetup -f
/dev/loop0
[root@iscsi-disks ~]# losetup /dev/loop0 /lun1.img
[root@iscsi-disks ~]# losetup
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE
/dev/loop0 0 0 0 0 /lun1.img
Step 3 安裝iSCSI target軟件包并啟動服務(wù)
[root@iscsi-disks ~]# yum -y install targetcli
[root@iscsi-disks ~]# systemctl enable target && systemctl start target
Step 4 在targetcli命令行中創(chuàng)建存儲設(shè)備
[root@iscsi-disks ~]# targetcli
Warning: Could not load preferences file /root/.targetcli/prefs.bin.
targetcli shell version 2.1.fb46
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
/> /backstores/block create ib_vol1 /dev/loop0 # 創(chuàng)建Block
Created block storage object ib_vol1 using /dev/loop0.
/> /iscsi create iqn.2016-06.com.iscsi-disks:iscsi-disks # 定義存儲節(jié)點
Created target iqn.2016-06.com.iscsi-disks:iscsi-disks.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
/> /iscsi/iqn.2016-06.com.iscsi-disks:iscsi-disks/tpg1/acls create iqn.2016-06.com.iscsi-host:iscsi-host # 將訪問主機(jī)加入設(shè)備ACL，這樣主機(jī)才能訪問到這個設(shè)備
Created Node ACL for iqn.2016-06.com.iscsi-host:iscsi-host
/> /iscsi/iqn.2016-06.com.iscsi-disks:iscsi-disks/tpg1/luns create /backstores/block/ib_vol1 # 使用Block定義存儲節(jié)點上的lun
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2016-06.com.iscsi-host:iscsi-host
/> ls # 查看上述創(chuàng)建的內(nèi)容及關(guān)系
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 1]
| | o- ib_vol1 .................... [/dev/loop0 (1.0GiB) write-thru activated]
| | o- alua ............................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ................... [ALUA state: Active/optimized]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 1]
| o- iqn.2016-06.com.iscsi-disks:iscsi-disks ....................... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
| o- acls ...................................................... [ACLs: 1]
| | o- iqn.2016-06.com.iscsi-host:iscsi-host ............ [Mapped LUNs: 1]
| | o- mapped_lun0 ........................... [lun0 block/ib_vol1 (rw)]
| o- luns ...................................................... [LUNs: 1]
| | o- lun0 .............. [block/ib_vol1 (/dev/loop0) (default_tg_pt_gp)]
| o- portals ................................................ [Portals: 1]
| o- 0.0.0.0:3260 ................................................. [OK]
o- loopback ..................................................... [Targets: 0]
注：同一臺主機(jī)可以創(chuàng)建多個target，每個target包含自己的lun和主機(jī)，實現(xiàn)主機(jī)組和lun的綁定
如下圖中定義了iscsi-disks和linuxha兩個target，其中iscsi-disks中的lun0(block/ib_vol1)只允許iscsi-host訪問，linuxha中的lun0(block/ha_vol1)只運(yùn)行ha-host1訪問，
o- iscsi ........................................................ [Targets: 2]
| o- iqn.2016-06.com.iscsi-disks:iscsi-disks ....................... [TPGs: 1]
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| | o- acls ...................................................... [ACLs: 1]
| | | o- iqn.2016-06.com.iscsi-host:iscsi-host ............ [Mapped LUNs: 1]
| | | o- mapped_lun0 ........................... [lun0 block/ib_vol1 (rw)]
| | o- luns ...................................................... [LUNs: 1]
| | | o- lun0 .............. [block/ib_vol1 (/dev/loop0) (default_tg_pt_gp)]
| | o- portals ................................................ [Portals: 1]
| | o- 0.0.0.0:3260 ................................................. [OK]
| o- iqn.2016-06.com.iscsi-disks:linuxha ........................... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
| o- acls ...................................................... [ACLs: 1]
| | o- iqn.2016-06.com.ha-host1:ha-host1 ................ [Mapped LUNs: 1]
| | o- mapped_lun0 ........................... [lun0 block/ha_vol1 (rw)]
| o- luns ...................................................... [LUNs: 1]
| | o- lun0 .............. [block/ha_vol1 (/dev/loop1) (default_tg_pt_gp)]
| o- portals ................................................ [Portals: 1]
| o- 0.0.0.0:3260 ................................................. [OK]
Step 5 放通防火墻
如果啟用了防火墻，則需要放通以下端口：
# iptables -I INPUT 1 -p tcp --dport 3260 -j ACCEPT
# service iptables save #該命令需要安裝iptables-services軟件包
4. 讓主機(jī)發(fā)現(xiàn)存儲設(shè)備
Step 1 安裝iSCSI initiator軟件包
[root@iscsi-host ~]# yum -y install iscsi-initiator-utils
Step 2 修改配置文件/etc/iscsi/initiatorname.iscsi
在其中定義主機(jī)Initiator的名字：
InitiatorName=iqn.2016-06.com.iscsi-host:iscsi-host
此處定義的Initiator名字要和在存儲節(jié)點中定義ACL時使用的主機(jī)名字一致。
Step 3 通過IP發(fā)現(xiàn)存儲節(jié)點
[root@iscsi-host ~]# iscsiadm -m discovery -t sendtargets -p 192.168.56.20
192.168.56.20:3260,1 iqn.2016-06.com.iscsi-disks:iscsi-disks
[root@iscsi-host ~]# iscsiadm -m node -o show
# BEGIN RECORD 6.2.0.874-2
node.name = iqn.2016-06.com.iscsi-disks:iscsi-disks
node.tpgt = 1
node.startup = automatic
node.leading_login = No
...
# END RECORD
Step 4 從主機(jī)發(fā)起iscsi登陸
- 方法一：使用指令登陸
[root@iscsi-host ~]# iscsiadm -m node --login
Logging in to [iface: default, target: iqn.2016-06.com.iscsi-disks:iscsi-disks, portal: 192.168.56.20,3260] (multiple)
Login to [iface: default, target: iqn.2016-06.com.iscsi-disks:iscsi-disks, portal: 192.168.56.20,3260] successful.
[root@iscsi-host ~]# iscsiadm -m session -o show
tcp: [1] 192.168.56.20:3260,1 iqn.2016-06.com.iscsi-disks:iscsi-disks (non-flash)
如果之前發(fā)現(xiàn)了多個target，此處可用--targetname指定登陸的target。
- 方法二：iscsi.service會進(jìn)行自動登陸，直接啟動該服務(wù)即可
[root@iscsi-host ~]# systemctl start iscsi && systemctl enable iscsi
登陸后可使用fdisk發(fā)現(xiàn)新存儲設(shè)備：
[root@iscsi-host ~]# fdisk -l
...
Disk /dev/sdb: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 126976 bytes
Step 5 使用新存儲設(shè)備
新的存儲設(shè)備可以當(dāng)成普通硬盤使用，如創(chuàng)建文件系統(tǒng)并掛載到主機(jī)目錄中：
[root@iscsi-host ~]# mkfs.xfs /dev/sdb
meta-data=/dev/sdb isize=512 agcount=4, agsize=65536 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=0, sparse=0
data = bsize=4096 blocks=262144, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
[root@iscsi-host ~]# mount /dev/sdb /mnt
[root@iscsi-host ~]# cd /mnt
[root@iscsi-host mnt]# ls
[root@iscsi-host mnt]# touch abcd
[root@iscsi-host mnt]# ls
abcd
5. iscsi服務(wù)和iscsid服務(wù)的關(guān)系
安裝iscsi-initiator-utils后，系統(tǒng)新增了兩個iscsi相關(guān)的服務(wù)，分別是iscsi.service和iscsid.service。
真正用于iscsi登陸的服務(wù)是iscsi.service，他啟動時會同時啟動iscsid.service，并且登陸完成后iscsi.service的進(jìn)程將自動退出，由iscsid.service繼續(xù)監(jiān)控iscsi設(shè)備的狀態(tài)。
[root@iscsi-host ~]# systemctl status iscsi
?? iscsi.service - Login and scanning of iSCSI devices
Loaded: loaded (/usr/lib/systemd/system/iscsi.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2018-04-17 07:57:07 UTC; 3min 21s ago
Docs: man:iscsid(8)
man:iscsiadm(8)
Process: 1179 ExecStart=/sbin/iscsiadm -m node --loginall=automatic (code=exited, status=0/SUCCESS)
Process: 1176 ExecStart=/usr/libexec/iscsi-mark-root-nodes (code=exited, status=0/SUCCESS)
Main PID: 1179 (code=exited, status=0/SUCCESS)
[root@iscsi-host ~]# systemctl status iscsid
?? iscsid.service - Open-iSCSI
Loaded: loaded (/usr/lib/systemd/system/iscsid.service; disabled; vendor preset: disabled)
Active: active (running) since Tue 2018-04-17 07:57:07 UTC; 36s ago
Docs: man:iscsid(8)
man:iscsiadm(8)
Process: 1168 ExecStop=/sbin/iscsiadm -k 0 2 (code=exited, status=0/SUCCESS)
Process: 1181 ExecStart=/usr/sbin/iscsid (code=exited, status=0/SUCCESS)
Main PID: 1183 (iscsid)
CGroup: /system.slice/iscsid.service
?à?¤1182 /usr/sbin/iscsid
???¤1183 /usr/sbin/iscsid
Apr 17 07:57:07 iscsi-host systemd[1]: Starting Open-iSCSI...
Apr 17 07:57:07 iscsi-host systemd[1]: Failed to read PID from file /var/run/...nt
Apr 17 07:57:07 iscsi-host iscsid[1182]: iSCSI daemon with pid=1183 started!
Apr 17 07:57:07 iscsi-host systemd[1]: Started Open-iSCSI.
Apr 17 07:57:08 iscsi-host iscsid[1182]: Could not set session4 priority. REA...d.
Apr 17 07:57:08 iscsi-host iscsid[1182]: Connection4:0 to [target: iqn.2016-0...ow
Hint: Some lines were ellipsized, use -l to show in full.
