References
- HowTo Configure LIO enabled with ISER for RHEL7 Inbox Driver
- HowTo setup RoCE connection using Inbox driver (RHEL, Ubuntu)
| Confg | value |
|---|---|
| OS | CentOS7 |
| Target | 10.10.10.1 |
| Initiator | 10.10.10.2 |
- Run the targetcli command to enter the LIO CLI console:
execute target cli
# targetcli
/> ls
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 0]
o- loopback ..................................................... [Targets: 0]
o- srpt ......................................................... [Targets: 0]
- Create Ram Disk named test1:
create ram disk named test1
/> backstores/ramdisk/ create name=test1 nullio=true size=100M
Created ramdisk test1 with size 100M.
- Create iscsi target:
create target
/> iscsi/ create wwn=iqn.2003-01.org.setup.lun.test
Created target iqn.2003-01.org.setup.lun.test.
Created TPG 1.
- check target
check target
/> ls
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 1]
| o- test1 ................................. [nullio (100.0MiB) deactivated]
o- iscsi ........................................................ [Targets: 1]
| o- iqn.2003-01.org.setup.lun.test ................................ [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
| o- acls ...................................................... [ACLs: 0]
| o- luns ...................................................... [LUNs: 0]
| o- portals ................................................ [Portals: 0]
o- loopback ..................................................... [Targets: 0]
o- srpt ......................................................... [Targets: 0]
- Create LUN.
create lun
/> iscsi/iqn.2003-01.org.setup.lun.test/tpg1/luns create /backstores/ramdisk/test1
Created LUN 0.
6, check LUN
/> ls
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 1]
| o- test1 ................................... [nullio (100.0MiB) activated]
o- iscsi ........................................................ [Targets: 1]
| o- iqn.2003-01.org.setup.lun.test ................................ [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
| o- acls ...................................................... [ACLs: 0]
| o- luns ...................................................... [LUNs: 1]
| | o- lun0 .............................................. [ramdisk/test1]
| o- portals ................................................ [Portals: 0]
o- loopback ..................................................... [Targets: 0]
o- srpt ......................................................... [Targets: 0]
/>
6, Create portal. Make sure to supply the IP address of the interface you wish to run iSER on top (RoCE enabled)
/> iscsi/iqn.2003-01.org.setup.lun.test/tpg1/portals create 10.10.10.1
Using default IP port 3260
Created network portal 10.10.10.1:3260.
7, Enable iSER
/> iscsi/iqn.2003-01.org.setup.lun.test/tpg1/portals/10.10.10.1:3260 enable_iser boolean=true
iSER enable now: True
8, Check iSER
/> ls
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 1]
| o- test1 ................................... [nullio (100.0MiB) activated]
o- iscsi ........................................................ [Targets: 1]
| o- iqn.2003-01.org.setup.lun.test ................................ [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
| o- acls ...................................................... [ACLs: 0]
| o- luns ...................................................... [LUNs: 1]
| | o- lun0 .............................................. [ramdisk/test1]
| o- portals ................................................ [Portals: 1]
| o- 10.10.10.1:3260 .......................................... [iser]
o- loopback ..................................................... [Targets: 0]
o- srpt ......................................................... [Targets: 0]
9, Allow access from any server (change ACL)
/> iscsi/iqn.2003-01.org.setup.lun.test/tpg1/ set attribute authentication=0 demo_mode_write_protect=0 generate_node_acls=1 cache_dynamic_acls=1
Parameter demo_mode_write_protect is now '0'.
Parameter authentication is now '0'.
Parameter generate_node_acls is now '1'.
Parameter cache_dynamic_acls is now '1'.
10,
/> ls
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 1]
| o- test1 ................................... [nullio (100.0MiB) activated]
o- iscsi ........................................................ [Targets: 1]
| o- iqn.2003-01.org.setup.lun.test ................................ [TPGs: 1]
| o- tpg1 .............................................. [gen-acls, no-auth]
| o- acls ...................................................... [ACLs: 0]
| o- luns ...................................................... [LUNs: 1]
| | o- lun0 .............................................. [ramdisk/test1]
| o- portals ................................................ [Portals: 1]
| o- 10.10.10.1:3260 .......................................... [iser]
o- loopback ..................................................... [Targets: 0]
o- srpt ......................................................... [Targets: 0]
/>
10, save
/> saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json
11, exit
/> exit
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json
12, Additinaol info
# /usr/sbin/getenforce
Disabled
check firewalld status
# systemctl status firewalld
firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled)
Active: inactive (dead) since Thu 2014-12-25 13:16:50 JST; 50min ago
Process: 1436 ExecStart=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS (code=exited, status=0/SUCCESS)
Main PID: 1436 (code=exited, status=0/SUCCESS)
Dec 25 12:47:42 centos7 systemd[1]: Starting firewalld - dynamic firewall d.....
Dec 25 12:47:42 centos7 systemd[1]: Started firewalld - dynamic firewall daemon.
Dec 25 13:16:49 centos7 systemd[1]: Stopping firewalld - dynamic firewall d.....
Dec 25 13:16:50 centos7 systemd[1]: Stopped firewalld - dynamic firewall daemon.
Hint: Some lines were ellipsized, use -l to show in full.
lsmod | grep iser
ib_iser 47000 0
libiscsi 57278 1 ib_iser
scsi_transport_iscsi 99754 3 ib_iser,libiscsi
rdma_cm 43464 2 ib_iser,rdma_ucm
ib_core 87335 12 rdma_cm,ib_cm,ib_sa,iw_cm,mlx4_ib,ib_mad,ib_ucm,ib_iser,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib
ping test
# ping 10.10.10.1
PING 10.10.10.1 (10.10.10.1) 56(84) bytes of data.
64 bytes from 10.10.10.1: icmp_seq=1 ttl=64 time=0.299 ms
64 bytes from 10.10.10.1: icmp_seq=2 ttl=64 time=0.213 ms
64 bytes from 10.10.10.1: icmp_seq=3 ttl=64 time=0.196 ms
64 bytes from 10.10.10.1: icmp_seq=4 ttl=64 time=0.251 ms
64 bytes from 10.10.10.1: icmp_seq=5 ttl=64 time=0.232 ms
64 bytes from 10.10.10.1: icmp_seq=6 ttl=64 time=0.221 ms
iscsi discovery
# iscsiadm -m discovery -t st -p 10.10.10.1:3260
10.10.10.1:3260,1 iqn.2003-01.org.setup.lun.test
10.10.10.1:3260,1 iqn.2003-01.org.setup.lun.test
set mode to iser
# iscsiadm -m node -T iqn.2003-01.org.setup.lun.test -o update -n iface.transport_name -v iser
#
iscsiadm
# iscsiadm -m node -l
Logging in to [iface: default, target: iqn.2003-01.org.setup.lun.test, portal: 10.10.10.1,3260] (multiple)
iscsiadm: Could not login to [iface: default, target: iqn.2003-01.org.setup.lun.test, portal: 10.10.10.1,3260].
iscsiadm: initiator reported error (8 - connection timed out)
iscsiadm: Could not log into all portals
dmesg
# dmesg
...
[ 911.124432] iser: iser_conn_terminate:Failed to disconnect, conn: 0xffff881027a602b8 err -22
[ 914.883417] iser: iser_conn_terminate:Failed to disconnect, conn: 0xffff881027a602b8 err -22
[ 918.642238] iser: iser_conn_terminate:Failed to disconnect, conn: 0xffff881027a602b8 err -22
[ 922.401246] iser: iser_conn_terminate:Failed to disconnect, conn: 0xffff881027a602b8 err -22
# iscsiadm -m node -l
Logging in to [iface: default, target: iqn.2003-01.org.setup.lun.test, portal: 10.10.10.1,3260] (multiple)
Login to [iface: default, target: iqn.2003-01.org.setup.lun.test, portal: 10.10.10.1,3260] successful.
Successful message example
[ 13.567319] sd 0:1:0:0: Attached scsi generic sg0 type 0
[ 13.567373] scsi 0:0:0:0: Attached scsi generic sg1 type 0
[ 13.567408] scsi 0:0:1:0: Attached scsi generic sg2 type 0
[ 13.619652] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 13.783032] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[ 13.792985] ip6_tables: (C) 2000-2006 Netfilter Core Team
[ 13.818537] Ebtables v2.0 registered
[ 13.828943] Bridge firewalling registered
[ 13.947678] mlx4_en: enp3s0: frag:0 - size:1526 prefix:0 align:0 stride:1536
[ 14.038263] IPv6: ADDRCONF(NETDEV_UP): enp3s0: link is not ready
[ 14.138212] IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready
[ 14.237275] IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready
[ 14.485342] mlx4_en: enp3s0: Link Up
[ 14.485385] IPv6: ADDRCONF(NETDEV_CHANGE): enp3s0: link becomes ready
[ 17.069999] igb: eno1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[ 17.070154] IPv6: ADDRCONF(NETDEV_CHANGE): eno1: link becomes ready
[ 116.586479] iscsi: registered transport (tcp)
[ 140.377856] scsi8 : iSCSI Initiator over iSER
[ 140.643454] scsi 8:0:0:0: Direct-Access LIO-ORG test1 4.0 PQ: 0 ANSI: 5
[ 140.643542] scsi 8:0:0:0: alua: supports implicit and explicit TPGS
[ 140.643729] scsi 8:0:0:0: alua: port group 00 rel port 01
[ 140.643763] scsi 8:0:0:0: alua: port group 00 state A non-preferred supports TOlUSNA
[ 140.643765] scsi 8:0:0:0: alua: Attached
[ 140.643942] sd 8:0:0:0: [sdb] 204800 512-byte logical blocks: (104 MB/100 MiB)
[ 140.643948] sd 8:0:0:0: Attached scsi generic sg3 type 0
[ 140.644085] sd 8:0:0:0: [sdb] Write Protect is off
[ 140.644089] sd 8:0:0:0: [sdb] Mode Sense: 43 00 00 08
[ 140.644178] sd 8:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 140.644908] sdb: unknown partition table
[ 140.645853] sd 8:0:0:0: [sdb] Attached SCSI disk