Skip to content

Instantly share code, notes, and snippets.

@hiroyuki-sato
Last active July 6, 2022 02:59
Show Gist options
  • Save hiroyuki-sato/1be13ebf4872a98315f2 to your computer and use it in GitHub Desktop.
Save hiroyuki-sato/1be13ebf4872a98315f2 to your computer and use it in GitHub Desktop.
iSER setting on CentOS7

Links

References

environment

Confg value
OS CentOS7
Target 10.10.10.1
Initiator 10.10.10.2

target setting

  1. Run the targetcli command to enter the LIO CLI console:

execute target cli

# targetcli
/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 0]
  | o- fileio ............................................. [Storage Objects: 0]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 0]
  o- iscsi ........................................................ [Targets: 0]
  o- loopback ..................................................... [Targets: 0]
  o- srpt ......................................................... [Targets: 0]
  1. Create Ram Disk named test1:

create ram disk named test1

/> backstores/ramdisk/ create  name=test1 nullio=true size=100M
Created ramdisk test1 with size 100M.
  1. Create iscsi target:

create target

/> iscsi/ create wwn=iqn.2003-01.org.setup.lun.test
Created target iqn.2003-01.org.setup.lun.test.
Created TPG 1.
  1. check target

check target

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 0]
  | o- fileio ............................................. [Storage Objects: 0]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 1]
  |   o- test1 ................................. [nullio (100.0MiB) deactivated]
  o- iscsi ........................................................ [Targets: 1]
  | o- iqn.2003-01.org.setup.lun.test ................................ [TPGs: 1]
  |   o- tpg1 ........................................... [no-gen-acls, no-auth]
  |     o- acls ...................................................... [ACLs: 0]
  |     o- luns ...................................................... [LUNs: 0]
  |     o- portals ................................................ [Portals: 0]
  o- loopback ..................................................... [Targets: 0]
  o- srpt ......................................................... [Targets: 0]
  1. Create LUN.

create lun

/> iscsi/iqn.2003-01.org.setup.lun.test/tpg1/luns create /backstores/ramdisk/test1
Created LUN 0.

6, check LUN

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 0]
  | o- fileio ............................................. [Storage Objects: 0]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 1]
  |   o- test1 ................................... [nullio (100.0MiB) activated]
  o- iscsi ........................................................ [Targets: 1]
  | o- iqn.2003-01.org.setup.lun.test ................................ [TPGs: 1]
  |   o- tpg1 ........................................... [no-gen-acls, no-auth]
  |     o- acls ...................................................... [ACLs: 0]
  |     o- luns ...................................................... [LUNs: 1]
  |     | o- lun0 .............................................. [ramdisk/test1]
  |     o- portals ................................................ [Portals: 0]
  o- loopback ..................................................... [Targets: 0]
  o- srpt ......................................................... [Targets: 0]
/> 

6, Create portal. Make sure to supply the IP address of the interface you wish to run iSER on top (RoCE enabled)

/> iscsi/iqn.2003-01.org.setup.lun.test/tpg1/portals create 10.10.10.1  
Using default IP port 3260
Created network portal 10.10.10.1:3260.

7, Enable iSER

/> iscsi/iqn.2003-01.org.setup.lun.test/tpg1/portals/10.10.10.1:3260 enable_iser boolean=true
iSER enable now: True

8, Check iSER

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 0]
  | o- fileio ............................................. [Storage Objects: 0]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 1]
  |   o- test1 ................................... [nullio (100.0MiB) activated]
  o- iscsi ........................................................ [Targets: 1]
  | o- iqn.2003-01.org.setup.lun.test ................................ [TPGs: 1]
  |   o- tpg1 ........................................... [no-gen-acls, no-auth]
  |     o- acls ...................................................... [ACLs: 0]
  |     o- luns ...................................................... [LUNs: 1]
  |     | o- lun0 .............................................. [ramdisk/test1]
  |     o- portals ................................................ [Portals: 1]
  |       o- 10.10.10.1:3260 .......................................... [iser]
  o- loopback ..................................................... [Targets: 0]
  o- srpt ......................................................... [Targets: 0]

9, Allow access from any server (change ACL)

/> iscsi/iqn.2003-01.org.setup.lun.test/tpg1/ set attribute authentication=0 demo_mode_write_protect=0 generate_node_acls=1 cache_dynamic_acls=1
Parameter demo_mode_write_protect is now '0'.
Parameter authentication is now '0'.
Parameter generate_node_acls is now '1'.
Parameter cache_dynamic_acls is now '1'.

10,

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 0]
  | o- fileio ............................................. [Storage Objects: 0]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 1]
  |   o- test1 ................................... [nullio (100.0MiB) activated]
  o- iscsi ........................................................ [Targets: 1]
  | o- iqn.2003-01.org.setup.lun.test ................................ [TPGs: 1]
  |   o- tpg1 .............................................. [gen-acls, no-auth]
  |     o- acls ...................................................... [ACLs: 0]
  |     o- luns ...................................................... [LUNs: 1]
  |     | o- lun0 .............................................. [ramdisk/test1]
  |     o- portals ................................................ [Portals: 1]
  |       o- 10.10.10.1:3260 .......................................... [iser]
  o- loopback ..................................................... [Targets: 0]
  o- srpt ......................................................... [Targets: 0]
/> 

10, save

/> saveconfig 
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json

11, exit

/> exit
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json

12, Additinaol info

# /usr/sbin/getenforce 
Disabled

check firewalld status

# systemctl status firewalld       
firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled)
   Active: inactive (dead) since Thu 2014-12-25 13:16:50 JST; 50min ago
  Process: 1436 ExecStart=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 1436 (code=exited, status=0/SUCCESS)

Dec 25 12:47:42 centos7 systemd[1]: Starting firewalld - dynamic firewall d.....
Dec 25 12:47:42 centos7 systemd[1]: Started firewalld - dynamic firewall daemon.
Dec 25 13:16:49 centos7 systemd[1]: Stopping firewalld - dynamic firewall d.....
Dec 25 13:16:50 centos7 systemd[1]: Stopped firewalld - dynamic firewall daemon.
Hint: Some lines were ellipsized, use -l to show in full.

initiator setting

lsmod | grep iser
ib_iser                47000  0 
libiscsi               57278  1 ib_iser
scsi_transport_iscsi    99754  3 ib_iser,libiscsi
rdma_cm                43464  2 ib_iser,rdma_ucm
ib_core                87335  12 rdma_cm,ib_cm,ib_sa,iw_cm,mlx4_ib,ib_mad,ib_ucm,ib_iser,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib

ping test

# ping 10.10.10.1
PING 10.10.10.1 (10.10.10.1) 56(84) bytes of data.
64 bytes from 10.10.10.1: icmp_seq=1 ttl=64 time=0.299 ms
64 bytes from 10.10.10.1: icmp_seq=2 ttl=64 time=0.213 ms
64 bytes from 10.10.10.1: icmp_seq=3 ttl=64 time=0.196 ms
64 bytes from 10.10.10.1: icmp_seq=4 ttl=64 time=0.251 ms
64 bytes from 10.10.10.1: icmp_seq=5 ttl=64 time=0.232 ms
64 bytes from 10.10.10.1: icmp_seq=6 ttl=64 time=0.221 ms

iscsi discovery

# iscsiadm -m discovery -t st -p 10.10.10.1:3260
10.10.10.1:3260,1 iqn.2003-01.org.setup.lun.test
10.10.10.1:3260,1 iqn.2003-01.org.setup.lun.test

set mode to iser

# iscsiadm -m node -T iqn.2003-01.org.setup.lun.test -o update -n iface.transport_name -v iser
# 

iscsiadm

# iscsiadm -m node -l
Logging in to [iface: default, target: iqn.2003-01.org.setup.lun.test, portal: 10.10.10.1,3260] (multiple)
iscsiadm: Could not login to [iface: default, target: iqn.2003-01.org.setup.lun.test, portal: 10.10.10.1,3260].
iscsiadm: initiator reported error (8 - connection timed out)
iscsiadm: Could not log into all portals

dmesg

# dmesg
...
[  911.124432] iser: iser_conn_terminate:Failed to disconnect, conn: 0xffff881027a602b8 err -22
[  914.883417] iser: iser_conn_terminate:Failed to disconnect, conn: 0xffff881027a602b8 err -22
[  918.642238] iser: iser_conn_terminate:Failed to disconnect, conn: 0xffff881027a602b8 err -22
[  922.401246] iser: iser_conn_terminate:Failed to disconnect, conn: 0xffff881027a602b8 err -22

without iser

# iscsiadm -m node -l
Logging in to [iface: default, target: iqn.2003-01.org.setup.lun.test, portal: 10.10.10.1,3260] (multiple)
Login to [iface: default, target: iqn.2003-01.org.setup.lun.test, portal: 10.10.10.1,3260] successful.

Successful message example

[   13.567319] sd 0:1:0:0: Attached scsi generic sg0 type 0
[   13.567373] scsi 0:0:0:0: Attached scsi generic sg1 type 0
[   13.567408] scsi 0:0:1:0: Attached scsi generic sg2 type 0
[   13.619652] ip_tables: (C) 2000-2006 Netfilter Core Team
[   13.783032] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[   13.792985] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   13.818537] Ebtables v2.0 registered
[   13.828943] Bridge firewalling registered
[   13.947678] mlx4_en: enp3s0:   frag:0 - size:1526 prefix:0 align:0 stride:1536
[   14.038263] IPv6: ADDRCONF(NETDEV_UP): enp3s0: link is not ready
[   14.138212] IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready
[   14.237275] IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready
[   14.485342] mlx4_en: enp3s0: Link Up
[   14.485385] IPv6: ADDRCONF(NETDEV_CHANGE): enp3s0: link becomes ready
[   17.069999] igb: eno1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[   17.070154] IPv6: ADDRCONF(NETDEV_CHANGE): eno1: link becomes ready
[  116.586479] iscsi: registered transport (tcp)
[  140.377856] scsi8 : iSCSI Initiator over iSER
[  140.643454] scsi 8:0:0:0: Direct-Access     LIO-ORG  test1            4.0  PQ: 0 ANSI: 5
[  140.643542] scsi 8:0:0:0: alua: supports implicit and explicit TPGS
[  140.643729] scsi 8:0:0:0: alua: port group 00 rel port 01
[  140.643763] scsi 8:0:0:0: alua: port group 00 state A non-preferred supports TOlUSNA
[  140.643765] scsi 8:0:0:0: alua: Attached
[  140.643942] sd 8:0:0:0: [sdb] 204800 512-byte logical blocks: (104 MB/100 MiB)
[  140.643948] sd 8:0:0:0: Attached scsi generic sg3 type 0
[  140.644085] sd 8:0:0:0: [sdb] Write Protect is off
[  140.644089] sd 8:0:0:0: [sdb] Mode Sense: 43 00 00 08
[  140.644178] sd 8:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[  140.644908]  sdb: unknown partition table
[  140.645853] sd 8:0:0:0: [sdb] Attached SCSI disk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment