The following instructions take you through setting up a SSH Key on O2. (Official instructions for generating SSH key)
The senario behind this is that in CyCIF pipeline, it is designed to first submit a job to transfer files from /n/files, get the job ID, and the following steps in the pipeline are dependent on the transfer job.
And the SSH Key allows you to transfer files from /n/files using the transfer server with ssh rsync
, without being prompted for password. So that you can put ssh rsync
from n/files to /n/scratch2 into an sbatch script. And the job could hang until the transfer is complete, then it will run the rest of the script.
On O2 login node
$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/your_ecommons/.ssh/id_rsa):
Hit Enter ⏎
Enter passphrase (empty for no passphrase):
Hit Enter ⏎
Enter same passphrase again:
Hit Enter ⏎
Your identification has been saved in /home/your_ecommons/.ssh/id_rsa.
Your public key has been saved in /home/your_ecommons/.ssh/id_rsa.pub.
The key fingerprint is:
a5:b5:38:73:b7:3c:a6:8a:1d:a8:bd:87:4e:be:33:21 your_ecommons@login01
The key's randomart image is:
The following command copies the public key to your authorized_keys file:
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
On O2 login node
$ chmod 0600 ~/.ssh/authorized_keys
$ chmod 0700 ~/.ssh
$ chmod 0711 /home/$USER
On O2 login node
$ srun -p interactive --pty -t 1:00:00 -n 2 bash
srun: job 50830472 queued and waiting for resources
srun: job 50830472 has been allocated resources
$ rsync -Pav transfer:/n/files/ImStor/sorger/data/RareCyte/yc296/YC-20180711-bleached-for-15-min/ /n/scratch2/yc296/transfer_test/
created directory /n/scratch2/your_ecommons/transfer_test
./
CSS08_A-bleach-light-new-S100-MCM6-p57-LSP2-Scan_20180711_144802_01x4x00090.rcpnl
995391488 100% 95.99MB/s 0:00:09 (xfer#1, to-check=5/7)
CSS08_B-bleach-light-new-S100-MCM6-p57-LSP2-Scan_20180711_144203_01x4x00110.rcpnl
1216587776 100% 90.51MB/s 0:00:12 (xfer#2, to-check=4/7)
CSS08_C-bleach-light-old-S100-MCM6-p57-LSP1-Scan_20180711_143205_01x4x00100.rcpnl
1105989632 100% 90.95MB/s 0:00:11 (xfer#3, to-check=3/7)
Scan_20180711_143205_01x4x00100.metadata
8344 100% 13.34kB/s 0:00:00 (xfer#4, to-check=2/7)
Scan_20180711_144203_01x4x00110.metadata
9024 100% 14.35kB/s 0:00:00 (xfer#5, to-check=1/7)
Scan_20180711_144802_01x4x00090.metadata
7636 100% 12.11kB/s 0:00:00 (xfer#6, to-check=0/7)
sent 128 bytes received 3318399628 bytes 99056709.13 bytes/sec
total size is 3317993900 speedup is 1.00