iRods data transfers.md

i-Drops file transfer

Log in to hpc via terminal (replace username with your username)

shh username@login.cx1.hpc.ic.ac.uk

Fill in your password when prompted Now we will go to the project folder and find a place we want to dump the files

pwd

will give you your present working directory i.e. your current location. Response should be /rds/general/user/username/home

cd ..

cd change directory .. backs you up one folder. So you are now in your own username space which will contain your Home directory, ephemeral directory and projects directory. You can pwd anywhere to check you are in the right place or ls (List) to check what files are in the location you are in.

cd projects

ls

LS means list so will list everything in your projects directory. You should see ukdrmultiomicsproject

cd ukdrmultiomicsproject

ls

Now you should see everything listed in our project folder. Go into live

cd live

ls

Now you can see everything in the live folder. Go into seqdata.

cd seqdata

This is where we will transfer the data to so from here we need to create an irods environment

mkdir -p ~/.irods

touch ~/.irods/irods_environment.json

this creates your environment then you can open the file and edit it using

vi ~/.irods/irods_environment.json

If you've not used vi before, you might think the terminal has gone wierd. It hasn't. You've entered a text editor. Hit "i" to enter insert mode. Then paste the following into the file and change the “YOUR_IGF_USERNAME” to the username of the person who submitted the seq data i.e. Amy or Karen (I’ll send over these details and passwords separately

{
      "irods_host": "eliot.med.ic.ac.uk",
      "irods_port":1247,
      "irods_default_resource": "woolfResc",
      "irods_user_name": "YOUR_IGF_USERNAME",
      "irods_zone_name": "igfZone"
}

You can close and save the file by hitting esc and typing :wq

Now you should be back in terminal and ready to start downloading

module load irods/4.2.0

iinit

It will ask for your password this is the password for the IGF username i.e. not your personal password but the password assigned to the person who submitted the sequencing run. Once you have entered this you will be in that persons irods server account

ils

will list all data associated with that account

ipwd

will tell you where you are in the irods server next type

screen

this opens a new screen and means that if your connection to the server gets interrupted the download will continue. To start the download type

iget -Pr /igfZone/home/USERNAME/PROJECT_NAME/*

Where username is the irods account username and project_name is the name the project was submitted under (i.e. one of the files you see listed when you do an ils command) PATH is

If you only require cell ranger files (analysis) or fastq files you can get them individually instead using the following but beware if you do this they will be dumped without project identifiers so be sure to rename the file with a project ID

iget -Pr /igfZone/home/USERNAME/PROJECT_NAME/analysis
iget -Pr /igfZone/home/USERNAME/PROJECT_NAME/fastq