Instantly share code, notes, and snippets.

Embed
What would you like to do?
How to setup galaxy with TORQUE in Amazon's cloud
#!/usr/bin/python
import sys
'''
Just a simple tool that adds the line number at
the end of each line
'''
with open(sys.argv[1]) as f_in, open(sys.argv[2], 'w') as f_out:
c = 0
for l in f_in:
c += 1
f_out.write(l.replace('\n', '') + '\t%i'%c + '\n')
<tool id="custom_tool_1" name="Custom Tool 1" version="0.1.0">
<description>Experimental tool 1</description>
<command interpreter="python">example_tool.py $input $output</command>
<inputs>
<param format="tabular" name="input" type="data" label="Source file"/>
</inputs>
<outputs>
<data format="tabular" name="output" />
</outputs>
<tests>
<test>
<param name="input" value="test_input.txt"/>
<output name="output" file="test_output.txt"/>
</test>
</tests>
<help>
The best tool ever.
</help>
</tool>

The location of the files

  • example_tool.py tools/mytools
  • example_tool.xml tools/mytools
  • my_tool_conf.xml config/
  • my_job_conf.xml config/
<?xml version="1.0"?>
<!-- A sample job config that explicitly configures job running the way it is configured by default (if there is no explicit config). -->
<job_conf>
<plugins>
<plugin id="local"
type="runner"
load="galaxy.jobs.runners.local:LocalJobRunner"
workers="4"/>
<plugin id="torque1" type="runner"
load="galaxy.jobs.runners.pbs:PBSJobRunner"
workers="2"/>
</plugins>
<handlers>
<handler id="main"/>
</handlers>
<destinations default="local">
<destination id="local" runner="local"/>
<destination id="torque1_dst" runner="torque1"/>
</destinations>
<tools>
<tool id="custom_tool_1" destination="torque1_dst"/>
</tools>
</job_conf>
<section name="MyTools" id="mTools">
<tool file="mytools/example_tool.xml" />
</section>
a b c
e f g
h i j
@kantale

This comment has been minimized.

Owner

kantale commented Mar 4, 2016

This guide assumes that you have setup and ssh-ed in an EC2 instance. The instance that I used was "Ubuntu Server 14.04 LTS (HVM), SSD Volume Type - ami-f95ef58a". I would love to see some comments.

Get packages:

sudo apt-get update
sudo apt-get -y install g++ make libssl-dev libxml2-dev libboost-dev git 

Install TORQUE

mkdir torque
cd torque

wget http://www.adaptivecomputing.com/index.php?wpfb_dl=2984 
mv "index.php?wpfb_dl=2984" torque-5.1.2-1448394813_f498aba.tar.gz 
tar zxvf torque-5.1.2-1448394813_f498aba.tar.gz 
cd torque-5.1.2-1448394813_f498aba/
./configure
make
sudo make install

Add to /etc/hosts

<IP> <HOSTNAME>

For example:

172.30.0.172 ip-172-30-0-172

Continue:

sudo echo "/usr/local/lib" > ld.so.conf 
sudo mv ld.so.conf  /etc/ld.so.conf 
sudo ldconfig 
sudo ./torque.setup root 

This should look like:

initializing TORQUE (admin: root@ip-172-30-0-172)

You have selected to start pbs_server in create mode.
If the server database exists it will be overwritten.
do you wish to continue y/(n)?y
root     29986     1  0 13:11 ?        00:00:00 pbs_server -t create
Max open servers: 9
Max open servers: 9

continue with:

sudo killall pbs_server 

Add HOSTNAME to

/var/spool/torque/server_name
/var/spool/torque/server_priv/nodes  
/var/spool/torque/server_priv/acl_svr/acl_hosts
/var/spool/torque/mom_priv/config 

Add root@HOSTNAME to:

/var/spool/torque/server_priv/acl_svr/operators
/var/spool/torque/server_priv/acl_svr/managers

Start everything:

sudo pbs_server 
sudo pbs_sched 
sudo pbs_mom 

Make sure this this command:

sudo pbsnodes -a

Produces something like this:

ip-172-30-0-172
     state = free
     power_state = Running
     np = 1
     ntype = cluster
     status = rectime=1457098876,macaddr=02:dd:30:22:59:4b,cpuclock=Fixed,varattr=,jobs=,state=free,netload=79993541,gres=ip:-172-30-0-172,loadave=0.00,ncpus=1,physmem=2048516kb,availmem=1870768kb,totmem=2048516kb,idletime=3461,nusers=1,nsessions=1,sessions=1327,uname=Linux ip-172-30-0-172 3.13.0-74-generic #118-Ubuntu SMP Thu Dec 17 22:52:10 UTC 2015 x86_64,opsys=linux
     mom_service_port = 15002
     mom_manager_port = 15003

Make a test job

cd
mkdir test
cd test

Make the file: test.sh

echo "YEAH" > /home/ubuntu/test/result.txt

Submit it:

qsub test.sh

Make sure that everything went ok:

ubuntu@ip-172-30-0-172:~/test$ qstat
Job ID                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
0.ip-172-30-0-172          test.sh          ubuntu          00:00:00 C batch          
ubuntu@ip-172-30-0-172:~/test$ ls
result.txt  test.sh  test.sh.e0  test.sh.o0
ubuntu@ip-172-30-0-172:~/test$ cat result.txt 
YEAH

Install pbs-python

cd
wget https://bootstrap.pypa.io/get-pip.py 
sudo python get-pip.py  
git clone https://github.com/radik/pbs-python 
cd pbs-python 
sudo python setup.py install 

Install galaxy

cd
git clone https://github.com/galaxyproject/galaxy/ 
cd galaxy
mv config/galaxy.ini.sample config/galaxy.ini

edit config/galaxy.ini and add: (MAKE SURE YOU ARE MAKING THIS CHANGE IN THE [server:main] SECTION!!)

host = 0.0.0.0

Make sure that it runs fine:

sh run.sh

Now it is a nice time to:

  • Create an account
  • Upload the file test_input.txt (Get_data --> upload file)

stop server with control-c
Let's add a tool: (Assuming you have a local copy of these files in ../)

mkdir tools/mytools 
cp ../example_tool.py   tools/mytools/example_tool.py
cp ../example_tool.xml tools/mytools/example_tool.xml
cp ../my_tool_conf.xml config/my_tool_conf.xml

Edit config/galaxy.ini and add the line:

tool_config_file = config/tool_conf.xml.main,config/my_tool_conf.xml 

Rerun server:

sh run.sh

You should be able to see the tool at the end of the tool bar:
tprczjt - imgur
Make a workflow to test the tool installation
y70sghm - imgur
Save it and run, giving as input the file test.txt
fldnnqp - imgur
The results should look like this:
n2ts0zr - imgur

Configure my_tool to run via TORQUE

Stop the serve (ctrl-c)

cp ../my_job_conf.xml config/my_job_conf.xml

Edit galaxy/config.ini and add the following lines:

job_config_file = config/my_job_conf.xml
cluster_files_directory = database/pbs

Create the cluster_files_directory

mkdir database/pbs

Rerun the server and run the same workflow again (exactly as before)
With qstat you should be able to confirm that the workflow was submitted:

ubuntu@ip-172-30-0-172:~$ qstat
Job ID                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
1.ip-172-30-0-172          ...@ics.forth.gr ubuntu          00:00:01 C batch          

Also the resulted file should be the same as before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment