Oliver Pelz OliPelz

## extract-ssh-keys.sh
#!/bin/bash
set -e

mkdir -p ~/.ssh
chmod 700 ~/.ssh

echo -n Enter password to decrypt keepass file:
read -s keepass_password

# load keepass db for the first time so we can set a password

## setup-passhole.sh
#!/bin/bash

mkdir ~/.venv
python3 -m venv ~/.venv
source ~/.venv/bin/activate
echo 'source ~/.venv/bin/activate' >> ~/.zshrc
pip3 install --upgrade pip
pip3 install passhole

## init.sh
# set location to fastest german mirror
sudo pacman-mirrors -c Germany
# update complete system
sudo pacman -Syyu --noconfirm
# check if restart is needed after updating
s1=$(pacman -Q linux | sed 's/linux[0-9]* //')
s2=$(uname -r | sed 's/-[A-Z]*$//')
if [ "$s1" == "$s2" ]; then
  # NO REBOOT needed
  # install essential dependencies to provision with ansiblet

## gist:a9eba6f150fa712944e9c0a174b7ca75
# https://serverfault.com/questions/576910/how-to-find-full-process-arguments-and-associated-listening-ports

while : ; do ss -nptu | awk 'NR>1 { split($7,p,","); split(p[2],pid,"="); printf "Listen: "$5 " Command: "; system("ps --no-headers -ww -o args p "pid[2]); }'; done   | grep curl

## Readme.md

      
              1 file
            
          
              0 forks
            
          
              4 comments
            
          
              0 stars
            
          
                OliPelz
                / Readme.md
            
            
              Last active
              November 20, 2019 19:08
            
              
                Install SikuliX on Fedora 25
              
          
    Notes on Installing SikuliX on fedora 25

dnf groupinstall -y "Development Tools" 
dnf install -y cmake gcc git pkgconfig ffjpeg g++ gtk+-devel libjpeg-devel libtiff-devel jasper-devel libpng-devel zlib-devel libicu-devel unzip numpy python-devel autoconf automake libtool pango pango-devel cairo cairo-devel

now compile and install opencv2 because the opencv3 which comes with fedora 25 does not work with SikuliX

mkdir ~/opencv-2.4.13; cd $_
wget http://downloads.sourceforge.net/project/opencvlibrary/opencv-unix/2.4.13/opencv-2.4.13.zip
unzip opencv-2.4.13.zip


## 1_How to create bigger - huge Docker images (>100gb) in CentOS 7.x (using Docker 1.12.5)
The default Docker daemon on CentOS 7.x from the official package repository has two very annoying limitations when trying to create bigger images.

First is the default base image is 10gb space and second there is another barrier with the thin layer which is set to 100gb.

I needed a Docker image containing a big database which was about 120gb.

When I COPY'ed the huge database in the Dockerfile to build the image, I got a

No space left on device

## 1 How to pass dynamic values such as username to a galaxy tool using dynamic_options
Tried a lot of different approaches to pass the username or user id to a galaxy tool configuration, nothing seemed to work,
e.g. the __user_id__ or user_name could not be passed to a function referenced by dynamic_options, e.g.:

<param name="mytool" type="select" dynamic_options="mystuff(userid=__user_id__)" />
 or
 <param name="mytool" type="select" dynamic_options="mystuff(username=__user_name__)" />

 did not work, neither other approaches such as username='$__user_name__' or ${__user_name__} or '${__user_name__}'

 it seems that the reserved variables from this document here cannot be accessed by a dynamic_options method:

## 1 how_to_use_torque_openpbs_in_python_subprocess
Just a small note, here is how you can convert commandline calls in python to use openpbs/torque
If the command is doing something in the background without any dependency to the main script thats easy,
just use qsub in your subprocess call and let if fork off.

But what you do if you have some subprocess calls which wait for a program or script to finish (it depends on it)?
After some Internet research I found qsub's parameter "-I -x"  to be exactly what I needed!

lets say you have the following command fragment:

cmd = 'lastdb -Q 0 %s %s' % (last_index, validated_ref)  # format the database

## How to make MinoTour clusterable with openpbs - torque
this gist contains all the script modifications which are needed
for minoTour's most computational intense components mT_control and minUP
to use openpbs/torque and run on the computenodes instead of constantly killing our webserver

i made this in a quickhack/hardcoded changes fashion


files to modify :

to make minUP alignments clusterable

## minoTour mt_control.pl systemd startup script
# log in as root, create log file destination
$ touch /var/log/mT_control.log
$ useradd minion
$ passwd minion
# check out minoTour repo into some dir e.g. /opt
$ cd /opt
$ git clone https://github.com/minoTour/minoTour.git
# change permissions to system user minion so we can run it as systemd user process
$ chown minion:minion /opt/minoTour/nefario -R
	#!/bin/bash
	set -e

	mkdir -p ~/.ssh
	chmod 700 ~/.ssh

	echo -n Enter password to decrypt keepass file:
	read -s keepass_password

	# load keepass db for the first time so we can set a password
	#!/bin/bash

	mkdir ~/.venv
	python3 -m venv ~/.venv
	source ~/.venv/bin/activate
	echo 'source ~/.venv/bin/activate' >> ~/.zshrc
	pip3 install --upgrade pip
	pip3 install passhole
	# set location to fastest german mirror
	sudo pacman-mirrors -c Germany
	# update complete system
	sudo pacman -Syyu --noconfirm
	# check if restart is needed after updating
	s1=$(pacman -Q linux \| sed 's/linux[0-9]* //')
	s2=$(uname -r \| sed 's/-[A-Z]*$//')
	if [ "$s1" == "$s2" ]; then
	# NO REBOOT needed
	# install essential dependencies to provision with ansiblet
	# https://serverfault.com/questions/576910/how-to-find-full-process-arguments-and-associated-listening-ports

	while : ; do ss -nptu \| awk 'NR>1 { split($7,p,","); split(p[2],pid,"="); printf "Listen: "$5 " Command: "; system("ps --no-headers -ww -o args p "pid[2]); }'; done \| grep curl
	The default Docker daemon on CentOS 7.x from the official package repository has two very annoying limitations when trying to create bigger images.

	First is the default base image is 10gb space and second there is another barrier with the thin layer which is set to 100gb.

	I needed a Docker image containing a big database which was about 120gb.

	When I COPY'ed the huge database in the Dockerfile to build the image, I got a

	No space left on device
	Tried a lot of different approaches to pass the username or user id to a galaxy tool configuration, nothing seemed to work,
	e.g. the __user_id__ or user_name could not be passed to a function referenced by dynamic_options, e.g.:

	<param name="mytool" type="select" dynamic_options="mystuff(userid=__user_id__)" />
	or
	<param name="mytool" type="select" dynamic_options="mystuff(username=__user_name__)" />

	did not work, neither other approaches such as username='$__user_name__' or ${__user_name__} or '${__user_name__}'

	it seems that the reserved variables from this document here cannot be accessed by a dynamic_options method:
	Just a small note, here is how you can convert commandline calls in python to use openpbs/torque
	If the command is doing something in the background without any dependency to the main script thats easy,
	just use qsub in your subprocess call and let if fork off.

	But what you do if you have some subprocess calls which wait for a program or script to finish (it depends on it)?
	After some Internet research I found qsub's parameter "-I -x" to be exactly what I needed!

	lets say you have the following command fragment:

	cmd = 'lastdb -Q 0 %s %s' % (last_index, validated_ref) # format the database
	this gist contains all the script modifications which are needed
	for minoTour's most computational intense components mT_control and minUP
	to use openpbs/torque and run on the computenodes instead of constantly killing our webserver

	i made this in a quickhack/hardcoded changes fashion


	files to modify :

	to make minUP alignments clusterable
	# log in as root, create log file destination
	$ touch /var/log/mT_control.log
	$ useradd minion
	$ passwd minion
	# check out minoTour repo into some dir e.g. /opt
	$ cd /opt
	$ git clone https://github.com/minoTour/minoTour.git
	# change permissions to system user minion so we can run it as systemd user process
	$ chown minion:minion /opt/minoTour/nefario -R