LiorZ LiorZ

## instructions.md

      
              2 files
            
          
              6 forks
            
          
              1 comment
            
          
              17 stars
            
          
                asmateus
                / instructions.md
            
            
              Last active
              July 21, 2024 16:16
            
              
                Simple Slurm configuration in Debian based systems
              
          
    Slurm Configuration Debian based Cluster

Here I will describe a simple configuration of the slurm management tool for launching jobs in a really simplistic cluster. I will assume the following configuration: a main node (for me it is an Arch Linux distribution) and 3 compute nodes (for me compute nodes are Debian VMs). I also assume there is ping access between the nodes and some sort of mechanism for you to know the IP of each node at all times (most basic should be a local NAT with static IPs)
Basic Structure

Slurm management tool work on a set of nodes, one of which is considered the master node, and has the slurmctld daemon running; all other compute nodes have the slurmd daemon. All communications are authenticated via the munge service and all nodes need to share the same authentication key. Slurm by default holds a journal of activities in a directory configured in the slurm.conf file, however a Database management system can be set. All in all what we will try to do is:

Install `munge


## x_means.py
"""
以下の論文で提案された改良x-means法の実装

クラスター数を自動決定するk-meansアルゴリズムの拡張について
http://www.rd.dnc.ac.jp/~tunenori/doc/xmeans_euc.pdf
"""

import numpy as np
from scipy import stats
from sklearn.cluster import KMeans

## sound_recorder.py
import pyaudio
import wave

FORMAT = pyaudio.paInt16
CHANNELS = 2
RATE = 44100
CHUNK = 1024
RECORD_SECONDS = 5
WAVE_OUTPUT_FILENAME = "file.wav"


## apt.py
from starcluster.clustersetup import ClusterSetup
from starcluster.logger import log

class addRepository(ClusterSetup):
     def __init__(self, repositories=None):
          self.repositories=repositories
          if self.repositories:
              self.repositories= [repo.strip() for repo in repositories.split(',')]


## gist:727625
from Bio import Entrez

def fetch_gene_coordinates(search_term):
    handle = Entrez.esearch(db="gene", term=search_term)
    rec = Entrez.read(handle)
    gene_id = rec["IdList"][0] # assuming best match works

    handle = Entrez.efetch(db="gene", id=gene_id, retmode="xml")
    rec = Entrez.read(handle)[0]
    gene_locus = rec["Entrezgene_locus"][0]
	"""
	以下の論文で提案された改良x-means法の実装

	クラスター数を自動決定するk-meansアルゴリズムの拡張について
	http://www.rd.dnc.ac.jp/~tunenori/doc/xmeans_euc.pdf
	"""

	import numpy as np
	from scipy import stats
	from sklearn.cluster import KMeans
	import pyaudio
	import wave

	FORMAT = pyaudio.paInt16
	CHANNELS = 2
	RATE = 44100
	CHUNK = 1024
	RECORD_SECONDS = 5
	WAVE_OUTPUT_FILENAME = "file.wav"
	from starcluster.clustersetup import ClusterSetup
	from starcluster.logger import log

	class addRepository(ClusterSetup):
	def __init__(self, repositories=None):
	self.repositories=repositories
	if self.repositories:
	self.repositories= [repo.strip() for repo in repositories.split(',')]
	from Bio import Entrez

	def fetch_gene_coordinates(search_term):
	handle = Entrez.esearch(db="gene", term=search_term)
	rec = Entrez.read(handle)
	gene_id = rec["IdList"][0] # assuming best match works

	handle = Entrez.efetch(db="gene", id=gene_id, retmode="xml")
	rec = Entrez.read(handle)[0]
	gene_locus = rec["Entrezgene_locus"][0]