Skip to content

Instantly share code, notes, and snippets.

@ckandoth
ckandoth / single_machine_slurm_on_ubuntu.md
Last active May 19, 2024 07:49
Install Slurm 19.05 on a standalone machine running Ubuntu 20.04

Use apt to install the necessary packages:

sudo apt install -y slurm-wlm slurm-wlm-doc

Load file:///usr/share/doc/slurm-wlm/html/configurator.html in a browser (or file://wsl%24/Ubuntu/usr/share/doc/slurm-wlm/html/configurator.html on WSL2), and:

  1. Set your machine's hostname in SlurmctldHost and NodeName.
  2. Set CPUs as appropriate, and optionally Sockets, CoresPerSocket, and ThreadsPerCore. Use command lscpu to find what you have.
  3. Set RealMemory to the number of megabytes you want to allocate to Slurm jobs,
  4. Set StateSaveLocation to /var/spool/slurm-llnl.
  5. Set ProctrackType to linuxproc because processes are less likely to escape Slurm control on a single machine config.
@ssaamm
ssaamm / scraper.py
Last active December 20, 2022 18:48
Hacking course registration with Twilio and Python
import time
import requests
from bs4 import BeautifulSoup
from twilio.rest import Client
import redis
from secrets import twilio_account_sid, token, my_number
client = Client(twilio_account_sid, token)
@hisplan
hisplan / cwltoil-help.txt
Last active February 3, 2018 00:35
cwltoil --help
usage: cwltoil [-h] [--logOff] [--logCritical] [--logError] [--logWarning]
[--logInfo] [--logDebug] [--logLevel LOGLEVEL]
[--logFile LOGFILE] [--rotatingLogging] [--workDir WORKDIR]
[--stats] [--clean {always,onError,never,onSuccess}]
[--cleanWorkDir {always,never,onSuccess,onError}]
[--clusterStats [CLUSTERSTATS]] [--restart]
[--batchSystem BATCHSYSTEM] [--disableHotDeployment]
[--scale SCALE] [--mesosMaster MESOSMASTERADDRESS]
[--parasolCommand PARASOLCOMMAND]
[--parasolMaxBatches PARASOLMAXBATCHES] [--provisioner {aws}]
@alaingoldman
alaingoldman / Drawer.jsx
Last active January 11, 2017 17:30
Class Notes compendium
import DrawerLayout from 'react-native-drawer-layout';
export default class Inside extends Component {
openSideBar(){
this.refs['DRAWER'].openDrawer();
}
render() {
var navigationView =
@ckandoth
ckandoth / ensembl_vep_86_with_offline_cache.md
Last active July 15, 2021 16:26
Install Ensembl's VEP v86 with various caches for running offline

Ensembl's VEP (Variant Effect Predictor) is popular for how it picks a single effect per gene as detailed here, its CLIA-compliant HGVS variant format, and Sequence Ontology nomenclature for variant effects.

To follow these instructions, we'll assume you have these packaged essentials installed:

## For Debian/Ubuntu system admins ##
sudo apt-get install -y build-essential git libncurses-dev

## For RHEL/CentOS system admins ##
sudo yum groupinstall -y 'Development Tools'
sudo yum install -y git ncurses-devel
@acdlite
acdlite / app.js
Last active January 20, 2023 08:23
Quick and dirty code splitting with React Router v4
// getComponent is a function that returns a promise for a component
// It will not be called until the first mount
function asyncComponent(getComponent) {
return class AsyncComponent extends React.Component {
static Component = null;
state = { Component: AsyncComponent.Component };
componentWillMount() {
if (!this.state.Component) {
getComponent().then(Component => {
@nvictus
nvictus / runlength.py
Last active October 7, 2023 19:54
NumPy run-length encoding / decoding
"""Run Length Encoding utilities for NumPy arrays.
Authors
-------
- Nezar Abdennur
- Anton Goloborodko
"""
from __future__ import division, print_function
import numpy as np
@josh-padnick
josh-padnick / download.sh
Last active May 8, 2024 12:37
Download a private binary release file from GitHub in bash
#!/usr/bin/env bash
#
# This is an adaptation of code I wrote to download a private binary from GitHub. Such...pain.
# Why can't GitHub just offer a standardized URL you can download a release binary from and attach
# your Github Personal Access Token as a header?
#
# Since this code is an adaptation it hasn't been directly tested, but the code it was adapted from works
# and hopefully you can get the missing piece you're after by looking here.
#
@vanvaridiksha
vanvaridiksha / csds-spark.md
Last active July 3, 2017 15:51
Spark Assignment 3

#Spark Assignment 3

This assignment is based on the 3rd chapter of the book - 'Advanced Big Data Analytics with Spark'. You can find a copy of the book here.

In this assignment, you will be required to build a recommendation system using Spark and MLib using a dataset published by AudioScrobbler. This data is 500MB uncompressed and can be downloaded here.

You can glance through the chapter and understand how to build the recommendation system. The chapter will walk you through Scala code;

@vasanthk
vasanthk / System Design.md
Last active July 23, 2024 06:27
System Design Cheatsheet

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

  1. Clarify and agree on the scope of the system
  • User cases (description of sequences of events that, taken together, lead to a system doing something useful)
    • Who is going to use it?
    • How are they going to use it?