Skip to content

Instantly share code, notes, and snippets.

View clbarnes's full-sized avatar

Chris Barnes clbarnes

View GitHub Profile
@clbarnes
clbarnes / HCMST_ver_3.04.csv
Created October 15, 2019 00:55
How Couples Meet and Stay Together dataset, in CSV
We can't make this file beautiful and searchable because it's too large.
,caseid_new,weight1,weight2,ppage,ppagecat,ppagect4,ppeduc,ppeducat,ppethm,ppgender,pphhhead,pphouseholdsize,pphouse,ppincimp,hhinc,ppmarit,ppmsacat,ppreg4,ppreg9,pprent,ppt01,ppt1317,ppt18ov,ppt25,ppt612,children_in_hh,ppwork,ppnet,ppq14arace,pphispan,pprace_white,pprace_black,pprace_nativeamerican,pprace_asianindian,pprace_chinese,pprace_filipino,pprace_japanese,pprace_korean,pprace_vietnamese,pprace_otherasian,pprace_hawaiian,pprace_guamanian,pprace_samoan,pprace_otherpacificislander,pprace_someotherrace,papglb_friend,pppartyid3,papevangelical,papreligion,ppppcmdate_yrmo,pppadate_yrmo,pphhcomp11_member2_age,pphhcomp11_member3_age,pphhcomp11_member4_age,pphhcomp11_member5_age,pphhcomp11_member6_age,pphhcomp11_member7_age,pphhcomp11_member8_age,pphhcomp11_member9_age,pphhcomp11_member10_age,pphhcomp11_member11_age,pphhcomp11_member12_age,pphhcomp11_member13_age,pphhcomp11_member14_age,pphhcomp11_member15_age,pphhcomp11_member2_gender,pphhcomp11_member3_gender,pphhcomp11_member4_gender,pphhcomp11_member5_gend
@clbarnes
clbarnes / classname.py
Created July 29, 2019 15:43
Get the fully-qualified class name of a python object
def classname(obj):
cls = type(obj)
module = cls.__module__
name = cls.__qualname__
if module is not None and module != "__builtin__":
name = module + "." + name
return name
@clbarnes
clbarnes / zellij
Created March 10, 2024 21:58
Wrapper script for zellij
#!/bin/bash
# Script which uses the cached zellij if it exists;
# otherwise downloads a fresh copy.
# Deletes the cached copy every month.
set -euo pipefail
launchpath="/tmp/zellij/launch"
timepath="/tmp/zellij/last_updated"
@clbarnes
clbarnes / neuron_install.md
Last active February 29, 2024 13:11
These instructions are for installing NEURON and NetPyNE with linux and a conda environment.

Install instructions

These instructions are for installing NEURON and NetPyNE with linux and a conda environment.

Create a neuron directory in your home directory, put the gzipped source files in it, and expand them.

cd $HOME
mkdir neuron
mv iv-mm.tar.gz neuron
@clbarnes
clbarnes / httpdir.py
Created January 17, 2024 19:03
Given a manifest file, copy a directory tree from a URL base to a local directory
#!/usr/bin/env python3
"""
Copy a directory tree of files over HTTPS.
If HTTP basic auth is required, use an environment variable like
`HTTP_BASIC_AUTH="myuser:mypassword"`.
"""
import os
import sys
from urllib.request import Request, urlopen
from base64 import b64encode
@clbarnes
clbarnes / makepool.py
Last active January 3, 2024 11:06
Make a zpool command for creating large pools
#!/usr/bin/env python3
"""
Make a `zpool create ...` command for a set of identical disks,
calculating vdev size etc. from the given configuration.
You will need to know the disk model name;
you should be able to identify it from the output of
`lsblk -d -e 7 -o NAME,SIZE,STATE,MODEL`.
"""
import getpass
from argparse import ArgumentParser
@clbarnes
clbarnes / euler_scraper.py
Created November 3, 2023 13:41
Scrape Project Euler and create stub python scripts for each (ancient code, transferred from old account)
#!/usr/bin/python
from bs4 import BeautifulSoup
import urllib
path = '/home/tunisia/Desktop/Project Euler/'
for probnum in range(1,444):
html = BeautifulSoup(urllib.urlopen('http://projecteuler.net/problem=%d' % probnum))
title = html.h2
@clbarnes
clbarnes / rosalind_scraper.py
Last active November 3, 2023 13:41
Scrape all of the project rosalind problems and create python stubs for each (obsolete code transferred from an old account)
#!/usr/bin/python2
from bs4 import BeautifulSoup
import urllib
path = '/home/tunisia/Projects/rosalind/'
html = BeautifulSoup(urllib.urlopen('http://rosalind.info/problems/list-view/'))
tr_all = html.find_all('tr')
@clbarnes
clbarnes / browser_pool.py
Created August 18, 2023 18:45
Async client for fetching HTML content from pages requiring javascript execution, using a pool of tabs
from contextlib import asynccontextmanager
import asyncio as aio
from playwright.async_api import async_playwright
class BrowserPool:
def __init__(self, n_tabs=10, executable=None) -> None:
self.executable = executable
self.n_tabs = n_tabs
self.tabs_remaining = n_tabs
@clbarnes
clbarnes / aio_ratelimits.py
Created August 9, 2023 11:08
Semaphore-based asyncio rate limiters
import asyncio as aio
from collections import deque
from typing import Awaitable, Iterable, TypeVar
T = TypeVar("T")
class BaseLimit:
async def limit(self, awa: Awaitable[T]) -> T: