Skip to content

Instantly share code, notes, and snippets.

@mtholder
mtholder / gen_all_reversed.sh
Created December 31, 2019 17:42
script to populate a reversed_subproblems subdirectory in a synth tree build.
#!/bin/bash
inpdir=$1
if ! test -d "${inpdir}" ; then
echo "$0 must be passed in a directory of subproblems (with ott###.tre files)."
exit 1
fi
rpwd=`realpath .`
rpid=`realpath $inpdir`
if test "${rpwd}" = "${rpid}" ; then
echo "$0 Cannot be run in place."
@mtholder
mtholder / gen_reversed.py
Created December 31, 2019 17:41
simple script to reverse the first n-1 lines of a file with n lines.
#!/usr/bin/env python3
'''Reverses the order of the first n-1 lines of a file
and writes it to stdout.
'''
import sys
fp = sys.argv[1]
with open(fp, 'r', encoding='utf-8') as inp:
lines = inp.readlines()
last = lines.pop(-1)
lines.reverse()
@mtholder
mtholder / table_from_subproblem_size_summary.py
Created December 28, 2019 22:14
summary of subproblem_size_summary.json as a .tsv
#!/bin/env python
import json
import sys
import subprocess
import itertools
inpjson = 'subproblem_size_summary.json'
with open(inpjson, 'r', encoding='utf-8') as inp:
summary_blob = json.load(inp)
@mtholder
mtholder / generate-subproblem-size-summary.py
Created December 28, 2019 21:47
executed from the subproblems directory of an Open Tree synth tree to create subproblem_size_summary.json
#!/bin/env python
import json
import sys
import subprocess
import itertools
DEBUG = False
def count_tips_and_inf_splits(fn):
out = subprocess.check_output(['otc-degree-distribution', fn], stderr=subprocess.DEVNULL).decode('utf-8')
@mtholder
mtholder / create_num_tips_for_ott_internals_in_labelled_tree_json.bash
Created December 28, 2019 21:40
generating a mapping of internal OTT ID that shows up in the synth tree to the number of descendant leaves
#!/bin/bash
otc-tree-tool --indented-table labelled_supertree/labelled_supertree_simplified_ottnames.tre > subproblems/indented-table.txt
cat subproblems/indented-table.txt | sed -E '/ +mrca/d' | sed -E 's/.* ott([0-9]+) : ([0-9]+)/"ott\1": \2,/' > num_tips_for_ott_internals_in_labelled_tree.json
rm subproblems/indented-table.txt
@mtholder
mtholder / generate-subproblem-scaffold.bash
Created December 28, 2019 19:20
generates subproblems/subproblems-scaffold.tre and subproblems/subproblems-scaffold-only.tre for an Open Tree synth. tree build
#!/bin/bash
# This needs to be made part of the propinquity process.
# MTH ran steps that made up the basis of this before the
# Jan 2020 SSB workshop. So the resulting file will be in
# the https://files.opentreeoflife.org/synthesis/opentree12.3/opentree12.3/subproblems/
# folder, but (perversely enough) not in the tar archive at https://files.opentreeoflife.org/synthesis/opentree12.3/opentree12.3.tgz
# If all goes as planned, we'll start adding this to the tree build process so that it will be in
# builds and archives made after 12.3
# This is to be run with the working directory being the propinquity
@mtholder
mtholder / suppress-non-listed-ids-or-unnamed.py
Created December 28, 2019 19:14
suppresses nodes from an OT tree if they do not have an ID listed in a separate file. Takes newick-tree-filepath and file-with-ids-one-per-line as args
#!/bin/env python3
import sys
import dendropy
import re
id_extractor = re.compile('.*ott([0-9]+)$')
tree_filepath = sys.argv[1]
id_filepath = sys.argv[2]
tree = dendropy.Tree.get(path=tree_filepath, schema='newick', suppress_internal_node_taxa=False)
# Get the list of IDs to retain
@mtholder
mtholder / ott_ids_for_nexson.sh
Created December 12, 2019 00:08
takes a path to an Open Tree NexSON. writes a element of python dict mapping the filename to a frozenset of mapped OTT Ids
#!/bin/bash
fp=$1
fn=$(basename $fp)
if ! test -f $fp ; then
echo "$fp does not exist"
exit 1
fi
o=`grep '"^ot:ottId": ' "$fp" | sed 's/".*": //' | sort -g | sed ':a;N;$!ba;s/\n/ /g' | sed -E 's/ +/ /g'`
echo "\"${fn}\": frozenset([ ${o} ]),"
@mtholder
mtholder / construct_c_trie.py
Created June 7, 2019 22:55
start of generating a C-trie in python
#!/usr/bin/env python3
# -*- coding: <encoding name> -*-
from __future__ import print_function
from collections import defaultdict
from math import log
import codecs
from ctypes import c_uint64 as c_trie_node_type
import sys
""" Pseudocode from Maly, 1976 Appendix B
@mtholder
mtholder / crawl.py
Last active October 19, 2018 20:40 — forked from bredelings/crawl.py
Silly web crawler
#!/usr/bin/python3
import requests
import time
from threading import Thread
machine='http://localhost:1984'
# machine='https://ot39.opentreeoflife.org'
# machine='https://api.opentreeoflife.org'