Skip to content

Instantly share code, notes, and snippets.

View jdidion's full-sized avatar

John Didion jdidion

View GitHub Profile
@jdidion
jdidion / multiprocessing-test.py
Created February 29, 2016 21:13
A test of two different ways to implement a multiprocessing architecture for the problem "Read lines from a file, split them into chunks, have workers process the chunks in parallel, and have a writer thread write the results to an output file."
# Example of two different structures for multiprocessing code.
# The first uses queues to move chunks of data between the reader
# thread (which is the main process), the worker threads (which
# do the actual computation), and the writer thread (which writes
# the results to a file). The second using shared memory Values
# and Arrays: the reader thread reads data directly into shared
# memory char arrays, and each worker process the data in an array
# and writes the result back to that array; the writer thread
# copies the data from the finished char arrays to the output file.
#
<?xml version="1.0" encoding="utf-8"?>
<style xmlns="http://purl.org/net/xbiblio/csl" class="in-text" version="1.0" demote-non-dropping-particle="sort-only" default-locale="en-US">
<info>
<title>National Library of Medicine (grant proposals with PMCID/PMID) - Author, Year</title>
<title-short>NLMAuthorYear</title-short>
<id>http://www.zotero.org/styles/national-library-of-medicine-grant-proposals</id>
<link href="http://www.zotero.org/styles/national-library-of-medicine-grant-proposals" rel="self"/>
<link href="http://www.nlm.nih.gov/pubs/formats/recommendedformats1991-full.pdf" rel="documentation"/>
<link href="http://publicaccess.nih.gov/citation_methods.htm" rel="documentation"/>
<link href="http://grants.nih.gov/grants/funding/424/SF424_RR_Guide_General_Adobe_VerC.pdf" rel="documentation"/>
<?xml version="1.0" encoding="utf-8"?>
<style xmlns="http://purl.org/net/xbiblio/csl" class="in-text" version="1.0" demote-non-dropping-particle="sort-only" default-locale="en-US">
<info>
<title>National Library of Medicine (grant proposals with PMCID/PMID) - Author, Year</title>
<title-short>NLMAuthorYear</title-short>
<id>http://www.zotero.org/styles/national-library-of-medicine-grant-proposals</id>
<link href="http://www.zotero.org/styles/national-library-of-medicine-grant-proposals" rel="self"/>
<link href="http://www.nlm.nih.gov/pubs/formats/recommendedformats1991-full.pdf" rel="documentation"/>
<link href="http://publicaccess.nih.gov/citation_methods.htm" rel="documentation"/>
<link href="http://grants.nih.gov/grants/funding/424/SF424_RR_Guide_General_Adobe_VerC.pdf" rel="documentation"/>
# This list was boostrapped from the FastQC contaminants.txt.
# TODO: add missing Illumina adapters from official list at:
# http://support.illumina.com/downloads/illumina-customer-sequence-letter.html
>IlluminaSingleEndAdapter1
GATCGGAAGAGCTCGTATGCCGTCTTCTGCTTG
>IlluminaSingleEndAdapter2
CAAGCAGAAGACGGCATACGAGCTCTTCCGATCT
>IlluminaSingleEndPCRPrimer1
AATGATACGGCGACCACCGAGATCTACACTCTTTCCCTACACGACGCTCTTCCGATCT
@jdidion
jdidion / linux-one-liners.md
Created June 27, 2018 22:49
Linux one-liners

Interrogate a running command

strace -e 'trace=!read,write' -p <pid>
@jdidion
jdidion / upset_simple.R
Last active March 22, 2019 20:59
Create an UpSet plot given the counts for each intersection
# Create an UpSet plot given the counts for each intersection.
#
# The input is a data frame with N + 1 columns, where N is the
# number of sets, and M rows, where M is the number of intersections
# between the sets. The values in columns 1:N are binary values
# indicating which sets are involved in each intersection. The
# N+1 column must have the name "count" and holds the count values
# for the intersections. You can generate a template counts data
# frame using the create_counts_df() function.
#
@jdidion
jdidion / intervals.py
Created July 17, 2019 16:04
Interval management and searching, based on @brentp's InterLap code
from enum import IntFlag
import functools
import itertools
import operator
from ngsindex.utils import DefaultDict
from typing import (
Generic, Iterable, Iterator, Optional, Sequence, Sized, Tuple, Type, TypeVar, Union,
cast
)
@jdidion
jdidion / gist:2301c72b790e9e7b044b6ac759ea8529
Created October 20, 2020 18:12
Java CWL parsing error
```cwl
#!/usr/bin/env cwl-runner
cwlVersion: v1.2
class: CommandLineTool
inputs:
- id: reference
type: string
{
"class": "CommandLineTool",
"inputs": [
{
"type": {
"type": "record",
"fields": [
{
"type": "File",
"format": "http://example.com/format1",