Skip to content

Instantly share code, notes, and snippets.

Avatar

Daniel King danking

View GitHub Profile
View foo.py
import hail as hl
hl.nd.array([1, 2, 3, 4]).reshape((2, 2)).show()
# FIXME: use ndarray sum / fma
def block_product(left, right):
product = left @ right
n_rows, n_cols = product.shape
return hl.struct(
shape=product.shape,
View lambda.md

I realized I started using lambda without explaining what that is. My apologies.

In math class you probably defined functions by writing something like the following:

f(x) = x * 2
View foo.py
import hail as hl
# Download a small part of the One Thousand Genomes dataset
hl.utils.get_1kg('data/')
# Read the dataset as a Hail MatrixTable, a tool for exploring two-dimensional data.
mt = hl.read_matrix_table('data/1kg.mt')
# Show some of the data in a pleasant visual form
mt.show()
View shuffler-ir-design.md

Shuffler IR Design

UUID is probably a string

ShuffleStart(
  keyFields Array<String>,
  rowType (virtual) Struct,
  TypedCodecSpec codecAndEType
): id UUID
View co-op-resume-filtering.md

Co-op Resume Filtering

Background

In recent semesters, Hail has received >300 applications for our one open co-op position. We engage in an uncomfortable, severe filtering process to cope with this overwhelming number of applications.

For the Spring 2020 position, Hail received 316 applications. For the upcoming Fall 2020 position, Hail has thus far received 162 applications.

View poor person's hail pipeline.sh
set -ex
cat > foo.sh <<'EOF'
#!/bin/bash
set -ex
cores=$1
script_name=$2
free -ws 5 | grep --line-buffered Mem > free.log &
@danking
danking / README.md
Last active Oct 1, 2018 — forked from ce-carey/README.md
Hail PRS code
View README.md

PRS Scoring of the UK Biobank Using Hail

The run_prs.py script slack_utils.py was created to generate polygenic risk scores (PRS) in the UK Biobank (UKB) using Hail 0.2. Currently, I have hard-coded the file locations in-line with my own file structure and that of the Neale Lab UKB Round 2 GWAS.

NOTE: This is under active development and generally intended for my own use and the use of my collaborators. ADAPT AT YOUR OWN PERIL!!

How to Run

  1. This script assumes that you have generated an input file using my process-sumstats and LDPred repos.
View gist:47d5d996ef0e8337e895828df07c592c
{"action": "submitted", "review": {"id": 159193269, "node_id": "MDE3OlB1bGxSZXF1ZXN0UmV2aWV3MTU5MTkzMjY5", "user": {"login": "hail-ci-test-2", "id": 43152863, "node_id": "MDQ6VXNlcjQzMTUyODYz", "avatar_url": "https://avatars0.githubusercontent.com/u/43152863?v=4", "gravatar_id": "", "url": "https://api.github.com/users/hail-ci-test-2", "html_url": "https://github.com/hail-ci-test-2", "followers_url": "https://api.github.com/users/hail-ci-test-2/followers", "following_url": "https://api.github.com/users/hail-ci-test-2/following{/other_user}", "gists_url": "https://api.github.com/users/hail-ci-test-2/gists{/gist_id}", "starred_url": "https://api.github.com/users/hail-ci-test-2/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/users/hail-ci-test-2/subscriptions", "organizations_url": "https://api.github.com/users/hail-ci-test-2/orgs", "repos_url": "https://api.github.com/users/hail-ci-test-2/repos", "events_url": "https://api.github.com/users/hail-ci-test-2/events{/privacy}", "received_events
View run_prs_holdout.py
import hail as hl
import pickle
import time
import argparse
def flip_text(base):
"""
:param StringExpression base: Expression of a single base
:return: StringExpression of flipped base
:rtype: StringExpression
View gist:fa160f9ecf137359ea108fefa5230619
This file has been truncated, but you can view the full file.
2018-08-22 16:55:21 Hail: WARN: This Hail JAR was compiled for Spark 2.2.0, running with Spark 2.2.1.
Compatibility is not guaranteed.
2018-08-22 16:55:21 SparkContext: INFO: Running Spark version 2.2.1
2018-08-22 16:55:21 SparkContext: INFO: Submitted application: Hail
2018-08-22 16:55:21 SparkContext: INFO: Spark configuration:
spark.app.name=Hail
spark.driver.extraClassPath=./hail-devel-62679fc21687-Spark-2.2.0.jar
spark.driver.extraJavaOptions=-Xss4M -Dlog4j.configuration=file:///tmp/819ef837c3604c95828b61032b37a28e/driver_log4j.properties
spark.driver.maxResultSize=0
You can’t perform that action at this time.