Skip to content

Instantly share code, notes, and snippets.


Daniel King danking

View GitHub Profile
import hail as hl
hl.nd.array([1, 2, 3, 4]).reshape((2, 2)).show()
# FIXME: use ndarray sum / fma
def block_product(left, right):
product = left @ right
n_rows, n_cols = product.shape
return hl.struct(

I realized I started using lambda without explaining what that is. My apologies.

In math class you probably defined functions by writing something like the following:

f(x) = x * 2
import hail as hl
# Download a small part of the One Thousand Genomes dataset
# Read the dataset as a Hail MatrixTable, a tool for exploring two-dimensional data.
mt = hl.read_matrix_table('data/')
# Show some of the data in a pleasant visual form

Shuffler IR Design

UUID is probably a string

  keyFields Array<String>,
  rowType (virtual) Struct,
  TypedCodecSpec codecAndEType
): id UUID

Co-op Resume Filtering


In recent semesters, Hail has received >300 applications for our one open co-op position. We engage in an uncomfortable, severe filtering process to cope with this overwhelming number of applications.

For the Spring 2020 position, Hail received 316 applications. For the upcoming Fall 2020 position, Hail has thus far received 162 applications.

View poor person's hail
set -ex
cat > <<'EOF'
set -ex
free -ws 5 | grep --line-buffered Mem > free.log &
danking /
Last active Oct 1, 2018 — forked from ce-carey/
Hail PRS code

PRS Scoring of the UK Biobank Using Hail

The script was created to generate polygenic risk scores (PRS) in the UK Biobank (UKB) using Hail 0.2. Currently, I have hard-coded the file locations in-line with my own file structure and that of the Neale Lab UKB Round 2 GWAS.

NOTE: This is under active development and generally intended for my own use and the use of my collaborators. ADAPT AT YOUR OWN PERIL!!

How to Run

  1. This script assumes that you have generated an input file using my process-sumstats and LDPred repos.
View gist:47d5d996ef0e8337e895828df07c592c
{"action": "submitted", "review": {"id": 159193269, "node_id": "MDE3OlB1bGxSZXF1ZXN0UmV2aWV3MTU5MTkzMjY5", "user": {"login": "hail-ci-test-2", "id": 43152863, "node_id": "MDQ6VXNlcjQzMTUyODYz", "avatar_url": "", "gravatar_id": "", "url": "", "html_url": "", "followers_url": "", "following_url": "{/other_user}", "gists_url": "{/gist_id}", "starred_url": "{/owner}{/repo}", "subscriptions_url": "", "organizations_url": "", "repos_url": "", "events_url": "{/privacy}", "received_events
import hail as hl
import pickle
import time
import argparse
def flip_text(base):
:param StringExpression base: Expression of a single base
:return: StringExpression of flipped base
:rtype: StringExpression
View gist:fa160f9ecf137359ea108fefa5230619
This file has been truncated, but you can view the full file.
2018-08-22 16:55:21 Hail: WARN: This Hail JAR was compiled for Spark 2.2.0, running with Spark 2.2.1.
Compatibility is not guaranteed.
2018-08-22 16:55:21 SparkContext: INFO: Running Spark version 2.2.1
2018-08-22 16:55:21 SparkContext: INFO: Submitted application: Hail
2018-08-22 16:55:21 SparkContext: INFO: Spark configuration:
spark.driver.extraJavaOptions=-Xss4M -Dlog4j.configuration=file:///tmp/819ef837c3604c95828b61032b37a28e/
You can’t perform that action at this time.