Skip to content

Instantly share code, notes, and snippets.

View danking's full-sized avatar

Dan King danking

  • The Broad Institute of MIT and Harvard
  • New York, NY
  • 13:55 (UTC -04:00)
  • X @danking00
View GitHub Profile

Co-op Resume Filtering

Background

In recent semesters, Hail has received >300 applications for our one open co-op position. We engage in an uncomfortable, severe filtering process to cope with this overwhelming number of applications.

For the Spring 2020 position, Hail received 316 applications. For the upcoming Fall 2020 position, Hail has thus far received 162 applications.

set -ex
cat > foo.sh <<'EOF'
#!/bin/bash
set -ex
cores=$1
script_name=$2
free -ws 5 | grep --line-buffered Mem > free.log &
@danking
danking / README.md
Last active October 1, 2018 21:22 — forked from ce-carey/README.md
Hail PRS code

PRS Scoring of the UK Biobank Using Hail

The run_prs.py script slack_utils.py was created to generate polygenic risk scores (PRS) in the UK Biobank (UKB) using Hail 0.2. Currently, I have hard-coded the file locations in-line with my own file structure and that of the Neale Lab UKB Round 2 GWAS.

NOTE: This is under active development and generally intended for my own use and the use of my collaborators. ADAPT AT YOUR OWN PERIL!!

How to Run

  1. This script assumes that you have generated an input file using my process-sumstats and LDPred repos.
{"action": "submitted", "review": {"id": 159193269, "node_id": "MDE3OlB1bGxSZXF1ZXN0UmV2aWV3MTU5MTkzMjY5", "user": {"login": "hail-ci-test-2", "id": 43152863, "node_id": "MDQ6VXNlcjQzMTUyODYz", "avatar_url": "https://avatars0.githubusercontent.com/u/43152863?v=4", "gravatar_id": "", "url": "https://api.github.com/users/hail-ci-test-2", "html_url": "https://github.com/hail-ci-test-2", "followers_url": "https://api.github.com/users/hail-ci-test-2/followers", "following_url": "https://api.github.com/users/hail-ci-test-2/following{/other_user}", "gists_url": "https://api.github.com/users/hail-ci-test-2/gists{/gist_id}", "starred_url": "https://api.github.com/users/hail-ci-test-2/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/users/hail-ci-test-2/subscriptions", "organizations_url": "https://api.github.com/users/hail-ci-test-2/orgs", "repos_url": "https://api.github.com/users/hail-ci-test-2/repos", "events_url": "https://api.github.com/users/hail-ci-test-2/events{/privacy}", "received_events
import hail as hl
import pickle
import time
import argparse
def flip_text(base):
"""
:param StringExpression base: Expression of a single base
:return: StringExpression of flipped base
:rtype: StringExpression
This file has been truncated, but you can view the full file.
2018-08-22 16:55:21 Hail: WARN: This Hail JAR was compiled for Spark 2.2.0, running with Spark 2.2.1.
Compatibility is not guaranteed.
2018-08-22 16:55:21 SparkContext: INFO: Running Spark version 2.2.1
2018-08-22 16:55:21 SparkContext: INFO: Submitted application: Hail
2018-08-22 16:55:21 SparkContext: INFO: Spark configuration:
spark.app.name=Hail
spark.driver.extraClassPath=./hail-devel-62679fc21687-Spark-2.2.0.jar
spark.driver.extraJavaOptions=-Xss4M -Dlog4j.configuration=file:///tmp/819ef837c3604c95828b61032b37a28e/driver_log4j.properties
spark.driver.maxResultSize=0
-- Logs begin at Wed 2018-08-08 13:03:33 UTC, end at Wed 2018-08-08 13:17:16 UTC. --
Aug 08 13:03:33 localhost kernel: Initializing cgroup subsys cpuset
Aug 08 13:03:33 localhost kernel: Initializing cgroup subsys cpu
Aug 08 13:03:33 localhost kernel: Initializing cgroup subsys cpuacct
Aug 08 13:03:33 localhost kernel: Linux version 4.4.111+ (chrome-bot@build279-m2.golo.chromium.org) (gcc version 4.9.x 20150123 (prerelease) (4.9.2_cos_gg_4.9.2-r175-0c5a656a1322e137fa4a251f2ccc6c4022918c0a_4.9.2-r175) ) #1 SMP Thu Feb 1 22:06:37 PST 2018
Aug 08 13:03:33 localhost kernel: Command line: BOOT_IMAGE=/syslinux/vmlinuz.A init=/usr/lib/systemd/systemd boot=local rootwait ro noresume noswap loglevel=7 noinitrd console=ttyS0 vsyscall=emulate security=apparmor systemd.unified_cgroup_hierarchy=false systemd.legacy_systemd_cgroup_controller=true dm_verity.error_behavior=3 dm_verity.max_bios=-1 dm_verity.dev_wait=1 i915.modeset=1 cros_efi root=/dev/dm-0 "dm=1 vroot none ro 1,0 2539520 verity payload=PARTUUID=72C5C8D0-6721-
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.