Skip to content

Instantly share code, notes, and snippets.

Avatar

Yosuke Tanigawa yk-tanigawa

View GitHub Profile
View README.md

Example usage of the figshare API

Yosuke Tanigawa

2020/5/22

To upload large files to figshare, one can use figshare API.

This notebook summarizes the basic usage of the figshare API.

View README.md

Save Kindle annotations to google drive

Electronic books (eBooks) are changing our way of reading books by providing us enhanced functionalities to interact with books. Amazon Kindle is one of the successful example of eBook servieces populated with millions of books. One of the notable functionality of Kindle is that readers can "highlight" the sentences in books and send the highlighted texts via email. While this enable users to easily export sections of the books, they need to manually organize their exported texts. Here I present a small Google Apps Script that automatically transfers the kindle annotations to Google drive. Given the search functionality of Google Drive, users can easily look up sections of books.

This Google Apps Script performs the following tasks: (1) checks your Gmail account; (2) identify email messages from Amazon Kindle service with your annotations (attachment files); and (3) save them to your google drive folder of your choice.

Note that some publishers has a maximum amoun

View import.py
%matplotlib inline
import numpy as np
import pandas as pd
import matplotlib, collections, itertools, os, re, textwrap, logging
import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
import matplotlib.patches as mpatches
from functools import reduce
@yk-tanigawa
yk-tanigawa / jupyter-remote-login.sh
Created Feb 2, 2018
ssh port forwarding scripts
View jupyter-remote-login.sh
#!/bin/bash
# parse args
if [ $# -lt 2 ] ; then
echo "usage: $0 <target-server> <login-server> [local port (default: 18888)] [target port (default: 8888)] [login port (default: rand)]" >&2
exit 1
fi
target=$1
View README.md

This script works as a wrapper for sbatch command.

Usage: sherlock-one-liner python --version

View data.csv
PC score
1 0.840188
2 0.394383
3 0.783099
4 0.79844
5 0.911647
6 0.197551
7 0.335223
8 0.76823
9 0.277775
@yk-tanigawa
yk-tanigawa / pgen_write.out.txt
Created Jul 4, 2017
plink2 pgenlib.PgenWriter caused a segmentation fault
View pgen_write.out.txt
$ python pgen_write.py
2017-07-04 00:44:50,706 pgen_write DEBUG pgen write test script
2017-07-04 00:44:50,706 read_alleles_range DEBUG reading alleles range 0:10 from ./pgen_in.pgen
2017-07-04 00:44:50,725 read_alleles_range DEBUG The shape of the numpy nd-array is (10, 224676)
2017-07-04 00:44:50,725 pgen_write DEBUG shape of buffer is (10, 224676)
2017-07-04 00:44:50,727 pgen_write DEBUG writing SNP 0 of 10 ...
2017-07-04 00:44:50,728 pgen_write DEBUG writing SNP 1 of 10 ...
2017-07-04 00:44:50,730 pgen_write DEBUG writing SNP 2 of 10 ...
2017-07-04 00:44:50,731 pgen_write DEBUG writing SNP 3 of 10 ...
2017-07-04 00:44:50,733 pgen_write DEBUG writing SNP 4 of 10 ...
View parallel.py
np.array(
Parallel(
n_jobs=multiprocessing.cpu_count()
)(delayed(pre_process_main)(read_3channel_file(lead_df, i))
for i in range(batch_interval[0], batch_interval[1])),
dtype = np.float64
)
View hello.sbatch
#!/bin/sh
# (@) hello world
#SBATCH --job-name=hello
#SBATCH --output=hello.%j.out
#SBATCH --error=hello.%j.err
#SBATCH --time=0:01:00
#SBATCH --qos=normal
#SBATCH -p normal
View parallel to replace loops
$ parallel mkdir -p dirs/{2}/{1} :::: <(seq 1 10) <(seq 1 5)
$ parallel -k 'echo {}' :::: <(seq 1 10)
$ # https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
You can’t perform that action at this time.