This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
\usepackage{ifxetex} | |
\ifxetex | |
\usepackage{fontspec} | |
\defaultfontfeatures{Ligatures=TeX} % To support LaTeX quoting style | |
\setromanfont{Palatino Linotype} %mijn fonts | |
\setmonofont{Source Code Pro for Powerline} %mijn vim font | |
\else | |
\usepackage[T1]{fontenc} | |
\usepackage[utf8]{inputenc} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from itertools import cycle | |
if __name__ == "__main__": | |
f1 = open("./fastq1.fq") | |
f2 = open("./fastq2.fq") | |
reference = cycle([f1, f2]) | |
for i in range(8): | |
print("Checking chunk{}".format(i)) | |
with open('./chunk{}'.format(i)) as chunkfile: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
diff = np.zeros((len(var),)) | |
for i in range(len(var)): | |
assert len(test_outputs[i, :]) == window_size | |
(test_pass, no_diff) = testmod.kupiec(test_outputs[i, :], var[i]) | |
diff[i] = no_diff | |
print(no_diff) | |
print("Mean diff {}".format(np.mean(np.abs(diff)))) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Function that ensures only unique data points get entered | |
into shared queue on a overlapping window data api. Achieves | |
this by keeping 1) a set with data points that have been entered | |
into the queue 2) a local queue that can be used to ensure that only | |
the last batch_size are in the set. This gives us constant time membership | |
checks and bounds the size of the hashset to two times the batch_size | |
""" | |
def process_data_to_queue(batch_size): | |
global_queue = Queue.Queue() |