Project:
Data streaming in scientific workflows, implementation for Toil
At the start of GSoC
I posted a blog post in CWL
community with my proposed design of the project and a short introduction about Toil
. It was cross-posted on Open Bioinformatics Foundation (OBF) blog:
Working on a CWL-Toil project with the Open Bioinformatics Foundation
I accomplished most of my goals. The main software artifact is the implementation of input data streaming in toil-cwl-runner
. I managed to achieve a bonus goal of the project which was allowing data streaming for both AWS
and Google Cloud buckets
by making use of existing cloud connectors in Toil
. The feature was merged.\