Installing CGCloud on a Mac is a bit of trouble, so installing to a linux VM or an EC2 instance on AWS might be helpful. Our client chose the latter, installing to a t2.small instance that is available all the time.
$ ssh -A cgcloud.foo.com
Michael L Heuer heuermh
heuermh
/ bwa-cannoli.scala
Last active
March 19, 2019 16:57
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import org.bdgenomics.adam.rdd.ADAMContext._ | |
| import org.bdgenomics.cannoli.Cannoli._ | |
| val reads = sc.loadPairedFastqAsFragments("sample_1.fq", "sample_2.fq") | |
| val bwaArgs = new BwaArgs() | |
| bwaArgs.sample = "sample" | |
| bwaArgs.indexPath = "/data/hs38DH.fa" | |
| bwaArgs.useDocker = true | |
| //bwaArgs.useSingularity = true |
heuermh
/ bwa-pipe.scala
Last active
June 27, 2018 21:33
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import org.bdgenomics.adam.models.{ RecordGroup, RecordGroupDictionary } | |
| import org.bdgenomics.adam.rdd.ADAMContext._ | |
| import org.bdgenomics.adam.rdd.fragment.InterleavedFASTQInFormatter | |
| import org.bdgenomics.adam.rdd.read.{ AlignmentRecordRDD, AnySAMOutFormatter } | |
| import org.bdgenomics.adam.sql.{ AlignmentRecord => AlignmentRecordProduct } | |
| import org.bdgenomics.formats.avro.AlignmentRecord | |
| val reads = sc.loadPairedFastqAsFragments("sample_1.fq", "sample_2.fq") | |
| implicit val tFormatter = InterleavedFASTQInFormatter |
heuermh
/ gist:5e04ce1b90a588c4f46605eef43d3236
Created
March 26, 2018 21:09
Reference.all fails with loadPartitionedParquetAlignments
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| scala> import org.bdgenomics.adam.rdd.ADAMContext._ | |
| import org.bdgenomics.adam.rdd.ADAMContext._ | |
| scala> import org.bdgenomics.adam.models._ | |
| import org.bdgenomics.adam.models._ | |
| scala> val partitioned = sc.loadPartitionedParquetAlignments("small.partitioned.alignments.adam") | |
| 2018-03-26 16:04:58 WARN ObjectStore:568 - Failed to get database global_temp, returning NoSuchObjectException | |
| partitioned: org.bdgenomics.adam.rdd.read.AlignmentRecordRDD = DatasetBoundAlignmentRecordRDD with 2 reference sequences, 0 read groups, and 2 processing steps |
heuermh
/ gist:8614becc278e98dd287d0fe9aa536682
Created
March 14, 2018 17:18
Shuffle region join with/without sequence dictionary
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ ./bin/adam-shell --driver-memory 14g | |
| Using SPARK_SHELL=/usr/local/bin/spark-shell | |
| 2018-03-14 12:02:18 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | |
| Setting default log level to "WARN". | |
| To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). | |
| Spark context Web UI available at http://l0504206.nmdp.org:4040 | |
| Spark context available as 'sc' (master = local[*], app id = local-1521046943951). | |
| Spark session available as 'spark'. | |
| Welcome to | |
| ____ __ |
heuermh
/ Drum8.ino
Created
September 28, 2016 00:31
— forked from anonymous/Drum8.ino
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // Copyright 2016 DSP Synthesizers Sweden. | |
| // | |
| // Author: Jan Ostman | |
| // | |
| // This program is free software: you can redistribute it and/or modify | |
| // it under the terms of the GNU General Public License as published by | |
| // the Free Software Foundation, either version 3 of the License, or | |
| // (at your option) any later version. | |
| // This program is distributed in the hope that it will be useful, | |
| // but WITHOUT ANY WARRANTY; without even the implied warranty of |
heuermh
/ gist:4105bd5b4ca111979652
Created
August 5, 2015 17:07
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ bash -x ./bin/adam-submit --jars adam-plugins_2.10-0.17.1-SNAPSHOT.jar -- plugin com.github.heuermh.adam.plugins.CountAlignments adam-core/src/test/resources/small.sam | |
| + set -e | |
| + DOUBLE_DASH=False | |
| + for ARG in '"$@"' | |
| + [[ --jars == \-\- ]] | |
| + for ARG in '"$@"' | |
| + [[ adam-plugins_2.10-0.17.1-SNAPSHOT.jar == \-\- ]] | |
| + for ARG in '"$@"' | |
| + [[ -- == \-\- ]] | |
| + DOUBLE_DASH=True |
heuermh
/ gist:928f654d8af67269d199
Created
April 14, 2015 15:31
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2015-04-14 10:29:00 ERROR MetricsSystem:96 - Sink class org.apache.spark.metrics.sink.MetricsServlet cannot be instantialized | |
| java.lang.reflect.InvocationTargetException | |
| at sun.reflect.GeneratedConstructorAccessor56.newInstance(Unknown Source) | |
| at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) | |
| at java.lang.reflect.Constructor.newInstance(Constructor.java:526) | |
| at org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:173) | |
| at org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:167) | |
| at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) | |
| at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) | |
| at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) |
heuermh
/ gist:14f7b33def990f4eb7a2
Created
January 22, 2015 20:59
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ mvn clean install -P scala-2.11 | |
| [INFO] Scanning for projects... | |
| [WARNING] | |
| [WARNING] Some problems were encountered while building the effective model for org.bdgenomics.adam:adam-core_2.11:jar:0.15.1-SNAPSHOT | |
| [WARNING] 'artifactId' contains an expression but should be a constant. @ org.bdgenomics.adam:adam-core_${scala.artifact.suffix}:[unknown-version], /Users/mheuer/working/fnothaft-adam/adam-core/pom.xml, line 10, column 15 | |
| [WARNING] | |
| [WARNING] Some problems were encountered while building the effective model for org.bdgenomics.adam:adam-apis_2.11:jar:0.15.1-SNAPSHOT | |
| [WARNING] 'artifactId' contains an expression but should be a constant. @ org.bdgenomics.adam:adam-apis_${scala.artifact.suffix}:[unknown-version], /Users/mheuer/working/fnothaft-adam/adam-apis/pom.xml, line 11, column 15 | |
| [WARNING] | |
| [WARNING] Some problems were encountered while building the effective model for org.bdgenomics.adam:adam-cli_2.11:jar:0.15.1-SNAPSHOT |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ brew --config | |
| HOMEBREW_VERSION: 0.9.5 | |
| ORIGIN: https://github.com/Homebrew/homebrew.git | |
| HEAD: 42050a0039b83f34b0793d60f7bb70f4a98d7229 | |
| HOMEBREW_PREFIX: /usr/local | |
| HOMEBREW_CELLAR: /usr/local/Cellar | |
| CPU: 8-core 64-bit sandybridge | |
| OS X: 10.9.4-x86_64 | |
| Xcode: 5.1.1 | |
| CLT: 5.1.0.0.1.1396320587 |