Skip to content

Instantly share code, notes, and snippets.

@chrisprobert

chrisprobert/gist:08a63955967426df1b4aace8d40c6eed Secret

Created April 18, 2016 06:51
Show Gist options
  • Save chrisprobert/08a63955967426df1b4aace8d40c6eed to your computer and use it in GitHub Desktop.
Save chrisprobert/08a63955967426df1b4aace8d40c6eed to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
== species settings
Chr. sizes file : /mnt/data/annotations/by_release/hg19.GRCh37/hg19.chrom.sizes
Genome size (hs,mm) : hs
Ref. genome seq. dir. : /mnt/data/ENCODE/sequence/encodeHg19Male
Umap dir. : /mnt/data/ENCODE/umap/hg19_allmappable/globalmap_k1tok1000
Umap dir. (HiC) : /mnt/data/ENCODE/umap/encodeHg19Male/globalmap_k20tok54
BWA index : /mnt/data/annotations/indexes/bwa_indexes/encodeHg19Male/v0.7.10/encodeHg19Male_bwa-0.7.10.fa
Bowtie index : /mnt/data/annotations/indexes/bowtie1_indexes/encodeHg19Male/encodeHg19Male
Bowtie2 index : /mnt/data/annotations/indexes/bowtie2_indexes/bowtie2/ENCODEHg19_male
V-Plot index : /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/hg19_RefSeq_stranded.bed.gz
Black list for IDR : /mnt/data/ENCODE/blacklists/wgEncodeDacMapabilityConsensusExcludable.bed.gz
Ref. genome seq. fasta : /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/encodeHg19Male.fa
Black list for ataqc : /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/Anshul_Hg19UltraHighSignalArtifactRegions.bed.gz
DNase bed for ataqc : /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/reg2map_honeybadger2_dnase_all_p10_ucsc.bed.gz
Promoter bed for ataqc : /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/reg2map_honeybadger2_dnase_prom_p2.bed.gz
Enhancer bed for ataqc : /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/reg2map_honeybadger2_dnase_enh_p2.bed.gz
Reg2map bed for ataqc : /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/dnase_avgs_reg2map_p10_merged_named.pvals.gz
Roadmap metadata for ataqc : /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/eid_to_mnemonic.txt
== report settings
URL root for output directory :
== align bowtie settings
# threads (bowtie) : 2
Walltime (bowtie) : 23h
Max. memory (bowtie) : 12G
== align bowtie2 settings
# threads (bowtie2) : 2
Walltime (bowtie2) : 23h
Max. memory (bowtie2) : 12G
== postalign bam settings
MAPQ reads rm thresh. : 30
Rm. tag reads with str. :
Walltime (picard markdup) : 24h
Max. memory (picard markdup) : 16G
== postalign bed/tagalign settings
# read for xcor subsample : 15000000
== preseq settings
Xmax for preseq : 500
== callpeak macs2 settings
# threads (macs2) : 2
Walltime (macs2) : 10h
Max. memory (macs2) : 8G
P-value threshold (macs2) : 0.01
== idr settings
IDR threshold : 0.02
== atac pipeline settings
Single ended data set? : false
Input data type : fastq
Fastqs are trimmed? : false
# Replicates : 1
MACS2 peak calling : true
Subsample replicates : false
No pseudo replicates : false
Perform IDR analysis on peaks : false
Vplot generation : false
Preseq analysis : false
No ATAQC (advanced QC report including vplot and preseq) : false
Use CSEM for alignment : false
Max. memory for ATAQC : 8G
Smoothing window for MACS2 : 150
== checking atac inputs (data type = fastq) ...
Checking parameters and data files for ATAQC. If you don't want ATAQC then add '-no_ataqc'
Rep1 fastq (PE) :
/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/fastqs/SKRT-3-ATAC-022616_S1_R1.fq.gz
/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/fastqs/SKRT-3-ATAC-022616_S1_R2.fq.gz
[E::hts_open] fail to open file '{1=/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}'
samtools: failed to open "{1=/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}" for reading: No such file or directory
Task failed:
Program & line : '/users/cprobert/dev/bds_atac/modules/postalign_bam.bds', line 440
Task Name : 'bam_to_bed_atac rep1'
Task ID : 'atac.bds.20160417_234224_243_parallel_16/task.postalign_bam.bam_to_bed_atac_rep1.line_440.id_4'
Task PID : '14211'
Task hint : 'nonMitoChromosomes=$(samtools view -H "{1=/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}" |; grep chr | cut -f2 | sed s/SN://g | grep -v chrM); samtools view -b "{1=/srv/persistent/cprob'
Task resources : 'cpus: 1 mem: -1.0 B wall-timeout: 8640000'
State : 'ERROR'
Dependency state : 'ERROR'
Retries available : '1'
Input files : '[/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/{1=/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}]'
Output files : '[/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.tn5.bed.gz]'
Script file : '/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/atac.bds.20160417_234224_243_parallel_16/task.postalign_bam.bam_to_bed_atac_rep1.line_440.id_4.sh'
Exit status : '1'
Program :
# SYS command. line 442
. /etc/profile.d/modules.sh; module add bowtie/1.1.1 bowtie/2.2.4 samtools/1.2 bedtools/2.21.0 picard-tools/1.129 ucsc_tools/3.0.9 java/latest texlive/2013 trim_galore/0.4.0 r/2.15.1 phantompeakqualtools/default idr/latest preseq/2.0.2 MACS2/2.1.0 csem/2.4; export PATH=/software/idrCode:/users/cprobert/dev/bds_atac/.:/users/cprobert/dev/bds_atac/modules:/users/cprobert/dev/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; export _JAVA_OPTIONS='-Xms256M -Xmx1024M -XX:ParallelGCThreads=1'; export MAX_JAVA_MEM='8G'; export MALLOC_ARENA_MAX=4;
# SYS command. line 444
nonMitoChromosomes=$(samtools view -H "{1=/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}" | \
grep chr | cut -f2 | sed 's/SN://g' | grep -v chrM)
# SYS command. line 447
samtools view -b "{1=/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}" ${nonMitoChromosomes} > "/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.nonchrM.bam"
# SYS command. line 448
samtools view -b "{1=/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}" chrM > "/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.chrM.bam"
# SYS command. line 452
bamToBed -i "/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.nonchrM.bam" | \
awk -F $'\t' 'BEGIN {OFS = FS}{ if ($6 == "+") {$2 = $2 + 4} else if ($6 == "-") {$3 = $3 - 5} print $0}' | \
gzip -c > "/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.tn5.bed.gz"
# SYS command. line 456
if [ -f ${PICARDROOT}/CollectInsertSizeMetrics.jar ]; then \
java -Xmx4G -Xms256M -jar ${PICARDROOT}/CollectInsertSizeMetrics.jar \
INPUT="/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.nonchrM.bam" OUTPUT="/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.nonchrM.bam.hist_data.log" \
H="/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.nonchrM.bam.hist_graph.pdf" W=1000 STOP_AFTER=5000000; \
else \
java -Xmx4G -Xms256M -jar ${PICARDROOT}/picard.jar CollectInsertSizeMetrics \
INPUT="/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.nonchrM.bam" OUTPUT="/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.nonchrM.bam.hist_data.log" \
H="/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}.nonchrM.bam.hist_graph.pdf" W=1000 STOP_AFTER=5000000; \
fi
StdErr (100000000 lines) :
[E::hts_open] fail to open file '{1=/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}'
samtools: failed to open "{1=/srv/persistent/cprobert/projects/jcharalel_atac_022616_ATAC_SKBR3_SKRT-28854828/analysis/SKRT-3-ATAC-022616_S1/SKRT-3-ATAC-022616_S1_R1.trim.PE2SE.nodup.bam}" for reading: No such file or directory
Fatal error: /users/cprobert/dev/bds_atac/atac.bds, line 501, pos 5. Task/s failed.
Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.
Fatal error: /users/cprobert/dev/bds_atac/atac.bds, line 279, pos 2. Task/s failed.
atac.bds, line 53 : atac()
atac.bds, line 272 : void atac() {
atac.bds, line 279 : wait
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment