Skip to content

Instantly share code, notes, and snippets.

@dtenenba
Created December 22, 2018 19:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dtenenba/61bcf60f129b817cd894ee222789369a to your computer and use it in GitHub Desktop.
Save dtenenba/61bcf60f129b817cd894ee222789369a to your computer and use it in GitHub Desktop.
[2018-12-22 19:38:42,36] [info] Running with database db.url = jdbc:hsqldb:mem:64cb552b-816c-4f93-bafd-21743b589b42;shutdown=false;hsqldb.tx=mvcc
[2018-12-22 19:38:55,72] [info] Running migration RenameWorkflowOptionsInMetadata with a read batch size of 100000 and a write batch size of 100000
[2018-12-22 19:38:55,83] [info] [RenameWorkflowOptionsInMetadata] 100%
[2018-12-22 19:38:56,42] [info] Running with database db.url = jdbc:hsqldb:mem:f9eb9b02-87e7-4e76-883b-2aaced01ab36;shutdown=false;hsqldb.tx=mvcc
[2018-12-22 19:38:57,18] [info] Slf4jLogger started
[2018-12-22 19:38:57,65] [info] Workflow heartbeat configuration:
{
"cromwellId" : "cromid-1478c70",
"heartbeatInterval" : "2 minutes",
"ttl" : "10 minutes",
"writeBatchSize" : 10000,
"writeThreshold" : 10000
}
[2018-12-22 19:38:57,72] [info] Metadata summary refreshing every 2 seconds.
[2018-12-22 19:38:57,92] [info] WriteMetadataActor configured to flush with batch size 200 and process rate 5 seconds.
[2018-12-22 19:38:57,92] [info] KvWriteActor configured to flush with batch size 200 and process rate 5 seconds.
[2018-12-22 19:38:58,01] [info] CallCacheWriteActor configured to flush with batch size 100 and process rate 3 seconds.
[2018-12-22 19:38:59,71] [info] JobExecutionTokenDispenser - Distribution rate: 50 per 1 seconds.
[2018-12-22 19:38:59,81] [info] SingleWorkflowRunnerActor: Version 36
[2018-12-22 19:38:59,82] [info] SingleWorkflowRunnerActor: Submitting workflow
[2018-12-22 19:39:00,03] [info] Unspecified type (Unspecified version) workflow 5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29 submitted
[2018-12-22 19:39:00,04] [info] SingleWorkflowRunnerActor: Workflow submitted 5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29
[2018-12-22 19:39:00,11] [info] 1 new workflows fetched
[2018-12-22 19:39:00,11] [info] WorkflowManagerActor Starting workflow 5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29
[2018-12-22 19:39:00,12] [info] WorkflowManagerActor Successfully started WorkflowActor-5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29
[2018-12-22 19:39:00,12] [info] Retrieved 1 workflows from the WorkflowStoreActor
[2018-12-22 19:39:00,12] [warn] SingleWorkflowRunnerActor: received unexpected message: Done in state RunningSwraData
[2018-12-22 19:39:00,13] [info] WorkflowStoreHeartbeatWriteActor configured to flush with batch size 10000 and process rate 2 minutes.
[2018-12-22 19:39:00,43] [info] MaterializeWorkflowDescriptorActor [5abd010a]: Parsing workflow as CWL v1.0
[2018-12-22 19:39:00,72] [info] Pre-Processing /tmp/cwl_temp_dir_12896957636850987565/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/root/reproducible-workflows/CWL/SingleStepWorkflow/cromwell-36.jar) to method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of org.python.core.PySystemState
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[2018-12-22 19:39:43,66] [info] Pre-Processing /tmp/cwl_temp_dir_12896957636850987565/bwa-pe.cwl
[2018-12-22 19:39:44,50] [info] MaterializeWorkflowDescriptorActor [5abd010a]: Call-to-Backend assignments: bwa -> Local
[2018-12-22 19:39:46,93] [info] WorkflowExecutionActor-5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29 [5abd010a]: Starting bwa
[2018-12-22 19:39:48,94] [info] BackgroundConfigAsyncJobExecutionActor [5abd010abwa:NA:1]: 'bwa' 'mem' '-M' '-R' '@RG\tID:A\tSM:A\tPL:Illumina' '/cromwell-executions/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl/5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29/call-bwa/inputs/-534407894/Homo_sapiens_assembly38.fasta' '/cromwell-executions/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl/5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29/call-bwa/inputs/-534407894/CM-M40-O40-K20-R1_R1.fastq.gz' '/cromwell-executions/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl/5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29/call-bwa/inputs/-534407894/CM-M40-O40-K20-R1_R2.fastq.gz' '-t' '8'
[2018-12-22 19:39:49,11] [info] BackgroundConfigAsyncJobExecutionActor [5abd010abwa:NA:1]: executing: # make sure there is no preexisting Docker CID file
rm -f /root/reproducible-workflows/CWL/SingleStepWorkflow/cromwell-executions/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl/5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29/call-bwa/execution/docker_cid
# run as in the original configuration without --rm flag (will remove later)
docker run \
--cidfile /root/reproducible-workflows/CWL/SingleStepWorkflow/cromwell-executions/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl/5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29/call-bwa/execution/docker_cid \
-i \
\
--entrypoint /bin/bash \
-v /root/reproducible-workflows/CWL/SingleStepWorkflow/cromwell-executions/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl/5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29/call-bwa:/cromwell-executions/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl/5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29/call-bwa \
genomicpariscentre/bwa@sha256:d59552f71821d0be66bb0cb8325b3b04e58f98601ee93b5a926c1960b544ccc9 /cromwell-executions/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl/5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29/call-bwa/execution/script
# get the return code (working even if the container was detached)
rc=$(docker wait cat /root/reproducible-workflows/CWL/SingleStepWorkflow/cromwell-executions/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl/5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29/call-bwa/execution/docker_cid)
# remove the container after waiting
docker rm cat /root/reproducible-workflows/CWL/SingleStepWorkflow/cromwell-executions/cwl_temp_file_5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29.cwl/5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29/call-bwa/execution/docker_cid
# return exit code
exit $rc
[2018-12-22 19:39:52,95] [info] BackgroundConfigAsyncJobExecutionActor [5abd010abwa:NA:1]: job id: 14670
[2018-12-22 19:39:52,96] [info] BackgroundConfigAsyncJobExecutionActor [5abd010abwa:NA:1]: Status change from - to WaitingForReturnCode
[2018-12-22 19:40:09,70] [info] BackgroundConfigAsyncJobExecutionActor [5abd010abwa:NA:1]: Status change from WaitingForReturnCode to Done
[2018-12-22 19:40:10,58] [error] WorkflowManagerActor Workflow 5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29 failed (during ExecutingWorkflowState): java.lang.Exception: The job was aborted from outside Cromwell
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor$$anonfun$5.applyOrElse(WorkflowExecutionActor.scala:247)
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor$$anonfun$5.applyOrElse(WorkflowExecutionActor.scala:184)
at scala.PartialFunction$OrElse.apply(PartialFunction.scala:168)
at akka.actor.FSM.processEvent(FSM.scala:687)
at akka.actor.FSM.processEvent$(FSM.scala:681)
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor.akka$actor$LoggingFSM$$super$processEvent(WorkflowExecutionActor.scala:49)
at akka.actor.LoggingFSM.processEvent(FSM.scala:820)
at akka.actor.LoggingFSM.processEvent$(FSM.scala:802)
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor.processEvent(WorkflowExecutionActor.scala:49)
at akka.actor.FSM.akka$actor$FSM$$processMsg(FSM.scala:678)
at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:672)
at akka.actor.Actor.aroundReceive(Actor.scala:517)
at akka.actor.Actor.aroundReceive$(Actor.scala:515)
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor.akka$actor$Timers$$super$aroundReceive(WorkflowExecutionActor.scala:49)
at akka.actor.Timers.aroundReceive(Timers.scala:55)
at akka.actor.Timers.aroundReceive$(Timers.scala:40)
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor.aroundReceive(WorkflowExecutionActor.scala:49)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:588)
at akka.actor.ActorCell.invoke(ActorCell.scala:557)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
at akka.dispatch.Mailbox.run(Mailbox.scala:225)
at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[2018-12-22 19:40:10,59] [info] WorkflowManagerActor WorkflowActor-5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29 is in a terminal state: WorkflowFailedState
[2018-12-22 19:40:17,92] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'.
[2018-12-22 19:40:17,94] [info] Workflow polling stopped
[2018-12-22 19:40:17,95] [info] Shutting down WorkflowStoreActor - Timeout = 5 seconds
[2018-12-22 19:40:17,95] [info] Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds
[2018-12-22 19:40:17,95] [info] Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds
[2018-12-22 19:40:17,95] [info] Aborting all running workflows.
[2018-12-22 19:40:17,95] [info] JobExecutionTokenDispenser stopped
[2018-12-22 19:40:17,95] [info] WorkflowStoreActor stopped
[2018-12-22 19:40:17,96] [info] WorkflowLogCopyRouter stopped
[2018-12-22 19:40:17,96] [info] Shutting down WorkflowManagerActor - Timeout = 3600 seconds
[2018-12-22 19:40:17,96] [info] WorkflowManagerActor All workflows finished
[2018-12-22 19:40:17,96] [info] WorkflowManagerActor stopped
[2018-12-22 19:40:17,96] [info] Connection pools shut down
[2018-12-22 19:40:17,96] [info] Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds
[2018-12-22 19:40:17,96] [info] Shutting down JobStoreActor - Timeout = 1800 seconds
[2018-12-22 19:40:17,96] [info] Shutting down CallCacheWriteActor - Timeout = 1800 seconds
[2018-12-22 19:40:17,96] [info] Shutting down ServiceRegistryActor - Timeout = 1800 seconds
[2018-12-22 19:40:17,96] [info] SubWorkflowStoreActor stopped
[2018-12-22 19:40:17,96] [info] Shutting down DockerHashActor - Timeout = 1800 seconds
[2018-12-22 19:40:17,96] [info] CallCacheWriteActor Shutting down: 0 queued messages to process
[2018-12-22 19:40:17,96] [info] Shutting down IoProxy - Timeout = 1800 seconds
[2018-12-22 19:40:17,96] [info] JobStoreActor stopped
[2018-12-22 19:40:17,96] [info] CallCacheWriteActor stopped
[2018-12-22 19:40:17,96] [info] KvWriteActor Shutting down: 0 queued messages to process
[2018-12-22 19:40:17,96] [info] WriteMetadataActor Shutting down: 0 queued messages to process
[2018-12-22 19:40:17,96] [info] DockerHashActor stopped
[2018-12-22 19:40:17,96] [info] IoProxy stopped
[2018-12-22 19:40:17,96] [info] ServiceRegistryActor stopped
[2018-12-22 19:40:17,98] [info] Database closed
[2018-12-22 19:40:17,98] [info] Stream materializer shut down
[2018-12-22 19:40:18,01] [info] WDL HTTP import resolver closed
Workflow 5abd010a-8fa0-4d1b-8d2a-9b591ebc5c29 transitioned to state Failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment