scottfrazer

Go vs. Scala (Akka) Concurrency

A comparison from 2 weeks using Go.

Actors vs. Functions

Akka's central principle is that there you have an ActorSystem which runs Actors. An Actor is defined as a class and it has a method to receive messages.

scottfrazer

clean:
	-rm DB kernel maple.log
compile:
	go build kernel.go db.go log.go parse.go

scottfrazer

package main

import (
    "net/http"
    "fmt"
    "io/ioutil"
		"strconv"
		"strings"
		"os"
		"bufio"

scottfrazer

[info] Loading global plugins from /Users/sfrazer/.sbt/0.13/plugins
[info] Loading project definition from /Users/sfrazer/projects/cromwell/project
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[info] Set current project to cromwell (in build file:/Users/sfrazer/projects/cromwell/)
[info] Compiling 2 Scala sources to /Users/sfrazer/projects/cromwell/engine/target/scala-2.11/classes...
[info] Running cromwell.Main run scatter.wdl -
[2016-05-25 11:47:02,566] [info] Slf4jLogger started
[2016-05-25 11:47:02,604] [info] RUN sub-command

scottfrazer

workflow w {
  Array[String] x = ["a", "b", "c"]
  String z = "d"
  
  call t as t0 {input: a=z} # a="d"
  
  scatter (y in x) {
    String z = y + "foo"
    
    call t as t1 {input: a=y, b=z} # (for first shard) a="a", b="afoo"

scottfrazer

task append_array {
  Array[Int] input_array
  Array[Int] elements_to_add
  command {
    python <<CODE
    for x in [${sep="," input_array}]:
      print(x)
    for x in [${sep="," elements_to_add}]:
      print(x)
    CODE

scottfrazer

  /** PBE: the return value of WorkflowExecutionActorState is just temporary.
    *      This should probably return a Try[BackendJobDescriptor], Unit, Boolean,
    *      Try[ActorRef], or something to indicate if the job was started
    *      successfully.  Or, if it can fail to start, some indication of why it
    *      failed to start
    */
  private def startJob(call: Call, index: Option[Int], attempt: Int, inputs: Map[FullyQualifiedName, WdlValue]): WorkflowExecutionActorState = {
    val jobKey = BackendJobDescriptorKey(call, index, attempt)
    val jobDescriptor = BackendJobDescriptor(workflowDescriptor.backendDescriptor, jobKey, inputs)
    val configDescriptor = BackendConfigurationDescriptor(ConfigFactory.parseString("{}"), ConfigFactory.load())

scottfrazer

Formerly WorkflowBackendActorFactory, the new interface is BackendLifecycleActorFactory:

trait BackendLifecycleActorFactory {
  def getWorkflowInitializationActorProps(...): Option[Props] = ???
  def getJobExecutionActorProps(...): Props = ???
  def getWorkflowFinalizationActorProps(...): Option[Props] = ???
}

scottfrazer

.
├── htcondor
│   ├── project
│   └── src
│       ├── main
│       │   ├── java
│       │   ├── resources
│       │   ├── scala
│       │   │   └── cromwell
│       │   │       └── backend

scottfrazer

Apr 13 02:21:26 gce-cromwell-dev01 docker/cromwell-app[4123]: 2016-04-13 02:21:26,183 cromwell-system-akka.actor.default-dispatcher-47 INFO  - WorkflowManagerActor submitWorkflow input id = None, effective id = 0790bc7e-bdc0-49f5-a255-54fe6caa9bbc
Apr 13 02:21:26 gce-cromwell-dev01 docker/cromwell-app[4123]: 2016-04-13 02:21:26,185 cromwell-system-akka.actor.default-dispatcher-36 WARN  - Unrecognized workflow option(s): account_name
Apr 13 02:21:26 gce-cromwell-dev01 docker/cromwell-app[4123]: 2016-04-13 02:21:26,241 cromwell-system-akka.actor.default-dispatcher-36 INFO  - WorkflowActor [UUID(0790bc7e)]: Start(Some(Actor[akka://cromwell-system/user/cromwell-service/Endpoint-endpointActorName7115701931068594/Endpoint-endpointActorName7115701931068594#-1335398672])) message received
Apr 13 02:21:26 gce-cromwell-dev01 docker/cromwell-app[4123]: 2016-04-13 02:21:26,241 cromwell-system-akka.actor.default-dispatcher-47 INFO  - JesBackend [UUID(0790bc7e)]: Creating authentication file for workflow 0790bc7e-bdc0-49f5