kevin_scala_functional_prog.md

Functional Programming features in Scala

I've been exploring functional programming with Scala and its eco system for the past few months.

In this post, I'll highlight some of the features of the language that enable intuitive creation of functional code for distributed systems and data operations.

Higher Order functions

As per the official documentation, Functions are first class objects in Scala, which means that they can -

• Take another function as an argument, or ...
• Return a function

An example of a function taking another function as an argument is the map() function in Scala's standard collections library.

val examplelist: List[Int] = List(2,9,8,14)
examplelist.map(x=> x * 2) // anonymous function as argument

When working with standard Scala collections it's also very intuitive to chain operators, especially with the infix notation. In the small code example below, I'm defining a list of numbers from 1 to 20, filtering on even numbers and then summing them up.

(1 to 20).toList filter (_%2 == 0) reduce (_ + _)

The _ is the wildcard operator - in the case of maps and filters, it refers to the value in the collection.

Recursion

The recommended way to do operations on all the items in a collection is to use the operators map, flatMap, or reduce.

In case those operators don't meet a use case's requirements, it's very useful to write a tail-recursive function to operate on all the items in a collection.

The code example below shows a tail-recursive function definition to compute the factorial of a number.

  import scala.annotation.tailrec

  @tailrec
  // Factorial Function Implementation that uses Tail Recursion
  def factorial(in_x: Double, prodsofar: Double = 1.0): Double = {
    if (in_x==0) prodsofar
    else factorial(in_x-1, prodsofar*in_x)
  }
  factorial(5)

In Scala, a tail-recursive function as above can be optimised by the compiler (using the @tailrec annotation above) to occupy just 1 stack frame - so there's no chance of a stackoverflow error even for many levels of recursion. This is possible out-of-the-box, without any need for frameworks or plugins.

As mentioned above, the recommended way is to use to collections operators (such as reduce, etc.). As a demo of the ease of use of the collection's APIs, the above factorial function can also be implemented by the 1-liner below -

(1 to 5).toList reduce (_*_)

To conceptually understand reduce, check out this great link!

(Also do check out the explanations of foldLeft, foldRight, map, flatMap to understand some commonly used data operations!)

Case classes

Case classes can be instantiated very easily with no boiler plate code, such as the example below.

case class BusinessTransaction(sourceaccountid: Long, targetaccountid: Long, amount: Long)

// create some transactions now to demo case classes
val 1_xaction = BusinessTransaction(112333L, 998882L, 20L) // I lend my friend
val 2_xaction = BusinessTransaction(998882L, 112333L, 20L) // My friend pays me back

Just 1 case class .. line above does the following useful things -

• Defines the 3 immutable values sourceaccountid, targetaccountid and amount
• Defines get methods to access the constructor arguments (eg: 1_xaction.amount)

While the ease of use is great, case classes are the recommended way to store immutable data instances in Scala. For example, in a big data application, each line of a large datafile can be modelled by a case class and stored.

An example of the use of a case class to store data is here.

In the linked example, the function rawPostings models each line of the datafile as an instance of case class Posting. Eventually it returns a dataset of type RDD[Posting].

Pattern Matching

In Scala, objects such as case classes, regular classes and collections can be decomposed through pattern matching.

By this I mean that you can use pattern matching to -

• Decompose an object's type (example below)
• Get the head of a collection (such as a List or a Seq)

The code example below shows how to use pattern matching to decompose a Seq.

val seq1: Seq[Int] = Seq(1,3,4,5,5)
seq1 match {
    case x::y => println(s"The first element in the sequence is ${x}")
    case Nil => println("The sequence is empty")
}

The cons operator (::) creates a list made of the head (x) and the rest of the list (called the tail, y).

Companion Objects

In OOP, a static variable is sometimes used in a class to store state or property across multiple instantiated objects.

However there is no static keyword in Scala. Instead what we use are Companion Objects aka Singleton Objects. A Companion Object is defined using the object keyword and has the exact same name as the class that it's accompanying.

The companion objects can define immutable values, which can then be referenced by methods in the class.

There are 2 common patterns to use companion objects in Scala -

• As a factory method
• To provide functionality that is common to the class (i.e. static function in Java)

  // The 'val specie' straightaway defines an immmutable class parameter
  abstract class Animal(val specie: String) {
    import Animal._

    // Common Behaviour to be mixed-in to Canine/Feline classes
    def getConnectionParameters: String = Animal.connectionParameter
  }

  object Animal {

    // .apply() is the factory method
    def apply(specie: String): Animal = specie match {
      case "dog" => new Canine(specie)
      case "cat" => new Feline(specie)
    }

    val connectionParameter:String = System.getProperty("user.dir")
  }

  class Canine(override val specie: String) extends Animal(specie) {
    override def toString: String = s"Canine of specie ${specie}"
  }
  class Feline(override val specie: String) extends Animal(specie) {
    override def toString: String = s"Feline of specie ${specie}"
  }

  // syntactic sugar, where we don't have to say new Animal
  val doggy = Animal("dog")
  val kitty = Animal("cat")

  doggy.getConnectionParameters

Options

Most application code checks for Null/None types. Null types are handled a little differently in Scala - the construct used is called an Option. This is best demonstrated with an example.

  val customermap: Map[Int, String] = Map(
      11-> "CustomerA", 22->"CustomerB", 33->"CustomerC"
  )

  customermap.get(11)                // Map's get() returns an Option[String]
  customermap.get(11).get            // Option's get returns the String
  customermap.get(999).get           // Will throw a NoSuchElementException
  customermap.get(999).getOrElse(0)  // Will return a 0 instead of throwing an exception

In a language like Python, if None: checks would be quite common throughout the codebase. In Java, there would be try-catch blocks that would handle thrown exceptions. Options allow for focusing on the logic flow with minimal diversions for type or exception checks.

A standard way of using Options in Scala is for your custom functions to return Option[String] (or Int, Long, etc.). Let's look at the Map structure's get() function signature -

def get(key: A): Option[B]

One (intuitive) way to use this is to chain it with the getOrElse() function as shown below -

  // Map of IDs vs Names
  val customermap: Map[Int, String] = Map(
    11-> "CustomerA", 22->"CustomerB", 33->"CustomerC"
  )
  customermap.get(11).getOrElse("No customer found for the provided ID")

A very useful way of using Options is together with a collection operator like flatMap that directly handles the types for you transparently.

  // Map of IDs vs Names
  val customermap: Map[Int, String] = Map(
    11-> "CustomerA", 22->"CustomerB", 33->"CustomerC"
  )

  val listofids: List[Int] = List(11,22,33,99)

  listofids flatMap (id=> customermap.get(id))

And that's it for this post from me! I'm looking into concurrent systems with Akka and the Actor model currently. In a future post, I'll share any learning on that topic (and it's relationship to Scala's approach to functional programming).