Code from the live demo of FastParse during my Scala in the City talk

fastparse.org

FastParse Live Demo, Scala In The City November 2018

Import the Fast parse library

As well as the unsurprising fastparse._ we also need to import some implicits which affect the behaviour of the repetition operators. This is because there are multiple ways to define repetition, for example whether or not you want to ignore whitespace and so in any given context you can have implicits which customise this behaviour without having to change your code.

In this case we are choosing to ignore whitespace when we search for repetitions.

import $ivy.`com.lihaoyi::fastparse:2.0.4`
import fastparse._, NoWhitespace._

Note that ammonite users might find an issue if they are using an older version of ammonite, as it actually imports fastparse into the classpath already, so if you want to use a later version in that case, you are a bit stuck (the onlyl fix that I found was to use a later ammonite).

Define some example input

This is just a string representing some sample input that we are intending to parse: a mocked up log file entry.

val input =
  """2015-05-22 16:46:46,985 - __main__ - INFO - Starting to Wait for Files
    |2015-05-22 16:46:56,645 - __main__ - INFO - Starting: Attempt 1 Checking for New Files from gs://folder/folder/""".stripMargin

Function for quickly testing a parser

This is just a little helper function for the demo which will take a parser as an argument and run it against the input defined above.

def t[T](p: P[_] => P[T]) = parse(input, p)

First Parser: year

Let’s first attempt to create a parser that will match the year portion of the date. We can start off by just searching for a character that matches a regex-/style/ range using CharIn(). This will actually create a lookup table behind the scenes that will run very efficiently once compiled.

We are also using the P() wrapper here around our parser, which just gives it an implicitly derived name (using the sourcecode library) to ease debugging output.

def year[_:P] = P(CharIn("0-9"))
parse(input, year(_)).get.value

The syntax that we are using here contains a neat little shortcut that comes from the examples on Li Haoyi’s website, where we indicate that we want an implicit argument of type P[_] by using the context bounds syntax. In other words, this will get desugared into something like this:

def year(implicit arg0: P[_]) = ???

where our implicit P[_] is now available for the calls to other parsers in the body of year().

Parsers that Capture

When we run the above parser, it will work:

@ parse(input, year(_))
res4: Parsed[Unit] = Success((), 1)

but we’ll notice that it just returns a Unit value. This is because, by default, parsers in FastParse do not capture (which is the opposite to some other libraries), on the assumption that this is the most common case, and so you explicitly need to mark a parser as capturing. This has the advantage of obviating the use of operators like <~ Parser Combinators library, the which matches the previous parser but discards its value.

We use the ! method with FastParse to turn a non-capturing parser into a capturing one:

def year[_:P] = P(CharIn("0-9")).!

This will now give us a result like:

@ parse(input, year(_))
res6: Parsed[String] = Success("2", 1)

Parser Repetition

This is capturing a number now, but only one digit. So this is where we need to use the repetition methods which depend on our import of NoWhitespace above.

def year[_:P] = P(CharIn("0-9").rep(1)).!

This will repeat with no upper limit, and a lower limit of 1. Its possible to give more arguments to this method to set upper bounds and include a separator.

Combining Parsers

Now we are successfully parsing the year, we want to parse the next parts of the date. However, this involves actually the same thing that we have just done: parsing a number, and so we should reuse that parser. Let’s rename it to number and then we can use the ~~~ operator to create a new parser by composing two other parsers together, with the semantics that the left-side parser must be followed by the right-side one in the input in order for the new parser to match.

def number[_:P] = P(CharIn("0-9").rep(1)).!

def date[_:P] = number ~ "-" ~ number ~ "-" ~ number

When two capturing parsers are composed in this way, the result will be a new parser who’s capturing type is a tuple of the capturing types of the two combined parsers. For example, combing two parsers of type String will end up with with a parser of type (String, String).

In the date(_) parser above, the hyphens are not capturing, and therefore, the result will be a Tuple3 of number strings.

Transforming a parser result

For our final step in the example, we will notice now that we are capturing the numbers, but they are strings, and therefore we can’t use them directly to create a Date instance.

So here we can use the map() method to transform the value associated with the parser into a different value, possibly of a different type, so transforming a parser of type A through a function of type A => B will result in a parser of type B.

This means we can therefore pass the strings through Integer.parseInt to convert them to integers, and then we can parse our date parser through a Java date factory method (using a pattern match to destructure the tuple) to create a Date instance:

def number[_:P] = P(CharIn("0-9").rep(1)).!.map(Integer.parseInt)

def date[_:P] = (number ~ "-" ~ number ~ "-" ~ number) map {
  case (year, month, day) => new java.util.GregorianCalendar(year, month, day).getTime
}