A Primer on Go, JSON, and More: Part 1

README.md

A Primer on Go, JSON, and More: Part 1

First, for those who will be writing software, I highly suggest first checking out some of the Go documentation. The website is available at golang.org, and I would encourage going through the interactive tutorial and watching this video, which goes over some of the topics I'm about to review.

If you can, try to go through more of the Go tour than just the "Welcome" section. Methods and Interfaces may be particularly useful.

Example 1: Parsing JSON

Compile and run the first example like so:

go build example1.go
./example1

The go toolchain is available at unix.ucsc.edu.

How JSON is turned into a struct

The function json.NewDecoder returns a decoder from the given file input, which here is example1.json. This decoder (dec) then has a .Decode method which takes a pointer to a struct, hence the &person.

Notice how the struct Person is defined:

type Person struct {
	Name string
	Age  int
}

When the decoder's Decode method goes to do its work, it looks for fields in the JSON that match the fields in the struct. This makes parsing JSON incredibly easy.

Things to take note of

Some format notes

Notice that the example1.json file declares four seperate JSON objects. The future GeoJSON examples we're going to look at will not be this way.

Notice that Abel's Age field is not set in the JSON. As a result, his age value gets set to the zero value for int (0). Also notice that Dr. Frankenstein has an extra field, Occupation, which is ignored by the program.

Case sensitivy

Try changing the Name string field to name string, and see what happens.

Spoiler: Recall from the Go tutorials that Go only makes public keywords that start with a capital letter. Making the name field lowercase makes it invisible to the json package, as name is no longer exported.

Try changing the Name field in the JSON to name for any of the people. The program will still work as expected. This is because the json package fills the structs case insensitevely.

Example 2: Using the data with struct reciever methods

Once again, this can be compiled and run like so:

go build example2.go
./example2

Take a second to look at the files and what they do first.

What it does

This example takes a list of coordinates in a JSON object and computes the magnitude of each. If you followed Go's method tutorial this will likely look very familiar.

Things to take note of

Struct literal

The data struct that is passed into the decoder has a "struct literal" type. That basically means it doesn't have a name, but was just defined on the spot. There was no real reason to do this, but it's possible and sometimes useful.

Arrays in the struct

The json package is smart enought to fill the coordinate array ([]Coordinate) with the coordinate list in the JSON without any extra work - all we had to do was specify it. This is pretty great for reading in our giant arrays of intersections and stuff.

The Magnitude method reciever

The last function in the file defines a method of the Coordinate struct. Notice that it recieves a pointer reciever. Try removing the * and see if the program still works. If you haven't already done the "Methods and Interfaces" tour of Go, you probably should.

Anyway, the whole point is that this is pretty easy - we just defined a type, defined a method that could operate on that type, and then the json package did all the heavy lifting for us and turned some text into useable data. Great!

Go specifics

Make sure you understand the for and range stuff. The use of Printf's %v notation is also a fun thing to be aware of.

Example 3: Putting JSON and a Simulation together (with some work)

This file is kind of long, so I tried to seperate the sections with large comments blocks.

This example shows a simple "simulation" which is entirely seperate from any specific format of data in JSON and how we can format some arbitrary JSON to be used in the simulation.

Take a look at the .go and .json and run it like the others.

On the "Simulation"

The simulation is a simple struct with two methods: AddWidget and Simulate. The simulation stores a list of widgets. These widgets are defined by the SimulationWidget. Any struct that defines a DoAThing() float64 method satisfies the SimulationWidget interface and can be put in the simulation.

On the Person struct

The Person struct stores a bunch of data that corresponds to fields in the json. Notice that we've also defined a DoAThing() float64 method for the Person struct. That means we can put it in the simulation! Also take note that DoAThing method does not have to directly follow from the data, and that the simulation won't care what that method is actually outputting. Try tinkering with the DoAThing function to return different values, or even adding new fields to the JSON. Any type of input JSON is possible for this simulation, as long as an appropriate DoAThing method is defined.

Why this is Cool and How it is Useful

The traffic simulation uses this strategy. At the time of writing, the model has an AddEdge method which recieves an Edge interface defined as such:

type Edge interface {
	Weight() float64
	To() string
	From() string
}

The Weight() method is used to determine the cost of an agent travelling along that edge (be it environmental cost, dollar cost, time cost, etc). In the future, these edges may ask for other messages like AddAgent() or RemoveAgent() (to respond to people congesting).

As such, we can import almost any kind of data and use it in the simulation, as long as we appropriately define some methods for the imported data.

I hope these examples have been illuminating about the specifics of data wrangling and how data can be imported into the simulation. The bottom line is we can accept almost any type of data as long as we implement a frontend for it.

In the next few days I'll be looking to put the simulation online, so we can all collaborate on interfacing for it, and then finishing up the simulation with agents et al.

example1.go

/*
Example 1 demonstrates simple parsing of JSON objects into go structs.
*/
package main

import (
	"encoding/json"
	"fmt"
	"io"
	"log"
	"os"
)

type Person struct {
	Name string
	Age  int
}

func main() {
	input, err := os.Open("example1.json")
	if err != nil {
		log.Fatal(err)
	}

	dec := json.NewDecoder(input)
	for {
		var person Person
		err = dec.Decode(&person)
		if err != nil {
			if err == io.EOF {
				break
			}
			log.Fatal(err)
		}
		fmt.Printf("%v\n", person)
	}
}

example1.json

{"Name": "Joe", "Age": 32}
{"Name": "Gary", "Age": 6}
{"Name": "Abel"}
{"Name": "Dr. Frankenstein", "Age": 26, "Occupation": "doctor"}

example2.go

/*
Example 2 demonstrates parsing a JSON list and a reciever method.
*/
package main

import (
	"encoding/json"
	"fmt"
	"log"
	"math"
	"os"
)

type Coordinate struct {
	X, Y float64
}

func main() {
	input, err := os.Open("example2.json")
	if err != nil {
		log.Fatal(err)
	}

	dec := json.NewDecoder(input)
	var data struct{ Coords []Coordinate }
	err = dec.Decode(&data)
	if err != nil {
		log.Fatal(err)
	}

	for _, c := range data.Coords {
		fmt.Printf("Magnitude of %v is %f\n", c, c.Magnitude())
	}
}

func (c *Coordinate) Magnitude() float64 {
	return math.Sqrt(math.Pow(c.X, 2) + math.Pow(c.Y, 2))
}

example2.json

{
    "coords": [
        { "x": 1,   "y": 1 },
        { "x": 1.2, "y": 1 },
        { "x": 8,   "y": 7 },
        { "x": 42,  "y": 145 }
    ]
}

example3.go

/*
Example 3 demonstrates bridging between JSON and an interface.
*/
package main

import (
	"encoding/json"
	"fmt"
	"log"
	"os"
)

/*
=======================
SIMULATION SECTION
=======================
*/

// An interface that the simulation can use
type SimulationWidget interface {
	DoAThing() float64
}

// A simulation struct that holds a bunch of widgets
type Simulation struct {
	Widgets []SimulationWidget
}

// Create a new simulation
func NewSimulation() Simulation {
	return Simulation{make([]SimulationWidget, 0)}
}

// Simulation method to add a widget
func (s *Simulation) AddWidget(w SimulationWidget) {
	s.Widgets = append(s.Widgets, w)
}

// "Run" the "Simulation"
func (s *Simulation) Simulate() {
	fmt.Println("Here we go, doing a heavy simulation!")
	for _, widget := range s.Widgets {
		fmt.Println("Here's a number", widget.DoAThing())
	}
	fmt.Println("Speedy! bla bla data etc")
}

/*
=======================
PERSON SECTION
=======================
*/

// Person struct, extended from ex1
type Person struct {
	Name   string
	Age    float64
	Height float64
	Mass   float64
}

// Person implements DoAThing, satisfying the SimulationWidget interface
func (p *Person) DoAThing() float64 {
	fmt.Println("My name is", p.Name)
	return p.Age + p.Height + p.Mass
}

/*
=======================
MAIN
=======================
*/

func main() {
	input, err := os.Open("example3.json")
	if err != nil {
		log.Fatal(err)
	}

	dec := json.NewDecoder(input)
	var data struct{ People []Person }
	err = dec.Decode(&data)
	if err != nil {
		log.Fatal(err)
	}

	// Create simulation
	mySimulation := NewSimulation()

	// Add everybody to the simulation
	for i, _ := range data.People {
		mySimulation.AddWidget(&data.People[i])
	}

	// Run it
	mySimulation.Simulate()
}

example3.json

{ 
    "people": [
        {"Name": "Joe", "Age": 32, "Height": 4.5, "Mass": 75},
        {"Name": "Gary", "Age": 6, "Height": 6, "Mass": 60},
        {"Name": "Abel", "Height": 5.8, "Mass": 72},
        {"Name": "Dr. Frankenstein", "Age": 26, "Height": 6.2, "Mass": 55}
    ]
}