Crusty Notes

crusty.txt

# Beyond Crusty: Real-World Protocols

## Slide 1

Who here saw last WWDC's talk Protocol-Oriented Programming in Swift? The Crusty talk?

It is without a doubt the most important Swift talk. You have to watch it. It is probably the best talk I've ever seen at WWDC. And it's entire point is that subclassing is a needless complexity and that the right solution to most problems is protocols. Apple goes so far as to say that Swift is the first Protocol-Oriented Programming language.
Now, I take some exception to that. I don't think Swift is the first Protocol-Oriented Programming language. I think Go is much more "protocol-oriented" than Swift today, but that's a debate for beer later. Swift clearly is protocol-oriented, and that's great.

## Slide 2

What Crusty said was that we've let classes cloud our understanding of data. We’ve created this huge class hierarchy to try to explain everything.

## Slide 3

When all we really wanted was something that can eat. Rather than worrying about what inherits from what, in Swift we want to focus on what things can do. Rather than inherit, we compose.
This takes us back to the Deep Magic from the Dawn of OOP. OOP was originally about objects and interfaces, not classes. We’re relearning lessons of SmallTalk from the 70s.

## Slide 4

But we still get into nasty jams with protocols, and particularly with associated types. If you want different kinds of animals to eat different kinds of food, we can get weird and surprising errors.
So today we're going to go back to the Deeper Magic from before the Dawn of OOP.
We’re going to relearn the lessons of Lisp from the 50s. And yeah, that means we're going to talk about functions. But this isn't a functional programming talk. This is about Swift and protocol oriented programming.

## Slide 5

So let’s make this more concrete with some code. We have Animals, and they all can eat different kinds of food.
<build>
Now let's make an array of grass-eating animals.
<build>
Yeah.
<build>
There's no syntax here that will fix it. We can't say "Animal, bracket Grass," or "Animal where Food equals Grass" or anything like that. We're stuck. I see this blow up for people with Equatable all the time because it includes a Self requirement. I see this blow up so often I’m giving a talk about it.

## Slide 6

OK, so how do we fix it? We make a type eraser. We make a struct that wraps the protocol.

## Slide 7

You have this specific Animal, and you want to encapsulate it. When we call the eat method on AnyAnimal, we want it to be forwarded along to the Cow’s eat method without the caller knowing anything about Cows. How do you build that?

## Slide 8

Function properties. We can store the function itself in the struct. This code doesn’t say that the variable eat is the result of calling Cow’s eat. It says that the variable eat is the method itself.

## Slide 9

So when I call it, it actually becomes a method call. My eat variable actually captures Cow() and uses it when I call this function.<build>
Just like a closure. They’re both just anonymous functions.

## Slide 10

So to build an AnyAnimal, you can just pass in an animal and the extract the functions that implement the protocol. We store them in properties, and then we conform to the protocol by just calling the functions we extracted. You can do this for most protocols.

## Slide 11

But look how mechanical this is. The typealias becomes the type parameter. The function becomes a function property. Swift won’t let you use a function property to conform to a protocol, so we have do this little underscore variable dance, which is annoying, but this is so mechanical that you could easily imagine the compiler doing it automatically.

## Slide 12

It's so mechanical that it suggests that the struct and the protocol are really just two versions of the same thing. If they’re the same thing, could we entirely replace one with the other?
So instead of initializing the AnyAnimal with an object that promises a function, we just initialize it with the function itself. It's really the same thing, isn't it? We can convert them back and forth. And each of these might be convenient in different situations.
Passing something that promises a function or pass the function? Where have we seen this before?

## Slide 13

Delegates and completion handlers. Cocoa has both, sometimes even in the same object. Because sometimes it’s convenient to pass an object that implements a function, a delegate. And sometimes it’s convenient to pass the function itself, a handler. One isn’t always better than the other. It depends.

## Slide 14

Short-lived (callback)
Ongoing (datasource)
These aren't hard and fast. There are good single-method protocols, and cases where you may call a function many times. But they've worked pretty well in deciding when to use blocks, so they should apply to deciding when to use functions in Swift.

## Slide 15

OK, so protocols are really just promises to provide some bundle of functions. And structs are mostly just bundles of functions. And we can pretty mechanically convert between protocols and structs. And we can pretty mechanically convert between struct methods and functions. So we can pick the mix of protocols, structs, and functions that best suits our needs and dodges our problems.

Let's see a more real-world example.

## Slide 16

I was doing a lot of work with NP Complete problems. Traveling salesman, knapsack packing, sudoku, they're really all just different versions of the same problem, and there a bunch of search algorithms that apply to all of them. Greedy, branch and bound, local search, bunch of them. And then there's all this scaffolding you need. Parse your input, manage parallel searchers, provide progress feedback. I was really getting tired of implementing it all over and over. I wondered if I could build a generic NP Complete solver that could mix and match problems with search algorithms.

## Slide 17

So I started with protocols. First I made a protocol for parsing that spits out some abstract problem. And then I made a protocol for searching, that converts an abstract problem space into some abstract solution. And finally I made protocol for outputting that takes a solution and displays it.

## Slide 18

So I can put it all together into this Solver structure. The trick is that the parser's output needs to match the searcher's input, and the searcher's output needs to match the outputter's input. The syntax is a bit heavy, and the resulting type is a bit complicated with three type parameters, but ok. It works.

But the problem is I had different Solvers I wanted. When you're developing your search algorithms, you want a simple, single-threaded solver with lots of debugging hooks. But when you're trying to solve a real problem with your searcher, you want a highly optimized, multi-threaded solver that doesn't waste time on anything that's not necessary. And I wanted to be able to explore different multi-threading approaches that require different solvers, but have them all fit into the same pretty user interface. I wanted a Solving protocol.

## Slide 19

So the protocol is pretty obvious. And I can add default methods on it in an extension like this.<build>
Seams simple, but it's got problems. The solve method is only available on Solving things that follow our rule that the types match across the three stages. But theres' no way in Swift to require that inside a protocol. So if I want to be able to call solve, I have to keep repeating that `where` constraint in every function or type that deals with Solving.

## Slide 20

And then you try to build an actual struct for it, and they all wind up looking like this. Good grief. I feel like I'm working in Java. I found this `where` clause cropping up all over the place. And this is just one layer with two simple rules. Imagine how this grows if you put a Solving into some other protocol.

It makes my head spin. Bah!

## Slide 21

What would Crusty do? Well, didn't Crusty get us into this trouble in the first place with all his fancy-schmancy protocols? Well maybe, but let's remember what he taught us. Let's get rid of unneeded complexity. Let's get rid of the layers of protocols and turn them into functions inside one protocol.

## Slide 22

That's pretty nice. I don't need a Parsing protocol. I just need a way to parse. I don't need a Searcher object. I just need a way to search. And by "a way to" I mean a function. Instead of four protocols, I have one, and it automatically ensures that my types are consistent. But it still has a problem. 

## Slide 23

Every Solver in my system has configurable parsers, searchers, and outputters, so they all wind up looking something like this.
Blah. That's unfortunate. The problem is that in Swift, a function property can't fulfill a protocol function requirement. It has to be a real function. So we need this intermediate underscore variable. But don't give up. We can still make this beautiful. There are a couple of ways.

## Slide 24

First, we could just change our protocol like this. Instead of requiring functions, we require function properties. Now all our solvers get much simpler. We don’t need all the intermediate properties to make the compiler happy. And this is pretty good. I like this.
But we could go further.


## Slide 25

Encapsulate what varies. Let’s revisit the problem.

## Slide 26

I have these different solvers, and they all take in a parser, a searcher, and an outputter. Then they have their own special functions and properties.
We’ve been thinking of these in inheritance terms. It’s a protocol, yes, but we’re really just treating protocols like multiple inheritance. What if we used composition instead?

## Slide 27

Instead of conforming to a three function protocol, accept a three-property struct. Now if you add a new piece, like a progress reporter, you don’t have to update all your solvers.

## Slide 28

No protocols at all. This is everything we need.
So we’ve gone from a four-protocol solution to a one-protocol solution to a no-protocol solution.
With the four-protocol solution, I had to create a lot of types, often just to wrap a single function, and the solver needed three type parameters, the parser, the searcher, and the outputter. Now, our solver requires fewer types, and is parameterized just on Problem and Solution. That makes a lot more sense.

## Slide 29

Parameterize on your data, not your handlers.

## Slide 30

So where does that get us? I started with lots of protocols and now there are none. Am I saying protocols are bad? No. Protocols are great. But you should think carefully about how you build your protocols and whether you should use them. Sometimes you should just pass a function, and sometimes you should replace a protocol with a generic struct. Swift makes it pretty easy to move between them, not as easy as I'd like, but pretty easy.

## Slide 31

Try refactoring your data structures different ways before you get too deep into your project. Encapsulate what varies. And Parameterize on Data, not Handlers.

## Slide 32

I think that would make Crusty proud, with or without protocols.