The idea here is that we want to display stories from Hacker News progressively as they arrive.
First, let's model our application.
Elm applications usually follow the following structure:
initialModel : Model
actions : Signal Action
update : Action -> Model -> Model
view : Model -> Html
main =
Signal.map view
(Signal.folp update initialModel actions)Our Model will consists of a list of stories
type alias Model = List Storywhere a Story is just a record containing information about the Hacker News story (modeled after the JSON produced by Hacker News)
type alias Story =
{ by : String
, id : Int
, score : Int
, time : Int
, title : String
, type' : String
, url : String
}The initialModel in this case is pretty trivial, it is simply the empty list:
initialModel : Model
initialModel = []From there we can make a simple view function using elm-html to view the stories.
view : Model -> Html
view stories =
ul
[]
( List.map viewStory stories )
viewStory : Story -> Html
viewStory story =
li
[]
[ a [ href story.url ]
[ text story.title ]
]We simply view our model as a list of links where each link displays the title of the story and links to the url of the story.
Now we need to model the actions that will update the Model. Since we want to display stories progressively, we want to add each story one by one.
type alias Action = Maybe StoryThis action will represent the new story that will come to update the model.
The reason that we model this as a Maybe is because initially, we will start with no story, hence, with Nothing.
So, now that we have our actions, we can update our model.
update : Action -> Model -> Model
update maybeStory stories = case maybeStory of
Nothing -> stories
Just story -> stories ++ [ story ]If we get Nothing, we leave the model untouched. But if we do get a story, we add the story to the list of stories (hence adding a new story to our model).
Now, all that is left is to actually get the new stories from Hacker News.
We use elm-http to get the JSON from Hacker News.
We would like a Signal of Actions:
actions : Signal Actionwhere each new action will get fed into the system. Given that we have all the other pieces of the application, we just need to get these actions and we're done.
To do so, we will use tasks. A task describes an eventual computation (if you know JS promises, then tasks are roughly the same).
Examples of tasks :
- Making an HTTP request
- Saving files to a database
- Playing music
- Requesting a file from a user
These are all examples of stateful computations that you would like the program to perform at sometime. Keep in mind that tasks are merely descriptions of computations. This subtle detail will come into play later.
For now, let's focus on the task of making an HTTP request.
elm-http provides the get function
get : Json.Decoder a -> String -> Task Http.Error aget takes in a JSON decoder and a url, and returns a task. This task describes how to send a GET request to the url and then decoding its contents with the decoder.
There are two groups of urls we are interested in getting. First there is the url with a list of all of the story ids. Second, there are the actual stories whose urls are parametrized by story id.
The first url is this:
idsUrl : String
idsUrl =
"https://hacker-news.firebaseio.com/v0/topstories.json"This url simply contains a list of integers representing the story ids.
To get the contents we just need to decode a list of integers in JSON, which is done as follows
-- import Json.Decode as Json exposing (list, int)
intListDecoder : Json.Decoder (List Int)
intListDecoder = list intAnd now we can create our task to get the ids:
getIDs : Task Http.Error (List Int)
getIDs = get intListDecoder idsUrlNow that we have the ids, let's construct the individual story urls from the ids
storyUrl : Int -> String
storyUrl id =
"https://hacker-news.firebaseio.com/v0/item/"++ toString id ++ ".json"Since we want to get a story out of each of these urls, we need a means to decode a story from JSON.
To, recall, this is the story type:
type alias Story =
{ by : String
, id : Int
, score : Int
, time : Int
, title : String
, type' : String
, url : String
}Thankfully, this type mirrors exactly the json data, so it's not so hard to decode it. A very easy way is as follows:
-- import Json.Decode as Json exposing (int, string, (:=), object2)
-- andMap = object2 (<|)
storyDecoder : Json.Decoder Story
storyDecoder = Story
`map` ("by" := string)
`andMap` ("id" := int)
`andMap` ("score" := int)
`andMap` ("time" := int)
`andMap` ("title" := string)
`andMap` ("type" := string)
`andMap` ("url" := string)This approach to decoding is nice in that it guarantees type safety and reads relatively well. There's almost no need to understand the details to see how it works. Basically, what is happening is that this is an easy way of mapping a function over an arbitrary number of parameters (in this case, 7) where each parameter is itself a decoder.
Now that we have our decoder, let's create our tasks that will get the stories given an id.
getStory : Int -> Task Http.Error Story
getStory id =
get storyDecoder (storyUrl id)Combining the two tasks, we can then get our main task:
mainTask : Task Http.Error (List Story)
mainTask = getIDs
`andThen` \ids -> sequence (List.map getStory ids)What this task does is :
- Get the ids
- And then, given the ids, get all the stories in sequence
Where sequence takes a list of tasks and converts it into a single task that returns the result as a list:
sequence : List (Task error value) -> Task error (List value)The problem of this approach is that this task will get the stories one by one as opposed to in parallel, which is our goal. This means that the page may only show up after all the stories have arrived. This is a problem given that Hacker News provides 500 stories (we could be waiting for a looooong time).
But before we tackle this problem, let's first see how we can actually run this task, because, if you recall, a task is just a description of a computation and not a computation in of itself. It must be performed somehow and then we need to somehow get a Signal of Actions out of it.
A mailbox is an object with an address and a signal:
type alias Mailbox a =
{ address : Address a
, signal : Signal a
}This object is special in that you can send values to a given mailbox if you know its address. Sending a value to a mailbox will cause the signal to update with the sent value as the fresh value for the signal.
The easiest way to send a value to a Mailbox is with the send function:
send : Address a -> a -> Task error ()The send function takes an address and a value to send to the address and returns a task signifying that it has successfully sent the value. This is what allows us to communicate between the world of tasks and the world of signals.
We can then create mailboxes with the mailbox function:
mailbox : a -> Mailbox aThis will take an initial value for the signal inside the mailbox.
So, now we can create a mailbox for our new stories:
newStoryMailbox : Mailbox Action
newStoryMailbox =
mailbox NothingAnd we could modify our getStory to send the story to this new mailbox:
getStory : Int -> Task Http.Error ()
getStory id = get storyDecoder (storyUrl id)
`andThen` \story -> send newStoryMailbox.address (Just story)Note: this will cause mainTask to change type to Task Http.Error (List ())
And now we finally have our actions:
actions : Signal Action
actions =
newStoryMailbox.signalBut, we're not quite done yet. While this will allow us to send values from tasks to mailboxes and hence extract signals, this is still not enough to actually perform tasks. The way to do so if via ports.
Ports are Elm's way of communicating with the outside world. This mechanism is used to actually perform task.
To do so, we create a port with the port keyword and define it as something of type: Signal (Task error value)
port taskPort : Signal (Task error value)
port taskPort = ...Now, how do we get something of type signal of tasks? We have a task (mainTask) and we could create a signal of them if we created a mailbox for the main task
This would lead to:
mainTaskMailbox : Mailbox (Task Http.Error (List ()))
mainTaskMailbox =
mailbox mainTask
port mainTaskPort : Signal (Task Http.Error (List ()))
port mainTaskPort =
mainTaskMailbox.signal
By actually having the signal of tasks as port, we explicitly tell Elm that this is a task that should be run. This is very convenient because this means that you can have a single file detailing every single effectful computation performed by the program.
By creating a mailbox with a task, that task will be run as soon as the program starts.
Note: This mailbox is just a regular mailbox. Don't be fooled by the fact that it holds tasks. You can send tasks whenever you want to this mailbox. A good example of this is sending a new task on button click. Each new task will be run as soon as it is sent to the mailbox. Therefore, ports aren't so much tied to individual tasks as much as they are tied to mailboxes. The mailboxes feed the ports. Thus different sources may use the same mailbox.
Now we just need to write our main function:
main : Signal Html
main =
Signal.map view
( Signal.foldp update initialModel actions )And we have a running program... except that it takes forever to load because we are running each task in sequence... But the good news is that the stories are loading progressively, which is a win.
So, let's see how we can make these task go in parallel
The problem is with the sequence function. It performs a task, then waits for the task to complete, then performs the next task, waits for it to complete, then performs the next task, and so on...
We would like to send them all at once. If somehow there were some parallel equivalent to sequence.
Currently, there isn't any in the core library but it can be easily implemented. All we need is the spawn function:
spawn : Task x value -> Task y ThreadIDspawn tasks a task and then wraps it in its own thread. This thread isolates this task from other tasks and tasks in other threads and thus allows it to run independently.
Note: these threads are not real threads. Javascript is single threaded so there's only one real thread. Elm just can move between threads to work on each progressively.
So, what we want for parallel is to take a list of tasks, and wrap them all in their own threads.
parallel : List (Task x value) -> Task y (List ThreadID)
parallel tasks =
sequence (List.map spawn tasks)Here sequence is used to spawn each task one by one. The spawning is what is done in sequence, not the computation. The actual task is now in its own thread.
So, now, we can re-write our mainTask as follows:
mainTask : Task Http.Error (List ThreadID)
mainTask = getIDs
`andThen` \ids -> parallel (List.map getStory ids)Note: The types of mainTaskMailbox and mainTaskPort will have to be changed to Mailbox (Task Http.Error (List ThreadID)) and Signal (Task Http.Error (List ThreadID)) respectively
And now, watch as these stories update in parallel. Ah!