In our last blog series, we discussed how testing across module boundaries could be made easier by creating a Behaviour for a collaborating module, then utilizing the wonderful framework Mox to substitute a lightweight mock module in tests.
This approach is well and good when you have very concrete module boundaries that are well-defined and coarse enough to warrant the ceremony of wiring up a behavior for a module boundary. But what if we aren't necessarily interested in all the work in creating a mock, and we need something simpler and more lightweight?
Join us as we discuss some alternative ways to write lightweight tests across function or module boundaries.
Dependency Injection (or DI) is a software development system capability popularized in the object-oriented world, in which a software system provides a framework for accessing system dependencies. The test framework in these systems oftentime hooks into this dependency injection framework in order to replace live dependencies with test doubles. Practitioners of TDD (like us here at Carbon Five!) use test doubles in our tests to write concise, focused unit tests that verify behaviors with collaborating concepts. Additionally, using test doubles lets us test-drive out certain feature paths without actually having to have first implemented our collaborating dependencies.
In our prior post, we leveraged the use of module attributes to do the "injection" of the dependency:
module MyApp.MyModule do
@my_collaborating_module Application.get_env(:my_app, :my_collaborating_module)
def perform() do
@my_collaborating_module.do_something()
end
endSo in this case, the module attribute lookup in tandem with a system configuration setting is the manner in which we inject the MyCollaboratingModule dependency.
However, there are other more lightweight ways to do dependency injection. Let's look at a few others:
In Jose Valim's popular "Mocks and Explicit Contracts" blog post, he discusses a very lightweight opportunity to decouple your dependencies. Say your app references an implicit dependency on MyApp.MyCollaboratingModule:
module MyApp.MyModule do
def perform do
MyApp.MyCollaboratingModule.do_something()
end
endThis makes your tests difficult to write, because then your tests will run everything under the hood of MyCollaboratingModule. What if it's particularly slow, or hairy, complicated, or makes a call to an external service? Let's refactor a bit.
module MyApp.MyModule do
def perform(do_something \\ &MyApp.MyCollaboratingModule.do_something/0) do
do_something.()
end
endNow I can write my tests and pass in a stub function (or verifying function):
test "calls MyCollaboratingModule.do_something()" do
fake_do_something = fn -> send self(), :do_something_called end
MyApp.MyModule.perform(fake_do_something)
assert_received :do_something_called
endThere is a great simplicity in this approach!
You can also do the same thing, but instead of passing the function, you pass in the entire module:
module MyApp.MyModule do
def perform(collaborating_module \\ MyApp.MyCollaboratingModule) do
collaborating_module.do_something()
end
end
# And in the tests:
test "calls MyCollaboratingModule.do_something()" do
defmodule FakeCollaboratingModule do
def do_something do
send(self(), :do_something_called)
end
end
MyApp.MyModule.perform(FakeCollaboratingModule)
assert_received :do_something_called
endSimilarly, there is a nice simplicity to this approach
- Simple, intuitive and understandable
- Easily refactorable
- Code changes are localized to the file (no config changes)
- Most directly carries over from dependency injection approaches in other languages
- External dependencies become explicit
- Cluttered function interfaces can cripple readability and resist refactoring
- You may find you need to "wire" dependencies through multiple levels of functions
- Functions cannot be type-checked with Dialyzer (TODO: verify)
Let's imagine we begin with this software system - an HTTP endpoint that receives a HTTP request and builds a string to be read back in a text-to-voice system.
defmodule ResponseHandler do
def handle_response(
http_response,
parse_http_response \\ &HttpResponseParser.parse/1,
generate_speech \\ &SpeechGenerator.generate/1
) do
parse_http_response.(http_response)
|> do_more_work()
|> generate_speech.()
end
end
defmodule HttpResponseParser do
def parse(http_response, get_timezone_from \\ &TimezoneHelper.timezone_from/1) do
timezone = get_timezone_from(http_response.headers["DATE"])
more_things = do_more_parsing()
%ApiResponse{timezone: timezone, status: http_response.status, more_things: more_things}
end
end
defmodule SpeechGenerator do
def to_speech(%ApiResponse{} = api_response, to_spoken_time \\ &TimeUtils.to_spoken_time/1) do
speech_time = to_spoken_time.(api_response.time)
did_this_action = do_something_with(api_response)
"At #{speech_time}, I noticed that you #{did_this_action}"
end
endHere we have a 3-level-deep dependency graph. The ResponseHandler calls out to HttpResponseParser and SpeechGenerator collaborating modules. Those two modules in turn call out to the TimezoneHelper and TimeUtils modules, respectively. While this approach is fine, each level of the system has some degree of coupling to other parts of the system. Is there another way to reduce the complexity here?
In this new approach, we reduce the responsibility surface area of module functions as much as possible, preferring that, as much as possible, lower-level modules merely acts on data and structs. Any coordination with external collaborating modules is delegated to an outside coordinator. Observe:
defmodule ResponseHandler do
# handle_response/1 becomes a coordinating function
def handle_response(
http_response,
get_timezone_from \\ &TimezoneHelper.timezone_from/1,
parse_http_response \\ &HttpResponseParser.parse/2,
to_spoken_time \\ &TimeUtils.to_spoken_time/1,
generate_speech \\ &SpeechGenerator.generate_speech/2
) do
with timezone <- get_timezone_from.(http_response.headers["DATE"]),
parsed_api_response <- parse_http_response.(http_response, timezone),
speech_time <- to_spoken_time.(parsed_api_response.time) do
do_more_work(parsed_api_response)
|> generate_speech.(parsed_api_response, speech_time)
end
end
end
defmodule HttpResponseParser do
# Note how the prior call to TimezoneHelper.timezone_from/1 is pushed "up" to the coordinating function
def parse(http_response, timezone) do
more_things = do_more_parsing()
%ApiResponse{timezone: timezone, status: http_response.status, more_things: more_things}
end
end
defmodule SpeechGenerator do
# Note how the prior call to TimeUtils.to_spoken_time/1 is pushed "up" to the coordinating function
def to_speech(%ApiResponse{} = api_response, speech_time) do
did_this_action = do_something_with(api_response)
"At #{speech_time}, I noticed that you #{did_this_action}"
end
end- All dependencies are handled at the same level
- Collaborator modules can be "dumb" and perform pure functional transformations
- If the higher coordinating function can justifiably isolate all the dependencies, this is a powerful tool.
- Some of the abstractions we originally had at different levels may be appropriate to in hiding complexity.
- The coordinating module or function gains a high degree of complexity and becomes more difficult to maintain.
- Feels a bit more like sweeping the problem under the rug.
The next approach is a simple one, and it's mainly aimed at easing the ergonomics of having to wire through a handful of function references in the argument body.
Observe how difficult it would be to work with a function signature like this:
defmodule ResponseHandler do
# handle_response/1 becomes a coordinating function
def handle_response(
http_response,
get_timezone_from \\ &TimezoneHelper.timezone_from/1,
parse_http_response \\ &HttpResponseParser.parse/2,
to_spoken_time \\ &TimeUtils.to_spoken_time/1,
generate_speech \\ &SpeechGenerator.generate_speech/2
) do
do_something_with_dependency("arg", parse_http_response)
|> do_something_else_with_dependency("arg2", get_timezone_from, to_spoken_time)
end
endOh my goodness. Imagine any collaborating functions within this function declaration that then in turn need to have each specific function wired through its own function declaration. It's hairy enough to make refactoring a miserable experience.
What if we instead grouped this scattering of functions and grouped them all into a map called dependencies?
defmodule ResponseHandler do
# handle_response/1 becomes a coordinating function
def handle_response(
http_response,
dependencies \\ default_dependencies()
) do
do_something_with_dependency("arg", dependencies)
|> do_something_else_with_dependency("arg2", dependencies)
end
def default_dependencies() do
%{
get_timezone_from: &TimezoneHelper.timezone_from/1,
parse_http_response: &HttpResponseParser.parse/2,
to_spoken_time: &TimeUtils.to_spoken_time/1,
generate_speech: &SpeechGenerator.generate_speech/2
}
end
endThen in downstream collaborating functions, we simply route the entire dependencies map as a final argument.
Tests are simple to write too:
test "ResponseHandler.handle_response/1 does stuff" do
fake_dependencies = %{
get_timezone_from: fn _arg -> "America/Los_Angeles" end,
# more function doubles here ...
}
ResponseHandler.handle_response(http_response, fake_dependencies)
end- Easy to extend with additional dependencies
- Dependencies are defined inline, together with minimal fuss
- Less "wiring" overall
- May confuse the compiler
- Lose Dialyz-ability
Here we get into more powerful forms of the Elixir language. For dependencies that represent a cohesive capability, we can use Protocols to represent an action that otherwise would have been a collaborating function. Let's go back to one of our original examples:
defmodule ResponseHandler do
def handle_response(
http_response,
parse_http_response \\ &HttpResponseParser.parse/1,
generate_speech \\ &SpeechGenerator.generate/1
) do
parse_http_response.(http_response)
|> do_more_work()
|> generate_speech.()
end
endWhat if we re-thought our dependencies in terms of protocols? We begin by reformulating our dependency in terms of the conforming data structure we wish our protocol to provide:
defprotocol HttpParsing do
def parse_http_response(t, http_response)
end
defprotocol SpeechGeneration do
def generate_speech(t, response)
endThen we create a new struct that will provide the type to dispatch this new protocol on:
defmodule ResponseHandler.Dependencies do
defstruct []
endNow we implement this protocol for this new struct:
defimpl HttpParsing, for: ResponseHandler.Dependencies do
def parse_http_response(_t, http_response) do
HttpResponseParser.parse(http_response)
end
end
defimpl SpeechGeneration, for: ResponseHandler.Dependencies do
def generate_speech(_t, options) do
SpeechGenerator.generate(options)
end
endLet's turn back to our original implementating call site and see what it might look like with a data structure that implements our new HttpParseable protocol:
defmodule ResponseHandler do
def handle_response(
http_response,
dependencies \\ %ResponseHandler.Dependencies{}
) do
response = HttpParsing.parse_http_response(dependencies, http_response)
|> do_more_work()
SpeechGeneration.generate_speech(dependencies, response)
end
endSince the ResponseHandler.Dependencies struct has an implementation of both the HttpParsing and SpeechGeneration protocols, it goes ahead and supplies the real methods at runtime. In our tests, however, we can define the following inline in our test:
defmodule FakeDependencies do
defstruct []
end
defimpl HttpParsing, for: FakeDependencies do
def parse_http_response(_t, http_response) do
%{ ... } # fake response
end
end
defimpl SpeechGeneration, for: FakeDependencies do
def generate_speech(_t, options) do
%{ ... } # fake data
end
end
test "does the thing" do
response = ResponseHandler.handle_response(http_response, %FakeDependencies{})
# assertions...
end- Strict guarantees provided by Elixir protocols = compile-time safety
- Lighter-weight and more flexible than implementing Behaviours. Protocols are extensible for arbitrary data structures.
- Can leverage Dialyzer for type checking
- Still lots of ceremony! A protocol requires a definition, a struct for both fake and real implementations, and implementations for each.
- Since protocols are dispatched on the types of the first arguments, this leads to potentially awkward function signatures
There we have it - four alternative lightweight approaches to dependency injection in Elixir, spanning from the most naive to the most powerful; from the simplest to the most complex.
Which one works for your code? The answer, as it usually does, is that it depends on your use case. I encourage the reader to always do the simplest thing that could possibly work. Start by doing simple function passing in the arguments list.
When you find yourself needing more and more function callbacks in your argument list, ask yourself if you can defer these calls to an outside caller.
If you must use all these functions, then consider consolidating them into a simple data structure and passing the structure around.
Finally, leverage Elixir's protocol system if you want some really powerful compile-time checks against your collection of dependencies.
Of course, don't forget that you may also find it appropriate to model your dependencies as Behaviours and mock function calls using Mox.
What do you think? What's worked for you? Let us know on Twitter!
Many thanks to Carbon Five coworkers Hannah Howard, Craig Lyons, Erin Swenson-Healey and Will Ockelmann-Wagner for conversations around and input on this post.