Great testing advice from Jose from https://groups.google.com/forum/#!msg/elixir-lang-talk/oYXyvEUi64Y/vSq_euStCQAJ

part_1

So the proposed solution is to change production code to be "testable" and making production code to call Application configuration for every function call? This doesn't seem like a good option as it's including a unnecessary overhead to make something "testable".

It is improving the design of your code.

A test is a user of your API like any other code you write. One of the ideas behind TDD is that tests are code and no different from code. If you are saying "I don't want to make my code testable", you are saying "I don't want to decouple some modules" or "I don't want to think about the contract behind these components".

Just to clarify, there is nothing wrong with "not wanting to decouple some modules". You don't want to decouple every time you call the URI module, for example. But if we are talking about something as complex as an external API, a SSH connection or such, defining a explicit contract and passing the contract as argument is going to make your code wonders and make it easier to manage your app complexity. 

Let's suppose you want to consume the Twitter API in your app. Your integration tests should not mock HTTPoison calls. A very simple way to see this is: if you replace HTTPoison by another HTTP client, should your integration test suite break? Probably not, your app behaviour is still be the same.

Therfore the best course of action is define the Twitter API your application needs in a behaviour:

defmodule MyApp.Twitter do
  @doc "..."
  @callback get_username(username :: String.t) :: %MyApp.Twitter.User{}
  @doc "..."
  @callback followers_for(username :: String.t) :: [%MyApp.Twitter.User{}]
end

Now *you know in a single place all you need from Twitter*. And every time you need to use Twitter,  receive the API as argument or via application configuration:

def twitter_api do
  Application.get_env(:my_app, :twitter_api)
end

Which can be configured per environment as:

# For dev and test
config :my_app, :twitter_api, MyApp.Twitter.Sandbox

Then testing it is a straight-forward consequence of good design.

The best is that you also be able to unit test your Twitter client for production cleanly, let's call it MyApp.Twitter.Prod, because all you need from twitter and what you expect as return values is in a single place and not sprinkled throughout your code. Use the @tag system in ExUnit to not run those tests by default but enable them in your build system:

defmodule MyApp.Twitter.ProdTest do
  use ExUnit.Case, async: true

  # All tests will ping the twitter API
  @moduletag :twitter_api

  ...
end

And in your test helper:

ExUnit.configure exclude: [:twitter_api]

And in your build system:

mix test --include twitter_api

Finally, the rest of your application absolutely does not care what you use to fetch the tweets. This is a good sign you have found the proper boundary between the components.

Note you relying on similar designs *all the time*: when you are using Plug, when you are using Ecto, when testing Phoenix channels, etc.

PS: the overhead is also minimum. Application configuration is stored in ETS tables... which means they are just directly read from memory.

part_2

One final note: part of testing your system is to find the proper contracts and proper boundaries between the components. If you define the rule that you will only mock if you define a explicit contract, it will:

1. protect you from overmocking because defining contracts for all interactions in your software is verbose and unnecessary (as said, you very likely don't want to hide interactions with the URI module behind a contract)

2. it will make testing your system easier because the contract is explicit (as per the previous post)

3. it will make testing easier because it will push you to move interaction between complex components to a single place.

Take a web application where usually [database] <-> [model] <-> [controller]. This is a pain to test because the complex component, which is the database, is hidden behind two layers, the model and the controller. That's why folks tend to isolate the model and the database. And then the controller and the model. This is such a pain.

Instead, we should make the model layer pure, to work only with data, and move the coordination between complex elements in the controller: [model] <-> [controller] <-> [database]. Then, if you need to isolate from components, it is all in the controller. It already handles the complexity that is the connection, so it is a natural fit for the database, mailer systems... anything that relies on side-effects.

With explicit contracts, you will have the opportunity to ask: maybe I should this contract up in my system instead of burying it inside layers. Your application will always have complexity. So make it explicit and try to manage it a single place instead of everywhere.

4. and it will make it easier to manage the complexity between the components themselves. Because every time you need a new function from twitter, you need to add it to the @callback list. If the list is getting bigger and bigger, it will be obvious and you will be able to act on it

part_3

Mind if I challenge you with some use cases?

Definitely.  :D
 
I'm starting to come around to this idea, but I still am not clear how to handle the following:

The challenge is to find the proper boundaries. I am assuming that, at this point, we all agree that [MyApp] <-> [Twitter API] is a poor boundary and that [MyApp] <-> [MyApp.Twitter contract] and then [MyApp.Twitter impl for prod] <-> [Twitter API] are better ones.

Therefore, we have effectively removed the dependency of MyApp on the actual Twitter HTTP client which effectively solves both the issues you mentioned below from the perspective of *your application*.

However, we still need to test [My Twitter contract for prod] <-> [Twitter HTTP], right? Ideally, we would like those tests to actually hit twitter whenever we wanted to, because that's the only way to guarantee it works. However, in practice, it may not work. So this may actually be the use case for mocking the HTTP responses as long as we respect the previous properties:

1. if you change your http client, your whole application suite won't break but only the tests for MyApp.Twitter.Prod

2. Don't mock your http client, pass it as a dependency via the app environment, as we have done with the twitter api

3. You could define a contract for this case but it is probably unnecessary as it would be tied to your HTTP client anyway

What do you think?