Writing testable and decoupled Elixir using the Adapter Pattern

The Problem

A lot of our code has to deal with external services to get some things done from them. For example, we might have to send an email, or make a request to an external API, or upload a file to S3. In such cases, we want to write code that is testable and decoupled from those external service. There’s two things we want to achieve here:

1. Testability: We want to be able to test our code without actually making a request to the external service. The external dependent code should be easily mock-able.
2. Decoupling & Extensibility: We want to be able to replace the external service with a different implementation without having to change any of our code that uses it.

In this post, I’ll go through how we are using Elixir Behaviours to achieve this.

The Solution

Behaviours in Elixir

In Elixir, we use Behaviours to define abstract interfaces. We can define a Behaviour and then implement it in different modules. We can then use that Behaviour in our code to accept any module that implements the Behaviour.

Then, for our tests, we can define a mock module that implements this Behaviour. And in test environment, we will use this mock module instead of the actual module that makes the calls to the external service. All our application logic code will remain the same and we can test it without actually making any calls to the external service.

Example

Let’s take an example. Say we are using Dynamodb in our application. We want to be able to mock the Dynamodb calls in our tests. We can define a Behaviour for Dynamodb and then implement it in a module that actually makes the calls to Dynamodb. We can then use this Behaviour in our code to accept any module that implements it.

defmodule MyApplication.Dynamodb do
  @type ok_tuple :: {:ok, any()}
  @type error_tuple :: {:error, any()}

  @moduledoc """
  The behaviour for dynamodb adapter
  """

  @doc """
    Put item
  """
  @callback put_item(table_name :: String.t(), record :: map()) :: ok_tuple() | error_tuple()
  @callback query(table_name :: String.t(), opts :: keyword()) :: ok_tuple() | error_tuple()

  defp impl, do: Application.get_env(:my_application, :adaptors)[:dynamodb]

  @spec put_item(String.t(), map()) :: ok_tuple() | error_tuple()
  def put_item(table_name, record), do: impl().put_item(table_name, record)

  @spec query(String.t(), keyword()) :: ok_tuple() | error_tuple()
  def query(table_name, opts), do: impl().query(table_name, opts)
end

Here, we have defined a Behaviour for Dynamodb. We have defined two functions, put_item and query. Rest of our app code will use this module to interact with Dynamodb. If you see, we are defining impl function which will return the module that implements the Behaviour. We will set this module in our application config.

Let’s see the module that will actually make the calls to Dynamodb.

defmodule MyApplication.DynamodbAdapter do
  require Logger
  alias ExAws.Dynamo

  @behaviour MyApplication.Dynamodb

  defp dynamodb_result(result, error_code) do
    case result do
      {:ok, r} ->
        {:ok, r}

      {:error, error} ->
        error |> Kernel.inspect() |> Logger.error()
        {:error, error_code}
    end
  end

  @impl true
  def put_item(table_name, record) do
    Dynamo.put_item(table_name, record)
    |> ExAws.request()
    |> dynamodb_result(:dynamodb_error)
  end

  @impl true
  def query(table_name, opts) do
    Dynamo.query(table_name, opts)
    |> ExAws.request()
    |> dynamodb_result(:dynamodb_error)
  end
end

This module makes the actual calls to DyanmoDb. Let’s set this module as the adapter in our application config for dev and prod environments. If you’re using Phoenix, you can set this in config/dev.exs and config/prod.exs.

# add in dev.exs and prod.exs
config :my_application, :adaptors, dynamodb: MyApplication.DynamodbAdapter

Now, in the rest of the app, whenever we want to, say query DynamoDB, we will use MyApplication.DynamoDb.query function. This function will call the impl function which will return the module that implements the Behaviour. In our case, it will return MyApplication.DynamodbAdapter, which will make the actual calls to DynamoDB.

What about tests? We can define a stub module that implements the Behaviour, and instead of making actual calls, it will just return some dummy data. We then use the Mox library to define the mock. Here are the steps ->

1. in test.exs, set the adapter value to mock adapter (we’ll create this in the next step)

# add in test.exs
config :my_application, :adaptors, dynamodb: MyApplication.DynamodbMock

2. Using Mox library, define the mock by specifying the behaviour module

# in test_helper.exs
# Dynamodb Mock
Mox.defmock(
  MyApplication.DynamodbMock,
  for: MyApplication.Dynamodb
)

3. Create a stub module (I like to create it in test/support/stubs directory) for the behaviour. It should use the Dyanmodb behaviour and should return test responses. Let’s say that stub module is called MyApplication.Stubs.Dynamodb.

defmodule MyApplication.Stubs.Dynamodb do
  @behaviour MyApplication.Dynamodb

  def put_item(table_name, record) do
    case table_name do
      "success_table" ->
        {:ok, record}

      _ ->
        {:error, :dynamodb_error}
    end
  end

  def query(_table_name, _opts) do
    {:ok,
     %{
       "Count" => 1,
       "Items" => [
         %{
           x: 1,
           y: 2
         }
       ],
       "ScannedCount" => 1
     }}
  end

4. Whenever the code you are using in tests is calling dynamo db, just call stub_with(MyApplication.DynamodbMock, MyApplication.Stubs.Dynamodb) in the setup of that test (make sure to import Mox). Here’s an example test case (it’s just testing the DynamoDb module, but in reality, we would be testing the application code that uses the DynamoDb module).

defmodule MyApplication.DynamodbTest do
  use ExUnit.Case
  import Mox

  setup do
    stub_with(MyApplication.DynamodbMock, MyApplication.Stubs.Dynamodb)
  end

  test "put_item" do
    assert {:ok, %{x: 1, y: 2}} = MyApplication.Dynamodb.put_item("success_table", %{x: 1, y: 2})
    assert {:error, :dynamodb_error} = MyApplication.Dynamodb.put_item("failure_table", %{x: 1, y: 2})
  end

  test "query" do
    assert {:ok, %{"Count" => 1, "Items" => [%{"x" => 1, "y" => 2}], "ScannedCount" => 1}} =
             MyApplication.Dynamodb.query("table_name", [])
  end
end

That’s it! Now, you can use this pattern to write extensible code in Elixir. You can also use this pattern to write adapters for other services like S3, SQS, etc.

References

• Mocks and explicit contracts
• Writing extensible Elixir with Behaviours