Elixir Property-Style Tests

Property-style tests check a rule that must hold for every input in a fixed list. One property replaces many single examples.

A property checks every input, not one example

Reversing a list twice returns the original list. That rule must hold for each list in the fixed set.

ExUnit.start(seed: 0, auto_run: false, formatters: [])

defmodule PropertyCheck do
  def run(inputs, property) do
    passed = Enum.count(inputs, property)
    {passed, length(inputs)}
  end
end

defmodule ReverseTest do
  use ExUnit.Case

  test "reversing twice returns the original" do
    {passed, total} =
      PropertyCheck.run([[3, 1, 2], [5], []], fn list ->
        Enum.reverse(Enum.reverse(list)) == list
      end)

    assert passed == total
    IO.puts("property passed: #{passed}/#{total}")
  end
end

result = ExUnit.run()
IO.puts("tests passed: #{result.total - result.failures}/#{result.total}")

The checker counts a failing input too

A property can fail for some inputs. The count shows how many inputs passed.

ExUnit.start(seed: 0, auto_run: false, formatters: [])

defmodule PropertyCheck do
  def run(inputs, property) do
    passed = Enum.count(inputs, property)
    {passed, length(inputs)}
  end
end

defmodule SquareTest do
  use ExUnit.Case

  test "squaring beats the input for larger numbers" do
    {passed, total} =
      PropertyCheck.run([0, 1, 2, 3], fn n -> n * n > n end)

    assert passed == 2
    IO.puts("property passed: #{passed}/#{total}")
  end
end

result = ExUnit.run()
IO.puts("tests passed: #{result.total - result.failures}/#{result.total}")

One list can cover edge cases together

Empty lists and single items belong in the same fixed list. The property must hold for all of them.

ExUnit.start(seed: 0, auto_run: false, formatters: [])

defmodule PropertyCheck do
  def run(inputs, property) do
    passed = Enum.count(inputs, property)
    {passed, length(inputs)}
  end
end

defmodule LengthTest do
  use ExUnit.Case

  test "reversing keeps the length" do
    {passed, total} =
      PropertyCheck.run([[], [7], [2, 1], [5, 5, 3]], fn list ->
        length(Enum.reverse(list)) == length(list)
      end)

    assert passed == total
    IO.puts("property passed: #{passed}/#{total}")
  end
end

result = ExUnit.run()
IO.puts("tests passed: #{result.total - result.failures}/#{result.total}")

Example

ExUnit.start(seed: 0, auto_run: false, formatters: [])

defmodule PropertyCheck do
  def run(inputs, property) do
    passed = Enum.count(inputs, property)
    {passed, length(inputs)}
  end
end

defmodule CombinedPropertyTest do
  use ExUnit.Case

  test "two properties hold over fixed lists" do
    {reverse_passed, reverse_total} =
      PropertyCheck.run([[3, 1], [9], []], fn list ->
        Enum.reverse(Enum.reverse(list)) == list
      end)

    {length_passed, length_total} =
      PropertyCheck.run([[3, 1, 2], [5, 4], []], fn list ->
        length(Enum.sort(list)) == length(list)
      end)

    assert reverse_passed == reverse_total
    assert length_passed == length_total
    IO.puts("reverse: #{reverse_passed}/#{reverse_total}")
    IO.puts("sorted length: #{length_passed}/#{length_total}")
  end
end

result = ExUnit.run()
IO.puts("tests passed: #{result.total - result.failures}/#{result.total}")

Key points

Property: a rule that must hold for every input, not just one example.
Invariant: the part of a result that stays the same across every input.
Fixed list: a chosen set of inputs, so the run gives the same result every time.
Checker: a function that runs a property over each input and counts how many passed.
PropertyCheck: a small module that checks a property over a list and returns the passed count and the total count.

In this example, PropertyCheck runs two properties over fixed lists. The first property checks that reversing a list twice returns the original list. The second property checks that sorting keeps the length of a list. The test asserts that every input passed for both properties.

Note

property-style tests here use fixed lists, not random inputs, so the output stays the same on every run.

Try it Yourself

Edit the code, then press Run.

Running is simulated

ExUnit.start(seed: 0, auto_run: false, formatters: [])

defmodule PropertyCheck do
  def run(inputs, property) do
    passed = Enum.count(inputs, property)
    {passed, length(inputs)}
  end
end

defmodule CombinedPropertyTest do
  use ExUnit.Case

  test "two properties hold over fixed lists" do
    {reverse_passed, reverse_total} =
      PropertyCheck.run([[3, 1], [9], []], fn list ->
        Enum.reverse(Enum.reverse(list)) == list
      end)

    {length_passed, length_total} =
      PropertyCheck.run([[3, 1, 2], [5, 4], []], fn list ->
        length(Enum.sort(list)) == length(list)
      end)

    assert reverse_passed == reverse_total
    assert length_passed == length_total
    IO.puts("reverse: #{reverse_passed}/#{reverse_total}")
    IO.puts("sorted length: #{length_passed}/#{length_total}")
  end
end

result = ExUnit.run()
IO.puts("tests passed: #{result.total - result.failures}/#{result.total}")

Exercise

Which property holds for Enum.sort for every list?

The sorted list is always longer than the original list. The sorted list always starts with the largest value. The sorted list has the same length as the original list. The sorted list always drops repeated values.