Published the 2020-08-12 on Willow's site

Unit tests, the curse worse than the cause

Criticizing unit tests is uncommon. Anyway I'll try to explain why I think we should stop writing them.

Unit testing is a practice that generally follow when you learning about the SOLID principle. Having a single responsibility per subject is commonly narrated to simplify testing.

class FooRepository
  def get_foos
    ...
  end
end

class GetFoosUseCase
  def initialize(foo_repo)
    @foo_repo = foo_repo
  end

  def call
    ...
  end
end

One of the approach to test this code is to write unit tests that will check every class independently. The idea is that if every tests passes, then the whole project should be fine. This structure, with injected dependency truly give the hand to the testes to create dedicated contexts around tested subjects.

class TestGetFoosUseCase
  mocked_repo = Mock.new.to("get_foos").reply(["a mocked foo"])

  subject = GetFoosUseCase.new(mocked_repo)

  def test_get_foos
    assert subject.get_foos == ["a mocked foo"]
  end
end

Do you think this test is stupid and tautological ? I think most of contextual tests actually are. Anyway, to write this kind of tests you got to know how the `GetFoosUseCase` works as you got to know that a call to `@foo_repo.get_foos` is done somewhere. That why I think the ideological "write tests first" is a myth in the unit test world.

So now my point is: You though very hard to write SOLID classes and method, limiting coupling as much as you can, and THEN you wrote a completely coupled test suite, as the tests got to know how the context around the subject is working.

Will force you to rewrite mocks in every test that use them, most of the time. This kind of test cost lot of time and are insane to maintain.

And this is not the only trade-off of unit tests

As you got to mimic the context, you'll write lot of code, generally way more than the code you actually want to check. A common "scientific" estimation is that a developer add 3 bugs every 100 lines of codes. Tests are code so if a test do not pass, which one between the code or the test do you trusts more? The SOLID code or the other one?

If think we should definitely stop thinking about code coverage but more about code value. What is the value of this code that mocks a dependency? Is it maintainable? Does this code will brings robustness to my project? And at what cost?

Furthermore, unit tests do not bring more robustness that functional ones.

class TestGetFoosUseCase
  def test_get_foos
    assert container("GetFoosUseCase").get_foos == ["a fixtured foo"]
  end
end

I think this test code got a huge value, way more than the previous one. It cover way more code as it will go through the `GetFoosUseCase` and the `FooRepository` classes. It will stay the same if you re-factorise both those classes so it is easier to maintain. You can add features in the `GetFoosUseCase` and this test will still be valid and tells you if it still return the fixtured foo.

Does this functional test is perfect? Not at all. But it will perfectly serve the automatic test suite purpose:

Both approach only test a ridiculous portion of the whole code flow possibilities. It is impossible to write tests that cover all cases. Having a 100% code coverage DO NOT MEAN that every cases are covered ! Don't be delusional about your tests. They'll fail you because bugs are bugs and you can't predict them. Write simple code. Evaluate your code value. It can be positive, bringing features, being maintainable, being debuggable. Or it can be negative, having a single almost useless purpose, being hard to understand, requiring regular rewrite.

RSS feed