The tragedy of 100% code coverage

This article originally appeared on IG’s blog

It is funny how things turn around. For fifteen years I have been preaching TDD, or at least for developers to write some unit tests. However, in recent times I have found myself saying more often, “Why did you write that test?” instead of, “You should write a test.”

What is going on?

While walking around the office, I was asked by a developer to help him with some unit tests. It seems that he had trouble using Mockito to test the following piece of code:

I think he was very surprised with my response: “You don’t need to test that.”

“But I have to!” he said. “How do I know then if the code works?!”

“The code is obvious. There are no conditionals, no loops, no transformations, nothing. The code is just a little bit of plain old glue code.”

“But without a test, anybody can come, make a change and break the code!”

“Look, if that imaginary evil/clueless developer comes and breaks that simple code, what do you think he will do if a related unit test breaks? He will just delete it.”

“But what if you had to write the test?”

“In that case, this is how I would test it:”

“But you are not using Mockito!”

“So what? Mockito is not helping you. Quite the opposite: it is getting in your way and it is not going to make the test more readable or simpler.”

“But we decided to use Mockito for all the tests!”

Me: “…”

Next time that I bumped into him, he proudly stated that he had managed to write the test with Mockito. I understand the mental satisfaction of getting it working, but nonetheless it made me sad.

Another example

I got pulled in by a developer all excited about the high code coverage of one of their new applications and their new found love for BDD. Looking around the code we found the following Cucumber test:

If you have used Cucumber before, you will not be surprised about the amount of supporting code that it needs:

And all of that to test:

Yes, a simple map lookup.

I had enough trust with the developer to bluntly say, “That is a big waste of time.”

“But my boss expects me to write test for all classes,” he replied.

“At the expense of?”

“Expense?”

“Anyway, those tests have nothing to do with BDD.”

“I know, but we decided to use Cucumber for all tests”

Me: “…”

I understand the mental satisfaction of bending the tools to your will, but nonetheless it made me sad.

Where is the tragedy?

The tragedy is that two bright developers (both of whom I would take to a team interview) are wasting time writing those kinds of tests, tests that are pointless, and that will need to be maintained by future generations of IG developers.

The tragedy is that instead of using the correct tool for the job, we decide to keep plugging away with the wrong ones, for no particular good reason.

The tragedy is that once a “good practice” becomes mainstream we seem to forget how it came to be, what its benefits are, and most importantly, what the cost of using it is.

Instead, we just mechanically apply it without too much thought, which usually means that we end up with at best mediocre results, losing most of the benefits but paying all (or even more) of the cost. In my experience writing good unit tests is hard work.

So is 100% code coverage worth pursuing?

Yes, everybody should achieve it … in one project. I am of the opinion that you have to go to the extreme to know what the limit is.

We already have plenty of experience of one extreme: projects that have 0 unit tests, so we know the pain of working on those. What we are usually lacking is the experience in the other extreme: projects where a 100% code coverage is enforced and everything is TDD.

Unit testing (especially the test first approach) is a very good practice but we should learn which tests are useful and which ones are counterproductive.

But remember nothing is free, nothing is a silver bullet. Stop and think.