Tech Tuesday – The Importance of Testing – Coverage and Quality
30 August 2016 |
Chris Wintle | About a 4 minute read
Ask any developer about the importance of writing tests and you’ll usually get the same set of answers about how critical it is. Yet when you actually look at how how much code gets created commercially with no tests, this just doesn’t add up.
Across my career I’ve seen dozens of companies creating software with partial (or zero) test coverage. Or even more bafflingly, codebases with high test coverage where large amounts of bugs are still being uncovered after go-live! Yet I’ve never met a developer that doesn’t believe in testing – so what’s happening?
The answer naturally isn’t simple, but we can break it down into five key areas:
- Rapid feature delivery at all costs
Firstly, if the culture of your business is one of ‘feature delivery at the expense of all else’, your focus is never going to be on writing tests. If your whole company or team is geared around delivering as rapidly as possible, compromises will be made (and testing is one that often gets included here). In this kind of environment, developers feel ‘under siege’ and writing tests is just less important than getting through the day and getting the next feature out of the door.
In this scenario, it’s easy to say from a developer’s point of view ‘the business doesn’t want to focus on quality, so I can’t write tests’. This is partially true, but doesn’t tell the full story. There’s no single point of blame or responsibility.
A culture that values quality (and by extension, testing) needs to be driven in both directions, but it starts with the delivery team. As developers, part of our responsibility is to show the business the value of tests by being a center of excellence and delivering quality, tested work. It’s the job of product owners, testers and other stakeholders to help support this view with the rest of the business.
- Partial test coverage gives a false sense of security
To my mind, partial test coverage is more dangerous than zero test coverage. If you have no tests, you are going to be aware of that fact – and will (in theory) be more cautious when changing things – you might run more manual tests, for instance.
Partial test coverage (or indeed, high test coverage with low test quality) gives you the false sense of security that your codebase has ‘some’ tests, so you can relax a bit. It helps if you know which bits are and aren’t tested, but in a complex system that’s not always simple.
- Writing good, comprehensive tests is a learned skill
Writing comprehensive tests requires a mind-set that emphasises defensive coding – always thinking ‘how else can I break this?’. That kind of thinking requires hours and hours of practice and often requires training on exactly what kinds of tests we should write. Writing good tests is an art in itself! Without spending the time on training for this, we’re often not going to see quality tests being written.
- High test coverage is not the same as a thoroughly tested product
One of the great fallacies in modern software development is that code coverage is the be-all and end-all metric of code quality. This simply isn’t true. It is trivial to write a test that exercises huge chunks of code, but doesn’t actually test how robust that code is. A high code coverage score is easily achievable if that’s your only end goal. That’s not to say that code coverage isn’t a useful metric – just don’t use it as your end goal. If you’re coding defensively and testing as you go along (or ideally using TDD), your code coverage should be high anyway. Chasing a specific code coverage number as your ‘definition of done’ just encourages abuse of an easily abused system.
- Unit tests, integration tests, end-to-end tests
When we talk about test coverage, we’re often talking about unit test coverage, which says nothing about integration or end-to-end tests, both of which are equally important. Having 100% unit test coverage says nothing about the state of your UI (writing unit tests for your React components is a step in the right direction, though) or your database interaction layer.
The reality is that writing all of these sorts of tests is very time consuming – so one or two of them often get skipped. end-to-end tests are very brittle, especially in a rapidly changing UI, so they are often written off as being too much effort. The problem is by sticking to just one or two types of test, we aren’t really seeing the full story, so we come back to our issue with having a false sense of security. 100% unit test coverage is great, but if we don’t test our UI or component integration, we are still potentially missing user-facing bugs.
Wrapping it up
Naturally, your business may be suffering from any number of the issues we just talked about – and fixing them all is not easy, or fast. The important thing to take away is that as a collective whole, you decide what your testing culture should be and move on from there. With a healthy culture that values quality and testing, the actual tests themselves will follow fairly automatically and software quality should increase.Read More From This Author
Senior Magento Developer
Technologists, experts in their domain, inspirational mentors and coaches.
Technologies you will be using
Leader in Product Delivery AND passionate about developing others.I'm Interested