A recent TDD for Embedded C attendee asks me “TDD does not help reduce the time I spend in the lab during system integration testing”
(This is asked in the context of embedded development. But the answer is half applicable to any development where there is integration.)
TDD should definitely help reduce the time you spend in integration. How does it do that? It helps you eliminate the non-integration problems before you get to the integration lab. An honest appraisal of what you do in integration may reveal that during integration you are finding problems you could have discovered before integration. It would help reduced integration time if you find fewer problems during integration. Don’t you think?
This cause/effect diagram helps to visualize the relationship of TDD to time spend in integration.
Implicit in your statement is that it is important to increase Integration Velocity. We don’t want to spend more time there than we have to. So integration velocity goes in the middle of the cause/effect diagram.
Find Defects During Integration
Following along in the diagram, when you Find Defects During Integration your Integration Velocity is decreased. In other words there is a opposite effect on the thing we want to improve when we find defects in integration that we could have found some other way.
If you find defects during integration, presumably some (if not all) need to be fixed. You may recall, or have experienced Debug Later Programming’s find the bug penalty (see The Physics of TDD). How long does it take to find a bug? That’s the unanswerable question. Every defect you prevent before integration potentially saves a lot of time during integration testing.
Not only do we make integration take longer, but we start to Inconvenience Others and Cause Delays. So we would like to increase integration velocity and decrease delays that take us off the timeline to deliver the product.
Because Manual Test is Unsustainable, each of those changes in the lab should include a full retest of the system. Without automated tests, you won’t have time to do a complete retest. (Did you know that 25% of the bugs in your bug list come from unintended consequences? The new change works while some random other feature is broken.)
With TDD you come a lot closer to doing a full retest whenever you run your regression test suite. Late changes become more practical and safe as the chance of unintended consequences is reduced. Reducing side-effect defects reduces the drag on Integration velocity.
Defects found during integration do not just impact the individual. They also impact others involved in the integration as well as the goals of the company.
Find Defects Before Integration
TDD helps find defects before integration. It follows that when TDD finds defects before integration, there will be fewer to find during integration. This improves Integration Velocity.
Is it any different for Embedded?
So far, the discussion applies to embedded and non-embedded alike. The savings in the lab can be even greater given the embedded hardware bottleneck that most embedded developers have to deal with.
Before my TDD training, I ask people how they do their testing. With few exceptions, people report that they do manual tests on the target. Some code must be manually tested on the target, but in my experience much more could be tested off-target in a friendlier and faster development environment.
I also ask how much time they spend in the edit/build/download cycle. The numbers I get range from less than a minute to up to a day! One to five minutes is the most common reply. Every small change incurs that cost. If you are finding problems that could have been prevented, the target hardware bottleneck costs can accumulate quickly, and insidiously.
In embedded it can be hard to tell if a problem is in the hardware of the software. The test-driven developer will have a comprehensive set of unit tests that help them to see what their code is doing. This can be helpful when looking for problems because the software has a reference point on how it runs off the target in the test environment. It won’t always mean there is a hardware problem, but it should make it easier to track down the problem and shorten the time in integration.
Not a new Problem
In the beginning of my career, one of the big problems was system integration. As it turns out it is still a big problem in our industry. Nicely the incremental approach integral to TDD helps chip away at the problem. TDD is incremental in nature. It means you have an opportunity to integrate early and often.
It has been known for decades that the sooner you find a problem the easier it is to fix. TDD takes that idea and applies it to the minute to minute activity of programming. Not that TDD is only about defect prevention, but each defect prevented can potentially save considerable time in the future, including time spent in integration.