The (Real) Cost of Software Testing

An airline ticket for $24.82? Unheard of. Yet that was my actual cost from Portland to Austin, when I joined the Praxent team. Although my ticket only cost me $24.82, it cost the airline a whole lot more.

When I say the phrase, “software testing,” do you immediately think, “extra work?” You are not alone. Testing is commonly considered an additional step separate from development; and, in a world of budget constraints, the testing budget is often cut to save money. As Delta can attest, however, there is a real cost to neglecting quality assurance testing.

You may have heard about Delta’s software mishap with their ticket price algorithms in December 2013 (or United making a very similar mistake that September). This unbelievably inexpensive ticket was a great boon to me, but this simple bug— which was fixed after only 15 minutes—cost Delta airlines a lot. How did it happen? Delta Airlines lacked proper test coverage.

Instead of thinking of testing as additional work, think of testing as writing specifications (specs). If you write specs before building software, you can prevent costly mistakes. Furthermore, the execution of software tests can be automated, which can save hours over manual testing.

The engineers at Delta and United (or any software development shop) cannot devote hours to testing every possible purchase scenario manually; that would be a serious misuse of time. Instead, automated tests can quickly ensure that what we write is what we are supposed to write, and that changes in the code don’t break anything written in the past.

All change is risky. With today’s complex systems, a single change in code could break the system elsewhere.  There isn’t enough time or budget to test that every feature still functions as expected after introducing code changes. At Praxent, we mitigate risk, save time, and reduce cost by using a computer to validate the specs. This is called continuous integration, and it helps us prevent costly and embarrassing bugs going out into production.

You wouldn’t dream of opening a car factory without the right blueprints, and you wouldn’t send a web product out to users without the proper specifications (or specs) to make ensure everything is working correctly. The cost isn’t in testing; the real cost – as Delta Airlines learned – is in not testing.