What is Technical Debt? A Definition and Brief History
Editor’s note: This piece was originally published in 2014. It’s been so popular that we decided to update it in 2019.
If you’ve ever designed a large computer program, commissioned custom software, or engaged in a heated conversation with your IT department, you have probably come across the term “technical debt”. What is it? Why does it keep coming up?
Defining technical debt
Technical debt (or “tech debt”) describes the consequence of implementing partial or quick software development solutions instead of more time-consuming, long-term solutions. Managing technical debt effectively is a task that requires wisdom and balance on behalf of custom software developers and organizations.
The birth of technical debt
The debt metaphor traces back to Ward Cunningham, an American computer programmer who developed the first wiki, among many other important achievements. In a report from Wyatt Software for the 1992 OOPSLA conference, Cunningham wrote that a serious pitfall for software development is “the failure to consolidate.” At the time of writing the report, Cunningham was developing a portfolio management system. Financial debt was likely on his mind.
He elaborated in the report, “Although immature code may work fine and be completely acceptable to the customer, excess quantities [of immature code lead to] an inflexible product. Shipping first time code is like going into debt. A little debt speeds development, so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation.”
The impact of technical debt
Since Cunningham’s 1992 report, the software development community has embraced the definition of technical debt to explain that a software project is never really done. Intentional and unintentional technical debt piles up as teams improve and update software. And as technical debt accumulates, it earns interest.
Software developers have learned to manage technical debt to their advantage. For example, developers may apply short-fix code for a launch, then revisit the code later to make it more robust. While creating new features, teams often develop best practices that render former features outdated. Developers pay off that debt by aligning the old features with current best practices. Teams even manage tech debt by updating entire products to meet new coding practices.
The beauty of Cunningham’s metaphor is this: the longer you sit on technical debt of any kind, the more time-consuming and expensive it will be to fix. And just as avoiding loan payments for too long results in serious repercussions, so does avoiding paying off your tech debt.
A classic example of the huge potential impact of technical debt is the Y2K crisis. The decision to define years only by their last two digits (e.g., 1968 as “68”) was a remarkable space-saver in the early decades of computer processing when the price of digital storage was still astronomical. But when civilization reached the brink of a new century, programs were still unable to distinguish between 1968 and 2068. Over $300 billion was spent worldwide to fix the problem before it was too late. By many estimates, that cost would have been significantly lower if the software had been updated over time. There was ample opportunity, as a man named Bob Bemer first called attention to the problem back in the ’50s. Talk about a hard-learned lesson in managing technical debt.
Is tech debt really such a bad thing?
Many product managers and developers would say yes. And that’s an understandable perspective. A swift, deal-with-it-now solution is sometimes seen as a kludge — something that works today, but inevitably costs you time, energy, and money down the road because it was never designed to meet future requirements. In other words, it generates technical debt you’ll have to pay off later, via more design and more coding.
Meanwhile, a slower, thoroughly-planned, more difficult solution is often seen as a better investment and more reliable at a system level — a true foundational architecture, on top of which you can build with confidence over time. This is partly because it incurs less technical debt.
There’s a balance between these two perspectives. The above argument focuses on the software per se – but is the goal to create custom software that’s very close to some initial plan or vision, regardless of all other factors? Or is the goal to get the best possible outcome in a larger sense?
The outcome can sometimes get better, not worse, when you incur tech debt by implementing a faster solution. That superior outcome applies not just to developers or their employers, but also for users and the customer base. The basic reason is very simple: the quality of software isn’t just about its stability, features, and performance. It’s also about the total productivity, or value, that that software creates over time in the real world when it’s used.
Consider:
When applying agile development methodologies (more on that in a minute), teams can generate higher overall quality by addressing the Schedule and Resources aspects of this triangle as well as the Scope.
Yes, a faster approach will incur more technical debt — but managing that technical debt will often be worth it, because the Quality, as defined in the graphic, is higher. The organization, and all its customers or users, will receive a better overall outcome.
Agile, scrum, and technical debt
Cunningham’s definition of technical debt in 1992 grew out of one of the first extreme programming environments. Extreme programming is the grandfather of workflow processes like Agile and Scrum. Back then, Wyatt Software was experimenting with a non-waterfall method of programming, which allowed software projects to adapt to discoveries made during the development process. Whereas the waterfall workflow requires teams to complete an entire iteration before responding to accrued debt, these more dynamic workflows allow developers to pay back it back continuously in small increments. In keeping with the metaphor, these continuous payments help keep interest low and allow teams to take full advantage of the loan.
In a 2009 YouTube video, Cunningham revisited his debt metaphor and cleared up some frequent misunderstandings of the term. He explained that not all technical debt is bad at first. A well-timed loan can help you buy a house or start a business. Similarly, incurring a technical loan by rushing software out the door can be a smart choice. But it’s only a good idea if you make time to refactor and update the program in the near future.
He also clarified that tech debt should never be an excuse for sloppy programming. In Cunningham’s words, “I’m never in favor of writing code poorly, but I am in favor of writing code to reflect your current understanding of a problem, even if that understanding is partial.” Rather, some intentional technical debt allows for a complete first draft of work to be tested and reviewed in order to make the second draft even stronger.
Technical debt, when combined with agile development, can also yield benefits you never expected to get, because it fosters innovation, which can be transformative. Just as Thomas Edison had to discover 999 ways not to invent the light bulb before he was able to produce the perfect design, developers will often be well served to experiment with short-term solutions until the long-term architecture becomes apparent.
In this view, tech debt can actually be seen as a form of investment, and like all investments, it can yield a return of some sort. Cases will obviously vary, but in general, Martin Fowler’s Technical Debt Quadrant is a helpful guide for analyzing decisions.
Consider technical debt not as a thing to be avoided at all costs, but a thing to be considered as an option — and something to be managed effectively, if the decision is to incur it. Take on debt in a deliberate, prudent fashion, and then pay it down in a proactive way.
Managing change and technical debt
Regardless of the development framework, technical debt can look like a recipe for ongoing bugs and constant maintenance. In theory, this makes sense. Thankfully, things aren’t so simple for this single reason: most major software projects change, in ways great and small, both frequently and unpredictably during development. They change in interface, expected operating environment, user requirements and features, associated business goals, and in myriad other ways. This means if you apply a traditional BRUF (Big Requirements Up Front) development methodology, which isn’t very good at responding to change, it, too, is going to generate bugs, and a lot of them.
Furthermore, an estimated 45% of software features created in BRUF-style projects are never even used. The costs of developing those features generates no value for users – it doesn’t matter how well or poorly they’re coded. There may be less technical debt in a BRUF project, but is that really an optimized outcome.
Instead of hoping to avoid change (which virtually never works), plan to incorporate change as it comes, as gracefully and efficiently as possible. Sometimes, that means deliberately incurring and managing technical debt.
The current state of technical debt
More than two decades after Cunningham first introduced and defined the technical debt analogy, it is becoming mainstream. There are entire workshops, conferences, and websites dedicated to tech debt diagnostics and best practices. Other top developers have coined terms branching from the same metaphorical tree — like “deficit programming,” “Enron developing,” and “technical inflation” — to further explain the need to constantly revisit and update code. When it comes to long-term development projects, we are seeing more and more teams plan and budget for tech debt management. Even C-suite executives with minimal programming knowledge are beginning to accept that software is a living system that has to be maintained and adapted over time.
As the art and science of software development evolves, new coding practices continue to roll out at an accelerated pace. Each new best practice improves both the software and the programming environment in which that software is developed. Perhaps this is why developers and business leaders are experiencing greater awareness of the concepts surrounding technical debt. As programming undergoes continuous and frequent change, all projects face the need to pay technical debt in order to keep software current.
Just as a financial loan can help you buy the right house or start a lucrative business, technical debt can be a friendly by-product of software development best practices. But if you’re not vigilant about your loan payments, technical debt can and will pull you under.
Want to know more?
Start a conversation with us about how to make your team more successful in technical debt management. We’d love to hear from you.
Legacy software is an asset, but only if you know how to leverage it.
>> Download our free CodePath Assessment e-book for evaluating the health of legacy software assets so you can innovate risk-free.
FREE EBOOK: “The Four Reasons Software Modernizations Fail (and 12 Strategies for Success)”
Struggling to plan or complete a large financial services software rewrite? This is the guide for you.