CUTTER IT JOURNAL VOL. 29, NO. 3
As an Agile coach and consultant, I hear a lot about technical debt from my clients. In fact, it is a major issue for almost all of them. In the 2011-2012 CRASH Report from CAST Software, the remediation cost for technical debt was estimated at $3.61 per line of code. I believe the actual financial impact to be much higher, as the report did not include secondary impacts such as increased time to market or the cost of replacing disgruntled software developers who grew tired of wrangling legacy code and moved on.
If there is rampant technical debt with most software products and services, why don’t organizations do something about it? Well, it’s complicated. Let’s take a step back and look at what we mean by technical debt and how we can measure it. Then let’s take a longer look at why technical debt exists and what you can do to address it.
WHAT IS TECHNICAL DEBT?
As most — if not all — of us know, it was Cutter Fellow Ward Cunningham who coined the term “technical debt.” Currently, there is an initiative within the Agile Alliance to further define and recommend ways to address technical debt. In a soon-to-be-released white paper, the group states the following:
When taking shortcuts and delivering code that is not quite right for the programming task of the moment, a development team incurs Technical Debt. This debt decreases productivity. This loss of productivity is the interest of the Technical Debt.
It can be hard to communicate the size and impact of technical debt. If you were to wander into the kitchen of a restaurant and see grease on the floors, flickering fluorescent bulbs, dirty dishes piled up, and mouse droppings everywhere, you would be appalled. You don’t need to be a foodie or restaurant owner to recognize that there is something seriously wrong and that it had better improve fast or the restaurant will be in serious trouble. It is difficult for most people to see technical debt in a similar way. I have resorted to printing out large methods or classes and taping them on the wall to make the complexity visible. However, this does not evoke the same visceral reaction or sense of urgency that a filthy kitchen does.
Technical Measures
There are static analysis tools you can use to measure technical debt, such as SQALE, FindBugs, PMD, and Code Climate. These tools can help provide insights, especially when you observe trends in the generated metrics over time (e.g., reduced cyclomatic complexity over time could be a sign that technical debt is decreasing).
Team Measures
Often, technical measures don’t provide a complete picture. For example, if there is legacy code rife with technical debt that rarely needs to be changed, then the technical metrics alone may provide an overly pessimistic view. For this reason, I recommend that teams also measure and track responses to the following questions:
- On a scale of 1 to 5, how happy are you working in the code base?
- What percentage of your time do you spend working around technical debt versus working on new features?
- How long do you feel it would take to address the main technical debt in the code?
You can measure these things right away with no infrastructure or tooling costs.
Make Your Technical Debt Visible
However you choose to measure technical debt, it is crucial to find ways to make the measures visible. Perhaps your teams could share technical measures and trend graphs at iteration reviews and planning sessions. Technical debt measures and trends could be “information radiators” in the team rooms.
WHY IS THERE RAMPANT TECHNICAL DEBT?
Technical Debt Is Normal, Isn’t It?
While technical debt as a metaphor is clear and strong, like any metaphor it has limitations. Most of us incur financial debt during our lives, and it is often the right thing to do. Most of us could not own a house or a car without incurring debt. So having technical debt, by analogy, seems like a normal state of affairs. No big deal. Shrug. Suck it up!
This attitude leads managers and decisions makers to not take technical debt seriously. Do not fall into this trap. Instead, include technical debt in project and portfolio planning. Include specific mechanisms to measure, track, avoid, and mitigate technical debt.
Technical Abilities
I was a professional programmer and independent consultant for 20 years before I was introduced to Agile methods. The level of craftsmanship I mastered in the following five years of applying Agile technical practices eclipsed everything I had previously learned at university and on the job. The level of craftsmanship in many companies is simply not up to the challenge of building products and services at Internet pace.
When I work with recent graduates, I usually ask them if they have been exposed to any of the Agile practices during their undergraduate work. The answer is almost always disappointing. Something like “We learned about Scrum — you know, the stand-ups, user stories, scrum master, and that stuff. But we never learned about test-driven development or simple design.” As a result, the code they write is usually complex and overdocumented. They have learned that they get the best marks when they stop coding the moment their program works and then document the crap out of it. Thus, the seeds of technical debt are born.
Technical Practices
Another trap I have seen organizations fall into is using Scrum without any supporting Agile engineering practices. Scrum provides tools for managing new features. However, there are no explicit mechanisms for managing technical debt work and no guidance on technical practices to deal with technical debt. This is a potent one-two combination that can result in Scrum teams building feature after feature as the technical debt mounts. Furthermore, this accumulation is often invisible because the teams naturally increase their story points to factor in the technical debt so their velocity stays the same. Everything looks fine as the big ball of mud grows and grows. Scrum without strong technical practices is a great way to build massive technical debt.
Silver Bullets
For most developers, working on a greenfield project is nirvana. We don’t have to deal with past mistakes and can often start with the coolest new technologies that will help us avoid some of the pitfalls we encountered in earlier projects. However, there are often high expectations of how quickly we can deliver, especially if the new technology choices were framed as more effective than earlier ones.
As we start the project, we are learning these new technologies and possibly about new markets and competitors as well. As a result, there will be pressure to deliver that will likely cause us to take on technical debt for the first release. But it’s not such a big deal, we tell ourselves, because we rocked the first release.
By the time we get to the second release, there is again pressure to deliver at a rate matching our pace for the first release. This can be difficult, though, as we have already accrued some technical debt. The predictable result is that we likely take on even more technical debt.
You can see where this is going. Fast-forward a few years and a few releases, and the project will be mired in technical debt. It will take forever to get new features out.
At this point, the organization will take stock of its options. Probably someone will suggest some cool new technology that will address a lot of the shortcomings of the current product. A rewrite will be proposed, and the organization will start the cycle over again. And this new project is doomed to the same fate because the organization has failed to address the root cause of technical debt — which is not taking it seriously in the first place. Very sad. Very wasteful. Very avoidable.
When first starting new products, organizations must take technical debt into account. They need to measure and track technical debt and mercilessly refactor the code on a regular basis.
Projects
Projects are one of the prime contributors to technical debt. Projects are fine for … well … project work. If the nature of your work is relatively short term with a clearly defined end of life, then projects may be a suitable management approach. If instead you build products or services that will require future releases and ongoing support, then projects are likely the worst way to manage this work. This is because every decision will be tuned to meeting the next project deadline. There will be no mechanism to optimize for the long-term sustainability of the product or service. Despite the best intentions of people working on the project, the technical debt will mount, because accruing this debt will not negatively impact the project delivery date. In fact, taking on technical debt is often the best way to achieve project success: “Let’s just hack this out, and we’ll fix it later.” Later never comes, however, and the cycle repeats with the next project.
To break this cycle, organizations need to shift planning away from projects and toward products and services. This allows planning horizons to extend beyond typical project timelines. The extended timelines enable organizations to staff appropriately for the product timeline. The teams can now make better long-term decisions about technical debt because their focus shifts to the long-term viability of the product or service.
Handoffs
Our current organizational structures reflect an emphasis on functional roles where we place people doing similar work together — as on an assembly line. This may have been a good way to build cars many decades ago, but it is wholly ineffective for responding at Internet pace to the ever-changing context in which information products and services live.
As information flows from customers through the organization’s process, information is inevitably lost. As information is lost and the systems are built, the developers will make assumptions or incorrect decisions about how the business works or how the system should operate. Each incorrect assumption or abstraction is embodied as technical debt in the code. In this way, technical debt accumulates around the knowledge loss in organizational handoffs.
Dynamic Inconsistency
In economics, dynamic inconsistency describes how our preferences for certain decisions may change at different points in time. For example:
Scenario A: Choose between:
A1 One apple today
A2 Two apples tomorrow
Scenario B: Choose between:
B1 One apple in a year
B2 Two apples in a year plus one day
While some people might pick A1, almost no one would select B1. However, the scenarios are formally identical. We exhibit dynamic inconsistency if we choose A1 today and in 364 days choose B2 instead.
With technical debt, we know that we should be taking care of it all through the project. However, our bias for dynamic inconsistency can lead us to build new features each iteration with little attention to technical debt. This is “double trouble,” because new code must coexist with the existing technical debt, and over time we end up with tangled layers of technical debt. And the interest on the technical debt compounds over time.
ADDRESSING TECHNICAL DEBT
I no longer think of technical debt as a problem. It is a symptom — a symptom of deeper system problems in our organizations. Trying to fix technical debt by simply fixing the code is like bailing a boat that is taking on water. It is likely necessary, but it won’t stop the water coming in. We need to find and fix the root causes of the technical debt. There are no silver bullets, but here are some things to consider:
Learn How to Write Cleaner Code
Support teams in improving their craft. Team-based learning is ideal, as the team members can support each other. Invest in technical training and encourage the use of pair and mob programming so that good practices spread and individuals can learn together.
Relentless Refactoring
Technical debt metrics may point to poorly written code, but if that code rarely changes, should we really invest time to make that code better? Our efforts might be better directed to improving code that changes more frequently. A simple tactic popularized by Robert Martin is to apply the Boy Scout Rule:
Leave the campground cleaner than you found it.
This is a simple and powerful approach that focuses improvements on the areas of code that change the most. Teams could incorporate this rule into their working agreements.
The improvements could be very small, such as renaming a variable or removing an unnecessary comment. Yet these improvements will compound over time.
Planning Technical Debt Work
Some organizations may feel sufficient pain from technical debt that they want to address it more quickly than application of the Boy Scout Rule alone can accomplish. Some teams create technical debt reduction stories and negotiate with their product owners to allocate a percentage of their capacity to technical debt reduction. I have seen teams negotiate for technical debt iterations. One interesting twist on this is to keep a separate backlog of technical debt tasks. Then, when a feature is being developed in that area of the code, pull the technical debt task into the work for that feature. Whatever you choose to do, make that work visible.
Proceed with Caution
Removing technical debt from legacy code that does not have tests can be unsafe. Your teams may need to learn and apply new techniques such as the approaches introduced by Michael Feathers in his classic book Working Effectively with Legacy Code. One technique Feathers suggests to extract existing code into new methods and classes (Extract Method and Extract Class refactorings) to build oases of code that can then be unit tested.
Retrospectives
Each team should run a retrospective focused on exploring technical debt. Have team members provide measures of the technical debt, root causes, and what they can do as a team about the situation. Then ask them to provide a list of obstacles they see to improving the situation and any suggestions they may have for overcoming them. Do the same at the middle management and executive levels. Collate and compare the results and support the teams and management in putting technical debt remediation measures into place.
Form Stable Teams
When teams have an extended responsibility to maintain products beyond project timelines, they will start to make better longer-term decisions, since they will have to live with the consequences. This will have a positive impact on technical debt. As a bonus, people will have more pride in their work, and there will be better onboarding for new team members.
Avoid Handoffs
Organize teams around the value flow and customer communication channels rather than around team functions. In other words, build teams that reach as close to the customers as possible both in understanding their needs and in delivering the product or service to them. Every handoff is a place where knowledge and value is lost.
Introduce Experiments
No matter how you decide to tackle technical debt, frame each change as an experiment rather than “rolling out” wholesale changes to the organization. Use retrospectives to guide experiments. Form a hypothesis, run the experiment, and measure the results. Most importantly, reflect on the results and let that new knowledge guide your next experiment. Create a backlog of experiments as a Kanban board to focus on sustainable change.
For example:
Action: Provide Agile technical training to three teams.
Hypothesis: Agile technical training will have a positive impact on technical debt, as teams will have better insights and skills for managing legacy code. We expect to see team happiness about working in the code improve by 1 point (on a scale of 1 to 5) after applying these techniques for two months.
WRAPPING UP
In 1967 Melvin Conway posited that “organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations” (aka Conway’s Law).
Technical debt is likewise a reflection of weaknesses in the organization’s communication channels. Treat technical debt as a symptom of larger systemic issues within your organization. It is giving you valuable feedback to better understand how to improve your product and service delivery.
Use retrospectives to tease out that feedback in a coherent way across multiple teams and management levels. Measure your technical debt and make the metrics and the debt itself visible. Perform experiments to reduce the impact of technical debt. Rinse and repeat.
