CUTTER IT JOURNAL VOL. 29, NO. 3
  

The term “technical debt” is generally used to describe the burden created by decisions to cut corners when designing and coding software. The catchy metaphor is attributed to Cutter Fellow Ward Cunningham, who helped us think about how quick-and-dirty solutions set us up for debt that has to be paid back with interest. Technical debt driven by software vendors is a less frequently discussed but significant variation on the theme.

I would like to broaden the conversation around tech­nical debt to include the challenges of keeping up with software vendors’ lifecycles. Such vendor-driven tech­nical debt requires the continual attention of CIOs and technology executives, who need to balance limited budgets to cope with technical debt.

In this article, we will examine aspects of the problem and evaluate some of the techniques to address and repay technical debt.

TECHNICAL DEBT IN CONTEXT

Technical debt attributable to vendor upgrade cycles is a topic of intense debate in corporate IT groups, as well as enterprise architecture forums and online communities. Enterprises of all sizes buy or license software products, solutions, and tools from vendors. These products range from small investments in worker productivity tools to large investments in ERP systems, CRM systems, databases, and specialized solutions designed to meet specific functional needs. The decision to implement a version of the software — for example, Oracle Database 12c Release 1 or SQL Server 2008 R2 or SAP ERP 6.0, EP 4 — is generally a strategic one, requiring considerable analysis, planning, and resources. Such decisions are taken at a particular point in time, while considering the organization’s business needs and constraints in the technology landscape.

Software vendors and solution providers continually upgrade their product offerings, promising newer technical and functional capabilities. In order to provide support, vendors expect clients to keep up with their upgrade cycles. Software support from a vendor is ­critical to IT operations in order to:

• Apply relevant security patches and technical fixes. The source code for licensed software is generally owned by the software vendor, which is in a position to apply relevant updates, patches, and technical fixes. Such patches — implemented and tested ­centrally — are pushed to all clients.
   
• Ensure compliance and controls over IT systems. CIOs are required to certify to their stakeholders that systems meet relevant regulatory and compliance requirements. For example, the US Sarbanes-Oxley Act (SOX) requires the CEOs and CFOs of public companies to attest to the accuracy of financial reports and establish adequate internal controls overfinancial reporting. Passage of SOX resulted in an increased focus on IT controls, as these support financial processing and therefore fall into the scope of management’s assessment of internal controls. Ensuring software systems are upgraded and managed as per vendor recommendations is a key aspect of compliance.

Upgrading to a newer version of software recommended by the vendor requires a deliberate impact assessment to understand the potential impact to systems upstream or downstream. Such an upgrade may have to be orchestrated with changes in the rest of the IT landscape; for example, during a large prescheduled program.

The vendor’s upgrade cycles may not align with the customer’s business or technology change cycle. Therefore, the organization’s IS and business stake­holders may consciously opt out of a vendor-driven upgrade cycle, in effect taking on a technical debt that will need to be repaid by upgrading later. After a few cycles of not upgrading, the software may fall behind the vendor’s support cycles, and the vendor may demand a penalty for supporting older versions. Some vendors call this “extended support,” and it can be expensive.¹ In some cases, after adequate notice, ­vendors may stop support of versions going back several generations, such as when Microsoft announced the end of the support lifecycle for Microsoft Windows NT 4.0 Server. Such action by the vendor may force the enterprise to repay the debt by upgrading the solution or looking for an alternative. Repaying such technical debt by planning an upgrade can be expensive and ­disruptive to business.

WHY DO ORGANIZATIONS TAKE ON TECHNICAL DEBT?

IT executives and leaders are increasingly using the technical debt metaphor to articulate the hidden costs and tradeoffs associated with decisions that impact business capabilities and technologies in the organization. Martin Fowler, a leading software expert, classifies technical debt into four types, along two axes: reckless or prudent, and deliberate or inadvertent.

Fowler’s elegant model is focused on software development, but it can be extended to take a broader view of the enterprise architecture and application portfolio (see Figure 1):

• Reckless/deliberate debt. Project teams, including managers and architects, may sometimes give in to time-to-market pressures without adequate analysis or forethought. Such decisions may be reckless, but they are deliberate. For example, a business unit might opt to implement a different version of a supply chain solution, unable to wait for the global ERP to be rolled out across the enterprise. Such a debt will have to be repaid when the standard version of a global solution is finally planned for implementation at the business unit.
   
• Prudent/deliberate debt. A project team may take ona deliberate short-term debt with clear plans to address it. Such a prudent decision may be taken to enable the business to react to an external market change. For example, toward the end of 2015, when the global price of oil suddenly dipped below US $30 a barrel, oil drillers began reacting by shelving projects and shutting operations. Across the oil and gas industry, some $400 billion in expected investment was ­cancelled or delayed. Amid widespread and abrupt cost-cutting pressures, executives may find it easy to postpone technology upgrade projects. In the oil and gas sector, IT executives are putting off technology upgrades, deliberately taking on technical debt that they hope to repay when the sector turns around.
   
• Reckless/inadvertent debt. This is a likely scenario inloosely governed organizations where teams are either ignorant of the consequences of taking on technical debt or recklessly flout the guiding principles. An example is that of a design team involved in a global SAP supply chain rollout that agrees to requests from the business to customize the solution, ignoring the fact that such custom development on a product will make future upgrades more difficult and expensive. In a review of enterprise systems, University of Pittsburgh researchers Narayan Ramasubbu and Chris F. Kemerer explain that ­”software update patches supplied by vendors as part of their maintenance plans could be incompatible with the customizations implemented by clients.”
   
• Prudent/inadvertent debt. Project teams may take a prudent decision that satisfies the functional needs at a point in time but may inadvertently miss other dependencies in the organization. An example is a global pricing program in North America that misses a key dependency with a corresponding global CRM solution implemented in Europe. Despite adequate due diligence, the inadvertent omission may result intechnical debt when the global solutions are implemented across the regions.
   

HOW TO MOVE FORWARD: ACKNOWLEDGE AND COMMUNICATE THE PROBLEM

Enterprise architects and IT executives recognize the problem of technical debt, but they may not have the resources and funding to deal with it. They need tools and techniques to communicate the problem to their stakeholders and engage with them.

In order to communicate with stakeholders, one needs to identify the magnitude and the span of the impact. In many cases, the impact of a vendor-recommended upgrade may be limited to a software platform used locally or regionally. Software platforms used across business units may have been designed to operate in aloosely coupled manner, in which case the effect of upgrades may also be limited. Such a debt may be addressed with line-of-business application owners or functional leaders.

In some cases, the problem may be widespread, attributable to an organizational culture of taking on debt during periods of high growth. For instance, the impact of not upgrading a critical ERP application may span business functions. Upgrading such a vital application platform may require engagement of senior business leaders who can influence their peers. Deutsche Bank offers one example of an engaged senior leader communicating thetechnical debt problem. During a press conference, co-CEO John Cryan publicly acknowledged the root causes of the problem of technical debt at the bank:

About 80 percent of our 7,000 applications were outsourced to a multitude of different vendors. Design was basically done in silos or by joint standards where they were either hardly used or not used at all. The result is that our systems do not work together, and they are cumbersome when it comes to the application and often incompatible. A figure that worries me in particular ... is that about 35% of the hardware in the data centers has come close to the end of its lifecycles or is already beyond that.

Most executives may not be inclined to use such a public forum to acknowledge their problem, but they can certainly take a page out of Deutsche Bank’s playbook. Acknowledging the problem of technical debt, and communicating that problem to stakeholders, helps set a foundation for resolution. Such acknowledgement by IT and functional executives can act as a call to action that enables project teams to find a way forward.

ADDRESSING TECHNICAL DEBT

In this section, we will look at some of the proactive and reactive recommendations for dealing with vendor-created technical debt.

Proactive Engagement

IT executives should proactively engage with stakeholders to continually plan for upgrades to minimize widespread occurrence of software version backlogs. The most effective technique for addressing technical debt is to embed vendor-recommended version upgrades into ­existing governance and IT management processes, rather than trying to deal with such upgrades in isolation. Specific strategies include:

• Align vendor roadmaps with business strategy. Continual business engagement is an effective way to align vendor upgrade recommendations with ­functional requirements. Business strategies and roadmaps may indicate the need for newer functional capabilities and investment plans. During review discussions, IT executives should highlight existing technical debt and upgrade recommendations that may also be addressed in the roadmaps. Such interactions will ensure stakeholders are continually educated about technical debt and actively participate in ­tackling it.
   
• Enterprise architecture governance. An effective Architecture Review Board (ARB) will periodically review business change proposals against functional and technical roadmaps. In a previous Cutter IT Journal article, I explain how an ARB can:

• Highlight architecture risk. Do so by enforcing the ­architecture principles and best practices during reviews.
   
• Ensure project alignment with predefined roadmaps to enable long-term strategies. By taking a cross-functional view and ensuring visibility into projects, a well-functioning ARB should be able to identify situations in which project teams inadvertently or deliberately take on technical debt. When identifying a potential debt, the ARB should also help teams communicate the impact and guide them in tactics for addressing it.

• Application architecture and design techniques. IT executives and enterprise architects have the most influence when new software platforms are being introduced into an organization. Design techniques that can minimize the impact of future technical debt include:

• Loose coupling. Thoughtful loose coupling of ­systems and interfaces and adopting standard, vendor-neutral integrations can pay dividends in the long run. Such loosely coupled application platforms and components could be upgraded without wider impact to the landscape.
   
• Cloud hosting. Cloud-based offerings from software vendors continue to mature. Moving to a platform as a service (PaaS) or software as a service (SaaS) offering from a vendor is a conscious way to cede control of application management in return for assured business capability and functionality. Doing so minimizes the need to manage vendor-driven upgrades. However, IS managers and teams may still need to keep track of integrations and data transfer that may be impacted by such upgrades.
   
• Configuration before customization. Most commercial software solutions provide the ability to config­ure business processes/workflows. Some ­functional requirements may be very unique and require customization of the platform via writing additional code. To the maximum extent possible, such customizations should be avoided or minimized.
   

Reactive Remediation

Proactively addressing issues arising from vendor-­driven upgrades may minimize the technical debt challenges in the future. However, not all such efforts can prevent technical debt. An ARB, for instance, may agree that a project should take on a prudent/deliberate debt, with a clear means of addressing it later. There are several ways organizations can deal with technical debt after it occurs:

• Pay as you go. An effective way to cope with debt is to periodically pay it off. However, organizations will have more than one vendor solution and will have to orchestrate upgrades across vendor platforms and solutions in the landscape. Such upgrades should be planned to coincide with other changes and scheduled maintenance in order to minimize business disruption. Effective techniques for managing periodic upgrades include:

• Defined policy on technology debt. Organizations caninstitute a policy explicitly stating that critical software will not get behind vendor-supported versions. For example, the policy might state that “all software must be version n-2 or higher,” where “n” is the most current vendor version. Such a ­policy should also be backed up with executive support and funding to ensure compliance.
   
• Dependency matrix. Organizations can use a dependency matrix to show compatibility of software and infrastructure versions recommended by vendors. The matrix should include most, if not all, of the dependent platforms in the landscape and should be used to guide upgrades.
   
• Cleanup releases. Where feasible, IT leaders should plan for “cleanup” releases to remediate the existing technical debt. If it is not feasible to plan such releases, one could explore opportunities to include upgrades with other scheduled changes.
   
• Budgeting for technical debt. The dependency matrix and other tools can be used to help estimate the effort involved in handling such upgrades.

• Landscape simplification. IT executives should seek opportunities in large transformation programs to tackle technical debt. An example of such a landscape review is illustrated in the case study (see sidebar “Case in Point: A Multinational Reacts to Platform End of Life”).
   
• Engagement with vendors. Software vendors provide several venues for engaging with clients. These range from informal online forums, wikis, and discussion boards to more formal surveys, technical conferences, and product planning sessions. CIOs and executives of enterprises with a large investment (or installation) of a vendor solution might benefit from engaging in such formal forums to (1) influence backward compatibility of the product in the vendor roadmap, and (2) seek assistance from the vendor in addressing technical debt attributable to its product.

• Other mitigation. An organization may review the impact of vendor end-of-life support and take actions to mitigate the risk, which could include:

• Self-insuring. The organization might manage ­without vendor support if the software is non­critical, there is no regulatory reason to seek support, and if analysis of recent history indicates very few instances where vendor support was actually sought.
   
• Outsourcing. The vendor may sometimes sell or license the source code to enable the client organization or a third party to support an aging software product.

CONCLUSION

Technical debt attributable to software vendors is an ongoing challenge that IT executives need to manage. As with financial liability, it may not be practical or prudent to avoid all technical debt. With awareness of the existence and impact of such debt, however, IT executives can plan to effectively address it.