"I believe that we must begin by evolving our organizations toward adaptive governance and away from plan-driven governance, just as agile projects have evolved toward adaptive management and away from plan-driven management."

Opening Statement

Agile methods (Scrum, XP, Crystal, DSDM, Feature-Driven Development, etc.) have moved into the mainstream over the past few years. Earlier in their adoption cycle, industry leaders such as BMC Software, British Telecom, Capital One, DTE Energy, and Yahoo! discovered that, when implemented appropriately, agile methods accelerate project delivery times, increase customer and employee satisfaction, and enable flexible changes in business requirements. These companies adopted agile methods and, as a result, realized faster throughput and higher business customer satisfaction on individual projects. Based on this evidence and growing success, the move toward agile methods is now a widespread industry phenomenon.

Yet despite undeniable wins with agile at the project level in many quarters, sustained large-scale initiatives to adopt agile methods are fewer and further between. Agile adoption initiatives are known to run into hurdles when the company culture is at odds with core agile values. VersionOne's 2008 State of Agile Development Survey¹ identifies the inability to change organizational culture as the leading barrier to further agile adoption. What are some of the organizational factors that are at odds with agile values and are preventing further adoption of agile within organizations?

One factor is IT governance. Many experts in the agile space believe that there is now a significant misalignment between the way agile projects are run and the way IT projects are governed in general. IT program and portfolio management, in particular, seem to be at the root of many of these alignment issues. As a consequence, corporate project portfolios remain challenged in many organizations. Executives still see projects that are late and overbudget, deliver poor value, and have less-than-satisfied business sponsors and end users. How can we scale agile methods beyond individual projects so that the programs and portfolios within which they exist can benefit unequivocally? I believe that we must begin by evolving our organizations toward adaptive governance and away from plan-driven governance, just as agile projects have evolved toward adaptive management and away from plan-driven management.

In most organizations that have adopted agile methods, the techniques used for program and portfolio management are still predictive or plan-driven. They consist of yearly budgeting cycles and capacity planning and heavily matrixed resource management. It's no surprise, then, that despite adopting agile methods for their projects, many organizations have yet to exploit their full benefits at the portfolio level. Or to put it another way, agile projects are constrained because portfolios are clogged with the debris of failing projects and with slow-moving projects that delay more critical initiatives. Consequently, organizations are unable to properly staff projects and deliver them at a rate that their business customers would appreciate.

How can the project portfolio be unclogged so that business value can flow and projects can complete faster? How, specifically, can program and portfolio management be improved? In this issue of Cutter IT Journal, our contributing authors explore a variety of different approaches to this challenge.

In our first article, Scott Ambler explores how portfolio management and agile software development can not only coexist peacefully, but actually enhance each other. Ambler presents an overview of both portfolio management and agile software development and then explores how portfolio management fits into the agile project lifecycle. Ambler takes a comprehensive view of that lifecycle, noting, "Many people like to solely focus on the predevelopment and development aspects of portfolio management, but that would be a mistake." Indeed, he sees portfolio management beginning with "Iteration -1" (a pre-preproject phase in which the organization weighs the risks and decides to pursue a project) and ending only with system retirement. Regardless of what stage you are at, Ambler argues that the greater project visibility and increased stakeholder control offered by agile development will "enable effective monitoring and management of the development projects within your IT portfolio." He offers an IT asset classification approach that can help guide organizations' refactoring efforts, and wraps up by relating portfolio management to other enterprise disciplines, including enterprise architecture, enterprise business modeling, and strategic reuse of assets across the portfolio.

Next, Cutter Senior Consultant Johanna Rothman delves into the details of selecting a ranking method for your project portfolio. Rothman begins by recommending that you ask the hard question of whether the project should exist at all, and she closes with the sage advice that you not forget to kill nonperforming projects. In between, she tells how you can rank projects by assigning them a point value, weighing the risk of doing (or not doing) them, reviewing the project context, using double- or single-elimination methods, or assessing them in light of your organization's mission and values. To get an idea of when each of these methods is called for, we follow Dave the CIO and his team on their portfolio management journey, seeing how they employ the various ranking methods as their circumstances dictate. And if committing to a definite project ranking seems too daunting, Rothman reminds us that ranking "isn't forever" and that "you have only to wait for one iteration to finish until you can rerank your portfolio." Knowing they have that flexibility, even commitment-phobes will find it easier to make the necessary project ranking decisions.

In our third article, Cutter Senior Consultant Jens Coldewey walks us through his agile portfolio planning experience at TWeb, a product company in the travel industry. TWeb originally hired Coldewey to introduce XP, with the aim of improving the productivity of their development team. However, it didn't take him long to discover that TWeb had much bigger problems, including a planning process in such disarray that "measured on a CMM® scale, this company was definitely on Level 1 -- chaos -- for the single reason that CMM does not define a Level 0." Some of TWeb's customers had been waiting on promised features for up to three years -- and were starting to call in their lawyers and/or publicly threatening to decamp. Coldewey describes how TWeb took their first steps toward introducing basic agile planning and how success in stabilizing their short-term plans eventually led to a portfolio planning game designed to forecast into the next two years. The devastating picture that emerged led to a tense confrontation, a momentous decision -- and the resignation of one angry executive. Coldewey concludes by bringing us up to date on what has happened at TWeb in the 15 months since then.

Next up, we have another case study -- but all resemblance between the last article and this one ends there. In contrast to TWeb, with its development chaos and customer defections, SciQuest is a fast-growing software as a service (SaaS) company that has embraced a Scrum-based, agile product development process and enjoys near 100% customer renewal rates. But even award-winning companies can find their numbers going in the wrong direction, as SciQuest's customer support department did last year. SciQuest COO Jamie Duke and Cutter Senior Consultant Sam Bayer narrate the story:

In the case of our customer support department, our mandate was clear. We needed to reduce the number of customer incidents reported per module per month by 20% in Q4 2008 as compared to Q4 2007. ... At the end of Q1 2008, we were startled to learn that not only had we not improved over the previous quarter, but in fact our performance had deteriorated significantly.

Customer support responded by assembling a cross-functional team in a two-day workshop to identify projects that could get the department back on track. Taking a lean portfolio management approach, the team came up with 17 potential incident-reducing projects and graded them on an A+ to C scale. Obviously the A+ projects -- those with "the highest potential to remove the greatest number of incidents in the shortest period of time with the least amount of cost and risk" -- were tackled first, and by the end of Q2 2008, the number of reported incidents per month had improved 22%. But where SciQuest's lean portfolio management approach really showed its mettle was not in these successes, but in one notable failure. When customers rejected SciQuest's efforts to productize internal tools so the customers could debug their own workflow snags, the company responded to the feedback by redeploying resources to higher-value projects in the portfolio.

Last but not least, Cutter Senior Consultants Bob Benson and Tom Bugnitz explore how lean portfolio management might be transformative for IT. I say "might," because the authors clearly feel that the jury is out on this question. They see two barriers to the widespread adoption of lean portfolio management:

The first is that lean portfolio management touches on the fundamental issue in IT decision making: justifying and reconciling conflicting demands for resources. This may require more structure and deliberation than lean portfolio management is prepared to provide. The second is that effective business management participation in this decision making may be hard to come by.

Benson and Bugnitz speak to the very real challenges of getting business partners on board who may not "want to spend the time and energy or risk the political exposure." They also analyze whether lean portfolio management is compatible with corporate IT decision-making processes. The authors then propose two approaches that may help us begin to define a lean portfolio management process -- one that relies on Warren McFarlan's "IT importance" quadrants and another based on the cultural assessment method they've devised in their 25 years of consulting. They conclude with the hard truth that lean portfolio management must enable organizations to kill off failing (or simply low-value) projects, for "without the possibility that projects can be stopped, management will have no stake in the process."

All our authors bring us valuable portfolio management perspectives and techniques as the industry attempts to figure out how to scale agile methods, sustain agile adoption over the long term and across the enterprise, and provide adaptive governance of our IT programs and portfolios. I'm confident that you will be able to glean insights from their work that will aid you in your own agile journey.

Happy Reading!

ACKNOWLEDGMENTS

I extend special thanks to Chris Generali and Karen Pasley of Cutter Consortium for their advice and invaluable review and editing assistance.

ENDNOTE

¹ 3rd Annual State of Agile Development Survey. VersionOne, August 2008 (www.versionone.com/AgileSurvey2008/index.asp).

2 3rd Annual State of Agile Development Survey. VersionOne, August 2008 (www.versionone.com/AgileSurvey2008/index.asp).