"There is a lack of easy-to-access war stories about how agility works in areas outside the traditional domains of agile projects. Yet reports from real projects are exactly what managers need to decide whether they want to use agility beyond the well-trodden paths."

"Agile" in Name Only

There's a reason agile development is limited to small collocated projects with low formal requirements. In any other setting, agile practices are bent to the breaking point. A remote customer? Teams in three different time zones? Formal verification processes? That's not agile!

Agile in Spirit

Agile methods can be used successfully in large or distributed teams or on projects with high criticality. Sure, you have to adapt your practices, but you still benefit from the agile value system. Frequent delivery of working software? Technical excellence? Shared responsibility? These agile principles are just as important for large distributed teams as for small ones — maybe more so.

Opening Statement: Exploring the Agile Frontier

by Jens Coldewey, Guest Editor

The idea for this issue was born during a panel discussion last fall. Being the "quota" agilist on a panel of mostly traditional-minded executives and academics, I was asked to answer spontaneously the moderator's opening question: "Are there any limits to agile development?"

For a fraction of a second, I was tempted to give the standard answer: "Well, if the teams are large or distributed, or you have high criticality, or a fixed-price contract, or anything else you would consider an 'important' project, then I'd be cautious with agility." However, in the next fraction, the internal QA part of my brain rebelled. "If agility leads to better quality, why not use it for high criticality?" my QA neurons wanted to know. "And you know guys who have successfully implemented agility in large or distributed teams!" So I frankly answered, "The only limitation I'm aware of is an organization that is designed to inhibit decision making. Anything else is manageable."

During the discussion that inevitably followed, I realized that there is a lack of easy-to-access war stories about how agility works in areas outside the traditional domains of agile projects. Yet reports from real projects are exactly what managers need to decide whether they want to use agility beyond the well-trodden paths of small teams that develop Web servers in a change-embracing environment.

Why is this discussion about the limits of agility important? From a methodological point of view, the agile value system supports close collaboration with the client, fast feedback, late and fast decision processes, incremental releases, and an absolute minimum of formal process. The more this description matches the characteristics of your organization and your projects, the easier it is to set up a process that supports these values. However, your project may not exist in a perfect agile world. For example:

1. Your team may be distributed over several sites, time zones, and cultures.

2. You may be developing software of extreme criticality, which usually needs to go through a formal certification process to ensure that the software does no harm.

3. Your organization may not be capable of fast decisions or may be averse to the sharing of responsibility that agile practices foster.

These are the three challenges we cover in this issue, and they are ranked in order of the increasing difficulty they impose on an agile project. People have dealt with these challenges successfully, and making their strategies public extends our knowledge about agile practices.

The structure of this issue follows these challenges. The first two articles discuss agile projects with large and distributed teams. Distribution dramatically decreases communication bandwidth inside the team. Decisions are slowed down significantly, and the project may even break into "local islands" that evolve faster than they are able to synchronize. Just a few years ago, Cutter Senior Consultant and agile pioneer Alistair Cockburn said, "If you ask me about distribution, I have never seen anyone doing this successfully; so my advice is: Don't do it." But is that really the final word?

Among those who have successfully used agile methods with distributed teams is our first author, Lise Hvatum of Schlumberger, the world's largest oilfield services corporation. The global nature of its business has led Schlumberger to engage in global software development, and Hvatum (herself a native of Norway) and her team have been involved in distributed development between the US and China for nearly a decade. For the last few years, Hvatum and her group have collected and documented successful practices from other distributed teams in the company. The result is one of the most comprehensive collections of agile practices for distributed teams you can find. Hvatum says Schlumberger's teams have found success "by pushing to achieve agile values rather than blindly following practices used by collocated teams." And though technology plays an important role in supporting communication in distributed teams, the values of the Agile Manifesto still hold: "You can easily overdo the emphasis on equipment, thinking that the tools solve your underlying problems. Simple tools tend to give the most benefit for the investment."

Her conclusion is twofold. On the one hand, Hvatum notes, "We have observed that distributed projects that apply agile development methodologies are more successful overall." On the other hand, she warns organizations not to be naive about the impediments distribution puts upon software projects: "Despite our work documenting the practices, we do not advocate for distributed development. ... An organization embarking on distributed development must seriously analyze the increased risk from the added challenges and weigh this against the perceived business benefits of a global presence."

In our second article, Jutta Eckstein, one of the early agilists in Germany and author of the book Agile Development in the Large, writes about her experiences with distributed agile teams. She recommends organizing the overall project team into feature teams on each site that are responsible for finishing particular functionality instead of organizing them around technical expertise. To jump-start collaboration between the sites, she proposes letting the teams work collocated for some time first -- advice you'll also find in Hvatum's article. In addition, Eckstein advocates having an "ambassador" at each site whose job it is to ensure communication among the sites.

Equally important is the frequent synchronization between the sites. Instead of the "same time, same place" rule collocated agile teams often use for their meetings, Eckstein suggests you use a round-robin schedule to accommodate team members in all involved time zones at least some of the time. And though you might think that slower communication would call for longer iterations, Eckstein comes to a different conclusion: "There is no need to actually change the length of the development cycles simply because you're working globally. Indeed, the majority of global projects tend to shorten the development cycles to reduce all risks."

Extreme criticality -- software on which human lives depend -- is the second challenge we discuss in this issue. The key QA question Gerald Weinberg used to ask was, "Would I like to wear a pacemaker developed by my team?" For Klaus Marquardt, a lead architect with Dräger Medical, such criticality is routine. His responsibilities include the architecture of systems that keep patients alive during surgeries, such as the Zeus anesthesia workstation whose development Marquardt discusses here.

Proponents of traditional processes often argue that systems of extremely high criticality are not suitable for agile development. Marquardt's article reveals this belief to be mere prejudice. A "healthy crisis" during the Zeus project prompted Dräger to adopt some key agile practices that proved vital to the success of the project. Changes in code ownership and a well-designed incremental build process enabled integration to happen more frequently and increased the quality of deliveries.

As successful as these practices were, however, the agile approach ended when the formal verification process (necessary to address the high criticality) began. As Marquardt notes, "Once functioning software entered formal verification, changes were no longer welcome." When human life is at stake, "doing no harm" is more important than reacting to change.

One of the surprising lessons of Marquardt's experience is how agility can emerge from necessity rather then the wish to "become agile." He writes, "The Zeus project never explicitly tried to apply an agile development process.... The agile mindset that evolved during the project was simply a successful attempt to mitigate primary project risks." Marquardt's contribution offers convincing testimony of the power of agile practices even for safety-critical software, as long as the team lives up to its responsibility and understands where its processes need to adapt to the requirements.

The third and most difficult challenge is an organization that is incompatible with the values and practices of agile projects. The last two articles describe two completely different approaches to this problem. Matt Ganis and Tom Hawkins -- both experienced members of the IBM Site Architecture team -- describe how they managed to embed an agile team into a "completely plan-driven" organizational environment. One issue they had to address was the coordination between two agile teams that had non-agile customers themselves. They did this by adding a "pre-planning game" that served to coordinate the priorities of both teams. And instead of trying to argue with the customer's waterfall-minded requirements team, they defined an interface between the waterfall group and the agile group. The agile team understood the requirements as "capabilities," broke them down into user stories, and thus integrated them into their process.

Though this is a frequent strategy, it comes with a significant downside. You can still use agile development practices to deliver high-quality software, but you lose the business advantages of fast reaction that contribute significantly to the ROI of an agile software project. Still, the goals of that team were more modest. "By working with our stakeholders and not replacing tools and processes they are comfortable with, we've been able to gently introduce them to the benefits of agile methods," the authors conclude. "It's not clear at this point if we've won them over, but we have shown them that the agile and waterfall worlds can coexist and successfully deliver on their organizational commitments without breaking into hostilities."

The last and probably most controversial article of this issue is by Brian Henderson-Sellers and Asif Qumer. They discuss using "method engineering" to introduce a hybrid agile/traditional process into an organization that was reluctant to adopt a fully agile method, "since it feared that would destroy, or at least harm, its existing organizational culture." The authors used an "Agile Toolkit" to create the new process "from small method fragments, which are chosen to suit the organization's individualistic needs." The authors note that "with the application of a newly engineered agile product enhancement process, [the company] managed to increase both the speed of the development of new features and the predictability of the software product features." But there was also a flip side: "A formal change control mechanism is used to control (read: avoid) change, and although the developers may give input to decisions, they are not allowed to make these decisions on their own." Considering that the formal part of the process was just a first step in a transition, one might argue that this is an acceptable price to pay.

To monitor transitions like these, the authors suggest an "Agile Adoption and Improvement Model" that defines six levels of agile implementation. The authors hit hot ground here, as Cutter Senior Consultant David Spann pointed out in his Executive Report "The Need for and Fear of Agile Certification" [1]. For me, it is an open question whether the future of agile agrees with the model proposed here, or whether there is a conflict between the method engineering approach and -- as the Agile Manifesto would put it -- valuing "individuals and interactions over processes and tools."

In any case, I hope this issue contributes new -- and controversial -- ideas to the debate over the frontiers of agility. During its preparation, I have learned that these frontiers are far more extensive than the public discussion has recognized up to now.

REFERENCE

1. Spann, David. "The Need for and Fear of Agile Certification." Cutter Consortium Agile Project Management Executive Report, Vol. 8, No. 4, April 2007.

About the Author