In the 90’s, the prevailing analogy used to describe software development to business leaders was manufacturing. In this analogy, software requirements, like raw materials, would flow into the factory and progress through a series of workstations (aka software developers) that spit out new features ready to be shipped. The book The Phoenix Project taught me that there are still some valuable lessons to be learned from this metaphor, however, in this post I’ll argue that the processes required to enable successful software development is much better described with rituals instead of phases.
In manufacturing, inputs and outputs are fixed. They are not in software projects.
In manufacturing the inputs and expected outputs are locked in and unchanging. Every product has a very detailed specification before it hits the assembly line. The cost to produce one unit as well as the required functional characteristics are set in stone. No matter how badly we want software to be like this, it just isn’t. Building software, it turns out, is much more like inventing than producing… and that’s because of change.
Building software, it turns out, is much more like inventing than producing… and that’s because of change.
In the late ’90s in early 00’s “change” became a four letter word in conference rooms. Developer’s braced themselves before warning of “scope creep!” meanwhile business stakeholders felt the frustration of inflexibility as their dev teams pushed back on new ideas and feature requests. In the height of this tension, defending scope and contract negotiation sadly became a part of most large, high stakes software projects.
The problem here was that teams were treating planning as a phase in the beginning of the project. Unlike a factory production line where parts are assembled by robotic workstations, features in a software system can be built in many different ways. Beyond that, the features that actually end up being built change as the project progresses. The customer’s knowledge of the system and the business objectives start to evolve as soon as they start to witness the system coming together. None of these dynamics exist on a factory floor.
According to a report on the Sources of Software Requirements Change, there are five primary sources of changes impacting software projects:
- Market: Differing needs of many customers, government regulations, external to the project.
- Customer Organization: Strategic direction of a single customer, customer organization considerations, external to the project.
- Project Vision: Problem to be solved, product direction and priorities.
- Requirements Specification: Specifying the requirements of the established problem.
- Solution: Technical answer to the problem.So, in a world where change is a constant, we have to cultivate the ability to deal with it. And surprise, surprise, our solution for dealing with change is: communication.
Some of these changes are within the control of the client or dev team, some simply are not. Often certain changes can be predicted, but many times change is introduced into a project as a surprise. What these well meaning teams discovered is that despite their best efforts to understand the system requirements in the beginning, change was inevitable. This is where the concept of agile software development began to emerge.
The evolution of the Agile Manifesto changed the analogy…
In 2001, in the Wasatch mountains of Utah, between good food and some skiing, a group of leading software developers assembled to update the analogy. Instead of manufacturing, they likened software development to new product design. These pioneers took a different approach: What if we embraced change as inevitable? How could we give business leaders the flexibility they craved while providing developers with enough clarity and constraints to actually execute?
What resulted from this meeting in Utah is called the Agile Manifesto. Now a mainstay in public and private sector software teams around the world, it has four main tenets:
We value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
There’s a lot of nuance in this, but for this post, let’s focus on these two lines: “Individuals and interactions over processes and tools” and “Responding to change over following a plan.”
By rituals, I mean meetings (err, interactions) that empower customers and developers to respond to change on a regular, recurring basis. The key is to design these meetings to empower customers and developers to share the responsibility of managing change… and more specifically, making trade-off decisions.
By “managing change,” I mean making trade-off decisions…
Do you remember that scene in Apollo 13 where the Houston team comes together to solve the problem with the CO2 levels in the shuttle? With a hodgepodge of parts strewn on a conference room table, all actors must accept the reality they face: the resources available to them to correct the problem are finite.
This is a classic example of a collaborative team coming together to solve a trade-off problem within a constrained environment. This is the kind of meeting I mean when I say “ritual.” Customers and developers should plan to come together routinely (weekly or bi-weekly) to update each other on the existence of constraints. This is a time to confront reality and respond to change with agility. A chosen technology might be fighting the team, eating up invaluable development cycles. Perhaps it’s time to scratch that work and pick a different path? A critical marketing window might be closing for the business and it might make sense to ship the product with fewer features.
A word of warning: trade-off decisions are hard and many customers would prefer not to have to engage in them. A typical decision may sound something like this: Do we want to hit our marketing deadline without feature XYZ, or postpone launch until the feature is complete?
A history of the Agile Manifesto sums it up nicely, “…marketing, or management, or external customers, internal customers, and, yes, even developers don’t want to make hard trade-off decisions, so they impose irrational demands through the imposition of corporate power structures.” The old saying “we cannot eat our cake and have it too” fits especially well. It is absolutely essential for both sides to come to these rituals with a spirit of collaboration and teamwork.
By scheduling planning and review rituals, customer and developer teams learn to have these discussions with trust and candor, all operating from the same sheet of music.