Agile Predictability

Predictability is probably the least hyped benefit of agile practices. It is not sexy or fun, nor does the team gain from it, in a positive way. But the team does benefit from it, from a management perspective.

The benefit of a predictable software delivery process is realized in three ways:

• Rational Planning Process – when I can measure the capacity of the team, I can create a rational plan. This allows consulting firms to bid project or enterprise software organizations to budget more accurately.
• Builds customer trust – when I can predict the delivery pace, I can communicate this to the customer, and build trust and reputation. This allows for repeat engagements in consulting, or greater latitude in decision making within the enterprise. When the team demonstrates predictable delivery, the customer can relax and focus on defining the output rather than the mechanisms of accountability.
• Sustainable Pace – when I can project delivery based on the capacity of the team, I can ensure a more sustainable pace. I can negotiate in trust with the customer, understanding their need. I can increase capacity when needed, rather than forcing the team to stretch (work overtime) to meet goals.

These are serious benefits, but we haven’t said that agile practices deliver – only why and how predictability benefits an organization. So how does agile deliver on this benefit?

1. Frequent, measurement on regular intervals – by defining arbitrary milestones at regular intervals ( iterations, sprints, time boxes, etc) and measuring the output of the team through each interval, agile practices generate the metrics that support predictability. While it is important to measure, it is also important to have a light weight measurement practice – so that the team doesn’t trade productivity for predictability. The velocity metric (calculated as completed units of estimated effort per unit of duration) is typical. Combined with a remaining estimated effort metric, and planned cost per unit of duration, or average cost per unit of estimated effort allows an accurate projection of both duration and cost. If my measurement frequency is a week, I can know each week how far off plan I am in terms of both schedule and budget. I can choose how I react to this knowledge.
2. Cross-functional milestones – what differs within agile software life cycles from more traditional gated software life cycles is that each milestone in agile is self-contained, and has elements of all phases of software delivery – analysis, design, development, testing. This means that each skill is consumed and measured in each milestone. In a phase gate life cycle, the last skill (testing) is not consumed or measured until the last milestone – and so is not predictable – so I am using measurements from other projects at best. If I form virtual teams for projects (consulting), I can only predict at best from prior individual performance, which does not account for team dynamics.
3. Repeatable, lightweight measurement framework – agile practices propose a lightweight estimation practice, a repeatable practice of measuring delivery against estimate. Agile estimates are done with less precision, but much less effort than gated methodologies – again trading productivity for predictability. Traditional project managers (Glen Alleman) get aggravated by this, because it does not have the precision of the more industrial strength estimation and measurement frameworks, however, it also does not have the overhead. Agile applies that same principle to planning that it applies to software delivery (YAGNI) – that is, I do not implement more sophisiticated measurement or estimation schemes, unless the nature of the work, or the project merits that level of precision. If the customer provides requirements that are less defined, I will by definition, provide estimates that are less precise, even using the industrial strength mechanism. Since software development is a largely non-repeatable process, with no standard units of measurement for output, agilists argue that using a precise estimation and measurement framework that costs more is a waste, because it does not yeild greater predicitability.
4. Lower planning skill requirement – agile planning practices are typically simpler, easier, and less effort, than other practices. This means that your project manager doesn’t need to know how to do a PERT chart, a Critical Path Method (CPM) or Earned Value measurement. In agile plans, you don’t focus on dependencies between tasks, you simply sequence the deliverable capabilities in the order of customer value, and go. All of the more sophisticated plan maintenance, are optimizations that may not be necessary, or beneficial, given the level of unknown in the work being contemplated. Here is the kicker – because it is easier to implement, it gets done more consistently. It gets done with more discipline – so a lower fidelity process executed with greater discipline actually delivers higher predictability over time than a higher fidelity process being executed with less discipline.
5. Emphasis on learning and improvement – agile practices call for retrospectives after each milestone. The purpose of the retrospective is to identify opportunities to improve practices in use. As the team works together, they find ways of working better, and have a stated process for proposing, deciding and implementing improvements. Since the team owns the process, and the improvements they are incented to implement (it was their idea) and thus each team adapts agile practices to suit their specific situation and goals. Through the process of self-improvement, the team becomes more predictable, because this milestones surprises become opportunities.
6. Progressive elaboration – in phase gated software development life cycles which assume that all inputs are known early – (requirements, design, etc.) the temptation is to plan in more detail than you actually know. This introduces fiction and reduces predictability. Agile’s preference for progressive elaboration in requirements, designs, and plans recognizes the FACT that you cannot know everything before you start. This leads to plans that grow and shrink as knowledge is acquired and accounted for, but always currently reflect the state of our understanding of the work. This provides the ability to decide and adjust based on a plan that is as accurate as it can be with the knowledge that we have. Anything more is speculation, or fiction, that leads to poor decisions and improper adjustments.

The benefit of predictability is one that is less intuitive that the rest. Who really benefits from it? In the end, all benefit. Management has more control, the customer is satisfied, and the team can reach a sustainable pace – that sounds like win-win-win.