Did ya ever work on a project that seemed like the schedule was too aggressive? Where the team had to constantly fight to stay on schedule, and to keep moving forward? Where things maybe got behind and we piled more resources on to catch up? Where things felt bad, but we kept on fighting until…
…it was too late. Like in baseball, when the only way get on base is to slide in? Like in football, when you just need that one more yard, and you then turn it over?
The thing is, software development isn’t a sport. We don’t have an opposing team. We don’t have a team owner or coach. Software development is a business activity. It is an activity of manufacturing, of logistics, of research and development and of analysis.
The enemy is risk. In order to defeat risk, we have to understand where risk comes from, and we have to understand how to retire it. Risk is tricky because it costs you little (or nothing) to carry it, but much to quantify, even more to retire it, and potentially even more when it is realized.
So what is risk, really? Risk is the inverse probability that any desired outcome will be manifest. So who cares? Why do I care that a desired outcome is not manifest? You care, because as a software professional, your job is to manifest desired outcomes. Your career path is closely coupled to your success at manifesting desired outcomes. You care because your customer pays you to care. If you don’t manifest desired outcomes, you (ultimately) won’t continue to get paid.
A Scenario: you are tasked to deliver “Some Awesome Feature” in some software by “not very far in the future”. The risk (in aggregate) is the probability that it won’t be awesome or done by the time your customer needs it. Stated more simply, risk is the probability that you will fail, and in general, you want to reduce that probability most of the time. In order to do that, you need to understand where these risks come from, and what can be done to reduce them.
Risk is the enemy of predictability. If you are a software developer and you want to make money, you have to be able to accurately predict how much effort will be needed to produce the software capabilities that are commissioned. Regardless of whether you work on time and materials type projects or on fixed bid type projects your ability to predict when things will be done and how much they will cost affect how you get paid. Truly, getting paid is the goal, right? As a contractor on T & M, the more you can accomplish in a week, the higher your value to the team, and this value allows you to command a higher pay rate. On a fixed bid project, the faster you get something done, the higher your effective hourly rate.
I want to talk about three kinds of risk in the software development process:
Risk of Communication – this risk comes from misunderstanding, imprecise specs, poor collaboration, etc. It is introduced when we all smile and nod afraid to admit we have no idea what is being discussed.
Risk of Unknown – this risk comes from the things we don’t yet understand about the problem, the solution, the technology. It is introduced when we define problems or propose solutions without the necessary expertise.
Risk of Impact – this risk comes from our lack of end to end thinking. It is introduced when we propose changes without contemplating the entire set of outcomes that is affected.
You will notice that in each type of risk I indicated how that type was introduced. Obviously, the easiest way to reduce risk is not to introduce it in the first place. The less time you spend mitigating, retiring, or cleaning up the impact of risk YOU introduced to the project in the first place, the faster you get done and the more you get paid. Intense discipline is required to work without introducing risk. Most of us don’t think often about the probability of our failure. Here are some common ways that people and organizations try to inhibit the introduction of risk into the software development process:
1) Process Frameworks – SLDC process or agile process frameworks are designed to introduce discipline to help us think through different activities in the process.
2) Design Patterns and Methodologies – following established patterns and becoming familiar with those patterns that are used by many developers and organizations can help us identify things that “don’t fit”.
These can be powerful tools to prevent the introduction of unnecessary risk into our software development process, IF we understand that their primary purpose is risk oriented and not otherwise. The primary problem with process frameworks and methodologies is that they are expensive. Often they are like “swatting a fly with a sledgehammer” – not altogether effective and not without other consequences. Process frameworks and methodologies are only effective when ALL practitioners reasonably understand:
1) Why the process framework or methodology is being used…
2) What the process framework or methodology is designed to do…
3) How to correctly perform the activities or steps in the process framework or methodology…
In my experience, many organizations that adopt or develop process frameworks or methodologies do not “certify” their staff as competent to participate – so
they fail to get the full benefit of their “system” because people who are not certified either shortcut the process, or they butcher it in ignorance. Most of the time, people complain about the “process” because they do not understand how to get value from it. Other times, the “goals” of the process are murky at best. “We have a process so that we can pass audit requirements” – not so we can build better software, not so that we can reduce software project risk.
Partially implemented process frameworks and methodologies give an organization a false sense of security or confidence. Sometimes process frameworks or methodologies are so “heavy” that staff finds themselves “mired” in activities that don’t really add value to the software. This can drive the cost of the software production so high that the “customer” organization doesn’t want to pay for it.
Sometimes you simply cannot avoid the risk. Sometimes your team is new and hasn’t fully formed yet. Communication practices are not mature within the team. Sometimes you have an unavoidable, irreversibly adversarial relationship with your customer. Sometimes you are required to use a technology that is immature, or that the team has limited familiarity with. Sometimes the schedule imposes risks because there is not time to work through team, technology or communication issues. For whatever reason, you simply cannot avoid the risks inherent in your project. What opportunity do you have to be “predictable”…?
In order to manage risk, you have to understand risk. Risk has three main components: its source, its trigger, and its impact. The source of a risk is how it is introduced to the project. When we “avoid risk”, we are simply “managing” at the source. Risk is necessary when I have insufficient control over the source to avoid it. The impact of a risk is “the bad thing that will happen” to my project. This could affect schedule, cost, scope and/or quality constraints around the project. The trigger of a risk indicates the conditions under which the impact will be realized. Risk is managed by understanding the trigger and the impact and planning the appropriate actions to either a) firewall the impact, b) disarm the trigger, c) realign the constraints. The simplest of these is to realign the constraints. I merely adjust my project plan and budget to allow for the impact.
The Bomb Squad
Risk management is kind of like being on the bomb squad. Ya don’t wanna be ’round here when this baby goes off. Could be a “career limiting move”.
Evacuate the Area
In this analogy, adjusting the plan to allow for the impact is kind of like an evacuation. It is the simplest easiest way to reduce the casualty count. The building still blows up, but nobody dies. People don’t usually die from software project risk impacts but they can certainly have their reputation besmirched or even lose their current job, especially if they weren’t even aware of the risk in the first place.
A Containment Field
Sometimes we can reduce the impact even further, but constructing some kind of containment field around the risk to limit the impact. This strategy is great for risk of unknown problems where we are missing key information, we can sequence the plan to buy time around the risk triggers, while we make design decisions that allow us to handle a wider variety of situations. This results in a more complex software product, but if the cost of that complexity is sufficiently less than the impact of the risks, we have successfully contained it. We can see these efforts in Proof of Concept or Spike or Walking Skeleton strategies during software projects.
Disarm the Trigger
Sometimes we can prevent the impact altogether, but disarming the trigger. If the bomb does not explode… No impact! However, that does not mean that disarming the trigger is without cost. There is work involved in this, whether it is political or analytical or redesign – you are expending effort to prevent an impending impact.
Disarming triggers and setting up containment fields are risk mitigation strategies. They are trading a (hopefully) small amount of impact now for a much larger impact later. The best part of this strategy is the lack of political fallout. When you have planned for the risk and executed a mitigation strategy, none of your stakeholders should be surprised by the additional cost or schedule. Whereas with the unpredicted impact, they usually get aggravated.
These strategies are great for risk of unknown. They don’t do as well for risk of impact. Risk of impact are the natural outworking of doing the work. However, we do the work in ways that introduces risk that we aren’t aware of. Sometimes this risk is caused by building things an a less than optimal sequence, so that we introduce the need to re-work earlier completed deliverables to finish later deliverables. Other times, the risk is introduced when we shortcut the design process and our design is insufficient to get to done. Many times, it is simply that we fail to think end to end, when planning, so are surprised by “unplanned work” that simply “jumps out and bites us in the butt” at the most inappropriate time.
Land Mines in the Plan
Risk of impact are like land mines. We don’t know they are there until someone steps on one. Then we have to perform a mine sweeping exercise. We spend time searching for and carefully marking the land mines especially those that are “in our way” so that we can clear a path to success. We have to carefully decide which mines to mark and ignore, which ones to carefully remove and which to simply blow up.
The problem with these land mines is that we ourselves created them. Either through poor planning, bad technical decisions, or poor understanding of the requirements or business domain, we laid the mines. These truly reflect Shakespeare when he reflected, “The engineer was hoist on his own petard”. We are blown away by our own work product….
Mine sweeper operations, like all risk mitigation strategies are expensive. Thy introduce unplanned work, and cost and schedule impact just to understand the risks that have already been embedded. Is there a preventative? Yes. But they also are expensive. Code reviews, design reviews and requirement reviews are effective ways to prevent these land mines from being laid in the first place, but they require a discipline that often succumbs to pressures introduced by project constraints.
The real source of risk of impact.
And there you have it. When we try to compress the activities, we introduce risk. We take shortcuts that introduce risk in unpredictable ways. We short cut our risk avoidance strategies and end up with waste – time wasted on activities that were shortcut and undermined by pressure and constraints.
Back to “The Slide”
How does risk then make it seem like we are “out of control”, or “caught with our pants down”, or simply don’t know what we are doing? It is because if we don’t see the risk, plan for its avoidance (in process) or mitigation (in diligence) or impact (as a last resort), we (and our customer) are somehow surprised when we realize the impact of risks that we simply didn’t see, failed to avoid, wouldn’t mitigate, or couldn’t prevent. The most ironic aspect to this is that our process and diligence are often the first things to get shortcut when we start running up against constraints. When our bosses and customers ask for faster and cheaper or more with less, and we respond without exposing the risk that those decisions introduce, we are rewarded by being blamed when things go south and the impact of that risk is realized.