Estimating Defects - Using Principled Negotiation

I’ve caught myself changing my stance on pointing defects this last month as I consider different contexts and reflect on what the real situation is in my current teams.

Previously, my general rule was that if we don’t fix it a bug in the iteration in which it’s found (which would be the best thing to do), then it needs to be estimated before being scheduled into a future iteration. If we don’t fix it now, it will be more expensive to fix later. We should recognize that cost. This will help us know when we'll be done with the release. If we have a significant amount of work to do represented in defects, I want that quantified and represented in the release burn down chart. I acknowledge that defects are hard to estimate. Their accuracy will be worse than that for stories, but I don’t see this as a significant problem. The value we get from knowing when we will be done is worth the difficulty.

Mike Cohn recommends estimating defects if you have a big backlog of them and are fixing new ones as they come in. But that 'if' is important. Don't ignore the context. A similar context was underlying my rule of pointing defects. I assumed that if you were having the discussion about whether to estimate defects, then you must have defects that weren't being fixed in the iteration in which they were found. In that case, you likely have more than a few defects in the backlog and you aren't likely to have a policy defining a systematic approach for fixing issues. You'll schedule the defects into iterations just like you schedule stories. A systematic approach, by comparison, could be knocking out all the defects in the next iteration, or knocking out a fixed number each week. In those cases, I wouldn't estimate.

It was Dan Neumann's thoughtful post that made me realize I was making too big of an assumption about the context at hand. He also helped me realize that interests matter. Principled negotiation is when you consider each party's interests, not their positions. ("Getting to Yes" is an excellent book by the way.) For this topic, the positions are whether to estimate defects, and if you do, whether to include those points in velocity and in the release burn down chart.

But the interests are more varied and may include:

INTEREST: A desire to measure "forward progress". As Cohn put it, "making visible that the team is going more slowly through the work than it could if these legacy bugs had been fixed when originally found."
CONTEXT:  Most any.
POSITION: Don't point defects. Or, do point, but don't include those points in the velocity or in the release burn down chart.

INTEREST: A desire to keep anyone from thinking that scope is increasing. A desire to keep the release burn down chart from going flat, or up, due to defects. (It’s embarrassing and requires lots of effort to answer executives' questions: “Why is scope increasing?" "Why do we have so many defects?”)
CONTEXT:  Applies to any context, but in particular when defects are not being attacked in a smooth, systematic manner.
POSITION: Don't point. Or, do point, but don't include those points in the velocity or in the release burn down chart.

INTEREST: A desire to make velocity appear unstable so that quality problems become visible.
CONTEXT:  Defects in the backlog are not being attacked smoothly (a set number per iteration).
POSITION: Don't point. Let the uneven defect fixing impact velocity.

INTEREST: A desire to make velocity appear more stable. (Similar to the case above with a rising release burn down chart, an unstable velocity makes executives ask questions that can be hard to answer.)
CONTEXT:  Since the work effort applied is generally stable each iteration but the balance between defects and stories might not be, pointing defects makes velocity more stable.
POSITION: Do point and include the points in the velocity.

INTEREST: A desire to know when we are going to be done with all the stories AND defects.
CONTEXT / POSITION 1: Defects are not being attacked in a smooth, systematic manner. This interest can be served by pointing defects and including the points in the release burn down chart and in the velocity.
CONTEXT / POSITION 2: This can also be served by not pointing AS LONG AS you are addressing issues on a steady basis (so that your velocity is stable and therefore useful in computing the release end date).
CONTEXT / POSITION 3: Another approach is to address all of the issues immediately in order to get them out of the backlog. Point them or not; doesn't matter.

See how a single interest can be served with different contexts and different positions?

INTEREST: A desire to avoid overstating velocity.
CONTEXT:  Estimating and including recently introduced defects in velocity. The excellent concern is that "the team’s anticipated velocity includes an allocation to fixing defects that are yet to be introduced" and pointed. (Dan Neumann)
POSITION: Don't point, or don't include in velocity.

INTEREST: A desire for velocity to "… represent the team’s true capacity to accomplish work." (Cohn) Or, a desire to make velocity look better than it really is. (A desire to pacify upper management that wants to see a rising velocity number. A losing game, by the way.)
CONTEXT:  Most any.
POSITION: Do point.

INTEREST: A desire to get credit for work done, which is less justifiable if it’s our own rework and more justifiable if the bug was caused by another team.
CONTEXT:  Most any, but needing to get credit signifies an illness in the organization.
POSITION: Do point.

INTEREST: A desire to avoid the need to argue about a bug report actually being a story (so that we can estimate it for whatever reason makes us want to estimate it).
CONTEXT:  Applies to most any context, but this probably signals some animosity or lack of trust between the Product Owner or QA and the rest of the Technical Staff. At best, this may just be a desire to hold to the team's aggressive commitment.
POSITION: Do point.  If you point them, then it's easier to quantify how much more you did beyond the commitment.

INTEREST: A desire for the Product Owner to weigh the cost before requesting a fix.
CONTEXT:  Most any.
POSITION: Do point. Whether to include the points in the velocity is another matter.

The pie can be expanded for each of these positions once the interests are known. One party may become aware of an illness in the organization and see a way to cure it. One of you may be able and willing to educate the executives or boldly attack the quality problem. Maybe an unstable velocity and an increasing release burn down chart should be welcomed as an opportunity to talk to those executives about agile, metrics and what-not. Agile believes that visibility is good.

Bottom line – Use principled negotiation to understand stakeholder interests and the full context. Then devise a defect handling policy that specifies when defects will be handled and whether they will be pointed. The way you use iteration and release metrics needs to be in tune with all of that.

Parting shot – A tremendous amount of effort is expended on this topic for a situation we'd rather not have in the first place.