What do you estimate and when do you do it?

Pick up most any book on agile or Scrum and see what it says about planning a product backlog or planning a release or about computing an end date for a project. It will say to identify the stories, estimate them in points, add up all the points, and compare the backlog point total with your velocity to figure out how many sprints it will take to implement all those stories. Then reserve some room for unknowns and for changes (agility).

So, that implies identifying and estimating stories far enough in advance to match whatever your planning and commitment horizon is. This implies that estimating story points in sprint planning or just before sprint planning or even one sprint in advance is likely not far enough in advance unless you are ultra-agile and aren't making commitments or setting date expectations more than a couple weeks out.

Most companies I work with have a planning horizon that is 3 to 6 months out. So, we put together a 4 or 6 month plan. That plan is going to be based on what we can know in advance, namely, user stories. (Sometimes we do this at the feature level, but that's another blog post.) We won't know research stories or spikes or defects or production support in advance. Whatever goes into the plan is what should go into the velocity metric. Stated differently, you shouldn't put anything into your velocity metric that you can't plan for 4 to 6 months in advance. Which means you shouldn't bother putting points on defect or on research stories. If you are going to compare apples to apples, your velocity must exclude unplannable stuff.

That's why I put points only on real user stories. Here is a series of blog posts I did on each of these topics:

Don't Estimate Defects

Don't Estimate Spikes

Don't Estimate in Sprint Planning

Estimating Placeholder Stories is a Bad Practice

Now, some people try to estimate this unplannable stuff. They try to quantify it in points and include it in placeholders in their plan. You absolutely should track this unplannable stuff, and you should reason about whether it will be more or less in the future, but trying to back it into points or plan it using points is folly. 

Don't Spike Defects

I've recently seen a small number of teams create Spikes for defects. That seems strange to me. If you have a good reason for doing this, please let me know.

It should be rare that we’d have a spike to research a defect. Defects almost always have some amount of unknown research and debugging to them. It would seem strange to me to have a SPIKE to debug a problem which would then be fixed under a Defect.

The exception case may be if we want a triage/debugging step where we figure out what’s wrong and estimate the effort to fix before agreeing to fix it. But typically with defects, once you know how to fix it, actually fixing may be faster than all the debugging. By the time you’ve got it debugged, you know exactly how to fix it. I rarely find that approval step necessary these days.

I haven't heard of spiking defects discussed in the agile community so I assume it's generally accepted that this is not a recommended or common practice.

Spike Solutions

I like James Shore's explanation of Spike Solutions. His focus seems to be on very short impromptu experiments (on the order of minutes). One thing about spikes that could use some more words are larger spikes. Sometimes I need to figure something out that may take more than a day. An example might be to experiment with one particular API for some new service I need to consume.

I recommend each team come to agreement on what's an acceptable maximum size for a spike. I recommend a day, or maybe two. If it will take more time than that, split it, just like you would for a story. Spikes need to have a clear acceptance criteria, a very specific question to answer. When the time-box is over, share the results with the team.

Often, such larger spikes are related to a user story and typically must be resolved before the story can be estimated. But for the spikes themselves, I recommend NOT estimating them. We want velocity to represent value added stories -- we want to measure the rate at which we add value. Either way, when using your velocity, when making release plans, you must understand your system and take into consideration when and how you come up with spikes and when you solve them. Do you find them late? Do you know about them but implement them late? If so, that represents unknown schedule risk.

I used to say that a consistent stream of day-sized spikes is good, but when I said that I would often forget to explain my context and assumptions. Having a continuous stream of new spikes that you solve almost as you create them makes sense if you are also doing continuous planning (without batching into large releases or without release commitments). Solving small spikes regularly keeps this activity from making velocity unstable.

Conversely, if you batch requirements into releases greater than, say, a couple months and if your organization is making release commitments, then in that case spikes represent risk in your backlog. Might want to knock out all spikes for the release right off the bat.

------------------

Wow, this is just my first post this year. I've been very busy with my LeadingAgile clients. It's time to get back to my writing!