What do you estimate and when do you do it?

Pick up most any book on agile or Scrum and see what it says about planning a product backlog or planning a release or about computing an end date for a project. It will say to identify the stories, estimate them in points, add up all the points, and compare the backlog point total with your velocity to figure out how many sprints it will take to implement all those stories. Then reserve some room for unknowns and for changes (agility).

So, that implies identifying and estimating stories far enough in advance to match whatever your planning and commitment horizon is. This implies that estimating story points in sprint planning or just before sprint planning or even one sprint in advance is likely not far enough in advance unless you are ultra-agile and aren't making commitments or setting date expectations more than a couple weeks out.

Most companies I work with have a planning horizon that is 3 to 6 months out. So, we put together a 4 or 6 month plan. That plan is going to be based on what we can know in advance, namely, user stories. (Sometimes we do this at the feature level, but that's another blog post.) We won't know research stories or spikes or defects or production support in advance. Whatever goes into the plan is what should go into the velocity metric. Stated differently, you shouldn't put anything into your velocity metric that you can't plan for 4 to 6 months in advance. Which means you shouldn't bother putting points on defect or on research stories. If you are going to compare apples to apples, your velocity must exclude unplannable stuff.

That's why I put points only on real user stories. Here is a series of blog posts I did on each of these topics:

Don't Estimate Defects

Don't Estimate Spikes

Don't Estimate in Sprint Planning

Estimating Placeholder Stories is a Bad Practice

Now, some people try to estimate this unplannable stuff. They try to quantify it in points and include it in placeholders in their plan. You absolutely should track this unplannable stuff, and you should reason about whether it will be more or less in the future, but trying to back it into points or plan it using points is folly. 

How Many Stories per Sprint? Rules of Thumb

I'm often asked how many stories you should have in a sprint. People are looking for guidance.

I've heard some coaches recommend “3-6 stories per iteration per developer”. That's a bad rule of thumb. For a team of 7 developers you would have over 20-40 stories which is likely way too many. That kind rule of thumb also subtly takes the focus off of swarming and puts attention toward a developer per story, a story per developer.

My rules of thumb:

5 to 15 stories per sprint are about the right number, particularly for the clients I often work with in which there are maintenance teams knocking down a backlog of small defects and teams doing Web work with lots of small changes to do. 4 stories in a sprint may be okay on the low end from time to time, and 20 is an upper limit for me.

Most stories shouldn't take more than half the sprint to develop and test. Having 1 story each sprint that takes more than half the sprint is about all I would advise, and in that case all the other stories should be very small. For a 2 week sprint, it's better if every story can be completed in 1 to 3 days. (Adjust that for longer sprints.)

I need to elaborate on my comment that it's better if every story can be completed in 1 to 3 days. After stating that I'm often asked whether that's the developers working independently or all together. The answer is "whatever you are doing today." It is best if the team can swarm stories such that multiple developers can work on a story at the same time. If  2 or 3 devs can work on a story at the same time, then you can have larger stories finished within that 1 to 3 day rule of thumb. But if the team isn't there yet, if that's not the way they work today, then having stories that are too big given the way they are working is counter productive.

What's the maximum number of points for a story? How big is too big? How many points is too big for a story varies according to the team's pointing scale. I've known teams that start with 5 (5, 10, 20, 30, 50, 80). For them, 50 and 80 were too big. I've also known teams where a 1 point story would take less than half a day. For them, a 13 might not be too big. If a 1 takes more than a day, then 13 is probably too big. Generally, too big is an order of magnitude larger than the typical small story.

Here's an example: Assume my 1 point story takes a day or two and once in a while we have something that is truly tiny and we call those half a point. The 1 pointer is my typical low end of the range. I have something smaller, but it's not typical. A 13 is an order of magnitude larger that 1 point story. It's very difficult to keep the scale linear when there is that much diversity in your story sizes.

Story Splitting: Where do I start?

I don't always follow the same story splitting approach when I need to split a story. It has become intuitive for me so I might not be able to write about everything I do or what goes through my mind or how I know. But I can put here what comes to mind at the moment:

Look at your acceptance criteria. There is often some aspect of business value in each acceptance criteria that can be split out into a separate story that is valuable to the ProductOwner.

Consider the tasks that need to be done. Can any of them be deferred (to a later sprint)? If so, then consider whether any of them are separately valuable to the ProductOwner. If so, perhaps that would be a good story to split out.

If there are lots of unknowns, if it's a 13 because of unanswered questions, make a list of the questions and uncertainties. For each, ask whether it's a Business Analyst (BA) to-do or a Tech to-do. Also ask for each whether it's easy and should be considered "grooming". If it's significant enough and technical, maybe you should split that out as a ResearchSpike. Then make an assumption about the likely outcome of the spike, or the desired outcome of the spike, note the assumption in the original story, and reestimate the original story given the assumption.

Look in the story description for conjunctions since and's and or's are a clue that the story may be doing too much. Consider whether you can split the story along the lines of the conjunctions.

Other ideas:

• Workflow steps: Identify specific steps that a user takes to accomplish the specific workflow, and then implement the work flow in incremental stages
• Business Rule Variations
• Happy path versus error paths
• Simple approach versus more and more complex approaches
• Variations in data entry methods or sources
• Support simple data 1st, then more complex data later
• Variations in output formatting, simple first, then complex
• Defer some system quality (an "ility"). Estimate or interpolate first, do real-time later. Support a speedier response later.
• Split out parts of CRUD. Do you really really really need to be able to Delete if you can Update or Deactivate? Do you really really really need to Update if you can Create and Delete? Sure, you may need those functions, but you don't have to have them all in the same sprint or in the same story.

Some of the phrases in the above list may have been inadvertently recited from Dean Leffingwell's book "Agile Software Requirements".

My Recent Posts on LeadingAgile

I started working with LeadingAgile in 11/2012 on some complex enterprise agile transformations and other lean management consulting. Related to that, I've done most of my recent blogging over there on LeadingAgile's blog. Here are some of those posts:

I Don’t Estimate Software Defects, but it's not as simple as just that. If you follow my advice, you'll also have a zero-defect mentality and you'll fix defects as soon as you find them. In general, I want a conservative measure of velocity, so I don't record in my velocity anything that was unplanned. Therefore, I likewise Don't Estimate Spikes.

I'm fond of The Theory of Constraints and Brooks’ Law. In that post I evaluated Brooks' Law in light of the Theory of Constraints in a way that I hope helps the reader understand both concepts.

Lack of Predictability is Your Biggest Problem. The senior leadership in most organizations I work with seem to agree. They aren't very interested in agile's agility. They want their design-build-test teams and their program teams to be able to make and meet commitments so they can make appropriate capacity constrained commitments across the portfolio and to external stakeholders.

Agile Health Metrics for Predictability was perhaps my most popular post on LeadingAgile's blog.

In Bottom-Up Implementation & Top-Down Intent, I discuss my recommended approach to agile transformations.

Related to that, I wrote a post arguing that you should fix your Structure 1st: Why You Should Not Start With Practices

Part of your structure involves your design-build-test teams, also known as your Scrum teams. In Use Feature Teams. Yes, Use Component Teams I explaining these approaches and why I recommend using both.

What do Scrum teams do during the Release Sprint? Good question. This post has the answer. In short, many things, but not developing code that can't be tested immediately.

Oh, I also published an article in Agile Journal entitled Identifying and Improving Bad User Stories along with my friend and co-author Chuck Suscheck. We put another article up on sticky-minds: The Problems with Overachievers on Agile Teams.

Spike Solutions

I like James Shore's explanation of Spike Solutions. His focus seems to be on very short impromptu experiments (on the order of minutes). One thing about spikes that could use some more words are larger spikes. Sometimes I need to figure something out that may take more than a day. An example might be to experiment with one particular API for some new service I need to consume.

I recommend each team come to agreement on what's an acceptable maximum size for a spike. I recommend a day, or maybe two. If it will take more time than that, split it, just like you would for a story. Spikes need to have a clear acceptance criteria, a very specific question to answer. When the time-box is over, share the results with the team.

Often, such larger spikes are related to a user story and typically must be resolved before the story can be estimated. But for the spikes themselves, I recommend NOT estimating them. We want velocity to represent value added stories -- we want to measure the rate at which we add value. Either way, when using your velocity, when making release plans, you must understand your system and take into consideration when and how you come up with spikes and when you solve them. Do you find them late? Do you know about them but implement them late? If so, that represents unknown schedule risk.

I used to say that a consistent stream of day-sized spikes is good, but when I said that I would often forget to explain my context and assumptions. Having a continuous stream of new spikes that you solve almost as you create them makes sense if you are also doing continuous planning (without batching into large releases or without release commitments). Solving small spikes regularly keeps this activity from making velocity unstable.

Conversely, if you batch requirements into releases greater than, say, a couple months and if your organization is making release commitments, then in that case spikes represent risk in your backlog. Might want to knock out all spikes for the release right off the bat.

------------------

Wow, this is just my first post this year. I've been very busy with my LeadingAgile clients. It's time to get back to my writing!

Sketchnotes of Agile Immersion

Here are the sketchnotes from my November Agile Immersion workshop.

These sketchnotes were done by a business partner of mine, Jenny Trautman of evenview. Sketchnotes aren't the main thing evenview does -- sketchnotes are an artifact. What evenview does is help people bring innovative strategies into focus. They lead fun, high-energy meetings using interactive visuals that draw a team together. With their support, groups collaborate to develop a shared vision, create a plan for action, and implement their ideas. Use them for your next strategy meeting or kickoff.

By the way, this workshop is designed for everyone. No agile experience or programming skills required. All types of roles have found it benefical: product managers, marketers, managers, office managers, business development executives, developers, testers, CFOs, and etc. Check out this interesting blog post from a recent non-developer attendee.

Use Physical Story Cards

Sometimes you want to use an electronic Agile Application Life Cycle Management (ALM) tool for managing your user stories. There are benefits to using a tool. They have some really nice features, some of which can't be done in the physical realm. And sometimes you've just got to go electronic; you have no choice. But even if you've gone virtual, there are still benefits for having physical cards as well.

I like having my user stories on physical cards. Conversations are richer, and shorter, when you can point and ask "What's the difference between these two stories?" and the answer is "THIS story is about… but THAT story is for…".

Planning poker is great, but the Team Estimation approach is often better, especially if you have a dozen stories or more. I love it when someone can tell the team "I think THIS story is smaller than THAT one and larger than THIS one" and everyone immediately understand which stories he's talking about.

Relative prioritization is easier when you can slide around some physical cards, especially if you need to prioritize collaboratively. "I'm okay with what we've laid out here, but if you think THAT story is THAT high a priority, you are nuts!"

I recommend having a pre-planning meeting to help make iteration planning will go well. That first pre-planning before a team's first iteration planning meeting is particularly difficult. You don't know what your velocity is. You don't know how much you can get done. You may be unsure of dependencies. All this agile stuff is new. And you need to get several people to feel confident that the 1st sprint planning will go well. You have this large backlog of stories and you want to be sure everyone understands what we're sure we're putting in, what we're considering to put in, and what we're not putting in. So lay it out on the table. I love being able to slide cards around on a table and to be able to literally put something aside for now if it's giving us grief. It's not off the table, it's not out of sight, it doesn't look like it's still in the list, but it won't be forgotten. "Let's come back to THIS one later."

Story mapping just has to be physical until we get inexpensive wall-size touch-screen monitors and story mapping software.

I've worked with clients that were very happy with their ALM tool and large monitors. I was impressed with their dedication to making agile work, their open space, and these monitors for each team. But as big as the monitors were, it still just seemed easier to me to manipulate and read fat sharpie on index card. And I can write SPLIT on a card faster than I can tag it as too big in an ALM tool.

Sketches are interesting. A picture is worth a thousand words. Would be harder to do this on anything other than paper.

But using an ALM tool doesn't mean you can't have physical cards too. It's pretty easy to write the story title and estimate on an index card. You can write the story's ID number from the ALM tool on the card. You can also print story cards from some of the tools. VersionOne has always had a printing solution, but the version coming out in August 2012 will have the ability to print directly from the tool.

I've heard of other tools that make it easy to print, but I can't name them off hand. If you'd like to mention your tool in the comments, please do.

Lots could be written about keeping physical cards and an ALM tool in sync. I'm going to ignore keeping a Scrum card wall in sync in order to keep this post on the short-side, but I will add the following. If you are using physical cards for a prioritization or sizing exercise then just enter the updates back into the ALM tool and discard the cards -- or hang on to them for some other purpose if you want. The point is, the cards served their purpose, they are transient, update the tool and you are done. I could also approve of doing a story map at the start of a small release, then leave it on the wall as is as a historical vision document while the ALM tool becomes the plan of record. I also recommend sticking with physical cards through Iteration Zero and Iteration One. What's cool in I0 and I1 is using the same cards for socializing, estimating, prioritization, pre-planning, and finally planning and the daily standup, then seeing a gold star put on that sucker once it's done.

Bottom line: Keep using physical cards. I promise it will be worth it.

Consider Stories at Multiple Levels

I often work with teams adding new large features to existing applications. It is common for teams like this who are new to agile to go into too much detail too soon. It seems that those accustomed to waterfall believe you've only got one shot to discuss any requirement, so they must discuss it in detail.

In agile you get many looks at each user story (requirement), preferably from many different angles and ultimately getting progressively more detailed as you go. I like to explain it with these four levels.

VISUALIZE -- The Concept

We typically start with a project name or enhancement name, don't we? It's the handle by which we call this thing. From there we hope to have the vision, a short narrative for what we're after. Perhaps we have the name of a few super-epics. Refine that into some epics. Begin to understand the minimal marketable features (MMFs). Metaphorically, this may be an aerial view of the forest, or maybe the planet. Keep the vision in front of the team throughout the project.

PRIORITIZE, ORGANIZE and SOCIALIZE -- The Forest

By prioritize and organize, I mean use any approach to help the team see how the stories fit into the vision and understand the minimal marketable feature set. An example technique is Story Mapping. Understand your epics. Metaphorically, this is understanding what the forest full of trees looks like.

None of these are a one shot deal. Iteratively refine your prioritization. Tune the MMF and re-socialize stories as you learn and get feedback. You aren't done socializing just because sprint zero is over. For anything of size, you don't get done socializing in just one meeting. Nor does all the socializing happen in meetings.

SIZE -- The Trees

When sizing (with abstract relative points), look at your stories in a little more depth, but not to the level of detail described below. Look at it just enough to give an estimate relative to the other stories. You don't need as much detail for a relative estimate as you would for an hours/days/weeks estimate. You quickly get to a point of diminishing returns. Consider that we use a Fibonacci series or something similar -- we use a diminishing level of precision as the stories get larger. Once you are pretty confident that it's a 21 point story, you don't need to spend more time to make sure it isn't really a 13 pointer.

As with the above, you may look at a story for sizing multiple times. If you've sized a bunch of stories at the start of a release, you should re-size the remaining work after 3 to 5 iterations. By then you've learned more. You will have become more familiar with the project, subject matter and the affected code.

Sizing also shouldn't be expected to be static. How big a 3 point story is will shift over time, especially but not only because of changes in the team. Hopefully the individuals are getting better, smarter, and the code is getting cleaner. Also, the work that we're comparing stories to changes. Point-drift happens. Old estimates are like milk -- they don't get better with time.

PLAN -- The Details

In iteration planning we break stories down into tasks. We estimate tasks in hours or half-days or maybe in pomodoros. We think in terms of detailed design and code and test cases.


Understanding that we're going to see each story multiple times with progressive levels of detail helps overcome the desire to go too deep too soon.

What approaches have you used to help people think about stories at the appropriate level?

Yet Another Agile Reading List

(Updated October 2018)

I'm often asked for an agile reading list and each time I wish that I had already blogged it. What's kept me from doing it so far is that I tend to write a somewhat new list each time depending on the requester's level of experience, unique situation, and context. Such a list is also likely to change over time, but that shouldn't stop me from posting it -- this post is my thoughts at this point in time. My thoughts are likely to change.

That said, and without further ado or proof reading, here is an agile reading list.

Getting Started
Just go read the Scrum Guide. If you've never read it, you really should. It's short. But you might need something beyond that to gain a better understanding of how and why agile works, so don't stop there...

I could make this easy for me and just say read Cohn's signature series plus the Beck signature series. That'd keep you busy for a while and cover the topic pretty well.

But I can put a little more effort into this. Don't let this long list scare you. Start with this 1st block, then supplement with some of the others. I'm writing this from the point of view of helping a total agile novice get started.

Most importantly, you need to understand the agile values: http://agilemanifesto.org/ and http://pmdoi.org/ and either http://www.extremeprogramming.org/values.html or http://www.softwarereality.com/lifecycle/xp/four_values.jsp or http://xp123.com/xplor/xp0209b/index.shtml.

Here are some links about agile in general and two specific agile processes, XP and Scrum, and one lean/agile approach called Kanban. Just read as little or as much of these as you desire to get the gist of what it's all about:
http://en.wikipedia.org/wiki/Agile_development
http://en.wikipedia.org/wiki/Extreme_Programming
http://en.wikipedia.org/wiki/Scrum_%28development%29
http://en.wikipedia.org/wiki/Kanban_%28development%29

You need a good understanding of the Scrum process since it's the market share leader right now. I really really like Kenny Rubin's way with words and his Essential Scrum book is excellent. This is good: Succeeding with Agile: Software Development Using Scrum by Mike Cohn. Subscribe to Mike's blog.

Here's a good short, quick, easy to read intro to Scrum: "The Elements of Scrum" by Sims and Johnson. It includes some extra stuff that you really should do if you are using Scrum. I liked that. Well written.

I think I'll throw in http://scrumpocketguide.com/# from my friend Peter Saddington. I haven't read it, but it sounds short. :-)

Alternatives or Slightly More Advanced Items
Subscribe to my blog and LeadingAgile's blog :-).

These next few are a bit old, but you need to have read something by "Uncle Bob" and Alistair. You could substitute some other book by them if you wish:
Agile Software Development, Principles, Patterns, and Practices by Robert C. Martin.
Agile Software Development by Alistair Cockburn.

These eXtreme Programming books are great and short and solidly apply to Scrum:
Planning XP by Beck.
XP Installed by Jeffries.
XP Explored by Wake.
Don't get the original white XP Explained book by Beck. It's talks about Load Factors and Ideal Engineering Days which I don't recommend.

If you want to broaden beyond Scrum and XP, read about Crystal Clear and the other Crystal methods by Alistair Cockburn.

Product Owner
Now we move beyond the basics into some specifics. First, let's take a look at the Product Owner related literature:

http://www.agilemodeling.com/artifacts/userStory.htm

Agile Estimating and Planning by Mike Cohn.

User Stories Applied: For Agile Software Development by Mike Cohn.

Here are some great posts from my colleague on The Product Owner:
http://www.leadingagile.com/2011/03/why-a-product-owner-team/
http://www.leadingagile.com/2011/03/product-owner-team-who-needs-to-play/
http://www.leadingagile.com/2011/03/one-real-life-product-owner-team/
http://www.leadingagile.com/2011/03/product-owner-team-design-considerations/

Scrum Product Ownership by Robert Galen. It's not an intro to agile. It's a good, thorough book on product ownership. It's full of good advice and tips and warnings against certain dysfunctional behaviors. The book contains some typos and passive voice that will annoy some people.

The Product Owner role in agile is a big job. And very important. Many say it can't be filled by just one person because no one has all the necessary skills. They say you often need a team of people with diverse skills and responsibilities to do the job well. I tend to agree. Here is some food for thought:
http://blog.xebia.com/2008/05/22/scrum-the-mythical-product-owner-role/

Scrum Master
Next, someone on your team needs to understand retrospectives in depth. If you don't get what you need to know out of the above books, this one is wonderful: Agile Retrospectives: Making Good Teams Great by Esther Derby. Subscribe to Esther's blog.

I highly recommend Scrum Mastery by Geoff Watts. This is an excellent book filled with very practical and though provoking advice.

Lyssa's book, Coaching Agile Teams, is very useful, but the novice should read the others first.

Technical
Here are some more assorted topics:

Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin. And check out her sequel also: More Agile Testing.

Read Beck's TDD book, Fowler's Refactoring book, Evans' Domain Driven Design book, and Williams' Pair Programming book.

Specification By Example by Gojko Adzic should be in this list as should The Cucumber Book by Wynne and Hellesoy.

Lean
At some point, you might need to know what people mean by Lean as it applies to software. Poppendieck wrote the books on it:

Implementing Lean Software Development: From Concept to Cash by Mary Poppendieck

Leading Lean Software Development: Results Are not the Point by Mary Poppendieck.

To really understand lean, really and truly, read Toyota Kata by Mike Rother. This is about getting everyone in the org to make process improvement part of their daily job, to be very intentional about understanding the current and target condition, and to be very intentional about mentoring everyone about everything I just mentioned. This text will help you have a proper understanding about everything else you read about lean, all the techniques, all the metrics, all the approaches. 

Then read The Goal: A Process of Ongoing Improvement by Eliyahu M. Goldratt, a delightful book. We also have his Theory of Constraints (ToC) text. Goldratt is becoming a very important person to have read, at least for lean/agile consultants.

However, if you read Goldratt, you must also read The Principles of Product Development Flow by Reinertsen. Reinertsen takes exception with the over simplified or at least naïve implementations of ToC (et. al.).

ToC brings us to the kanban process which is gaining popularity. Kanban by David J Anderson is a wonderful book,but lacks specifics on how to do the metrics when the rubber meets the road. Subscribe to the kanbandev yahoo group. Anderson's Agile Management I would say is more advanced and should be read by serious agile managers or coaches. It's a good cross section of lean, pre-kanban, ToC, etc.

Rother's book Learning To See is about value stream mapping from a manufacturing perspective. I enjoyed working through it, but i'm weird that way. Most people in software development organizations won't need this.

Scale
There are books on scaling agile: Practices for Scaling Lean & Agile Development by Larman and Vodde. And Scaling Software Agility, Leffingwell.

Manage
Management 3.0 by Jurgen Appelo is the most important book for anyone leading or managing IT knowledge workers.

Dan Pink's Drive: understand autonomy, mastery and purpose.

Not sure whether to put DeMarco's Slack, Getting Past Burnout, Busywork and the Myth of Total Efficiency here or up under Lean. I have a blog post on slack.

Fin
Sadly, I cannot recommend this book, even thought I have a chapter in it: XP 2003 Proceedings.

Finally, find a local agile/lean/scrum user group to attend and make connections. I can tell you all about the ones in Atlanta and can connect you to people who know all about the ones in Ohio and Chicago and RTP, NC.

There are probably a thousand other folks with their own agile reading list. I wouldn't mind it at all if you were to post your list here or link from here to your site.

Sizing with Incomplete Information

Some teams that I work with that are new to agile struggle to size their first big agile release. The team is often still trying to gel, assimilate new members, and increase their velocity while spending lots of time creating, understanding and refining the product backlog. One of the challenges they face is learning to estimate with imperfect and incomplete information. How quickly they adapt seems to depend on what they are presented with during their 1st agile/relative estimating session.

Some teams new to agile start with a new project and need to estimate the whole thing with little info, perhaps during a "sprint 0". Those teams are forced to deal with ambiguity right from the start. From then on they accept that they'll have to estimate fuzzy stories. This helps them accept that stories will change, both the details of individual stories as well as what stories make up the backlog. Sometimes revealed detail changes an estimate (up or down). Sometimes it doesn't. But that's life and we can deal with that quite well on an agile project. More on that in a moment.

Other teams I work with already have a project well underway. They have a product in maintenance when they go agile. They've got lots of small and well understood stories. Or if they aren't well understood, they can get that way quickly. The 1st stories they estimate are detailed and certain. But a day will come when the team is forced to estimate some new release or some new product with less than full information. That can be frightening. As these estimates are recorded, we can note that the estimates were made with less info and certainty, but that seldom provides much comfort to this type of team.

However, what you'll find (on a healthy team) is that:

• Large estimates on the larger stories tend to drive the Product Owner to decompose those behemoths and then ask for new estimates.
• The team is allowed and even encouraged to revise the estimates (up or down) as more information is revealed.
• Many of the estimates will not need to be revised. Any additional detail or certainty often doesn't affect the estimate after all.
• Consistency is good, but accuracy and precision are not necessary. Wrong estimates have an offsetting impact on velocity. That's the beauty of using relative estimates and the empirically measured velocity to compute a release end-date -- an incorrect increase in one gives an offsetting decrease in the other.
• There will be details that are overlooked. Developers tend to be optimistic with estimates. But that too will soon be taken into account in the velocity (unless you implement all the well understood stories 1st and always save the others for later). (Do note, however, that the shorter your release is the shorter your iterations need to be in order for a change in velocity to be noticed in time.)
• And management will heed the warning signs when presented with the estimates and velocity and not pressure the team into short-cutting craftsmanship, quality or sustainable pace. (Remember that I said /healthy/ team.)

Seeing is believing

On the second type of team this learning happens slowly. You can have a conversation around these things to try to speed up the learning or to provide some comfort. It's certainly worth trying. But don't be surprised if your convincing arguments are met with disbelief.

Scrum Fundamental: Measures of Progress

In my user stories blog post last week I claimed that problems on Scrum teams are often due to lack of knowledge about a Scrum fundamental. Today I’ll tackle the next item in my short-list of common problem areas:

• measures of progress,
• delivering value,
• demos,
• getting feedback,
• keeping it working,
• continuous integration,
• and testing.

Yeah, so, that’s more than one item, but one kind of leads to the next.

Requirements and Design

One way, the traditional way, to try to build trust and understanding between a development team and the business or client is to spend a lot of time on designs, analysis, and wirefames. The problem is that it’s difficult to predict the future with much accuracy. You will likely miss details and misunderstand how the app will be used in practice. Requirements and priorities will change.

The agile prescription, therefore, is to not spend much time figuring out details for some story that you may never implement or that may change drastically in the ensuing months. Understand the stories at a high level. Get into development and begin delivering working software as soon as possible. Get feedback sooner and begin reaping the benefits of the software sooner.

Defer any decision and any work that can be deferred. You’ll be smarter next month, you’ll understand the system and the requirements better. Let the design speak to you, refactoring as you go.

This would be a good place to say something about lean and systems thinking and Pillar Technology’s “Business value over working software” appendage to the Agile Manifesto and about not building software when it would be better to redesign the system, without IT, then automate what makes sense. But that’s another whole post.

One good thing about working software is that you can have demos.

Demos

Favor demos over status reports. After completing each small story, demo it at least internally. Get to where you can have frequent (weekly or bi-weekly) demos to your clients/users. That early feedback is very important. This also holds staff accountable for quality, for “keep it working”, and for getting to done.

The demo isn’t for the Product Owner. Your Product Owner should have been involved with the team while the story was being implemented. Your Product Owner should have seen the story before the demo. There was an interesting discussion on the Agilistas LinkedIn group about who should do the demo, and why.

Keep it Working

Employ the “keep it working” principle. Build a usable system early on and keep it working continuously. The source in the version control system must always be usable. Set up an automated build/test process using a tool like CruiseControl, Cruise, or Hudson. Have it inform everyone when the build breaks. You should have a demoable product at all times. At the end of each iteration you should have a shippable product.

This requires automated testing. Pick a level with which to start coding test cases. Most often this is at the lowest level, unit tests. I recommend the xUnit framework or RSpec, but there are other good choices. [Hmmm. I feel compelled to say something about cucumber or BDD, but I intend this to be a fundamentals post. BDD ought to be fundamental, but it isn't as fundamental as this other stuff, particularly if I'm talking about Scrum. Admittedly, this post has left Scrum and is off in generic agile-land. But I digress.]

Sometimes, however, your methods are too large and have too many dependencies to be tested well at this level without first doing much refactoring. In those cases, automate your tests at a higher layer, then refactor and employ unit testing as you go. I’ve used Selenium and Rational Functional Tester for UI-based testing, but there are other choices out there. Working Effectively With Legacy Code by Michael Feathers comes highly recommended.

If you do a good job here, you’ll reduce the amount of manual testing you have to do. You can limit that to usability, break-the-box, and exploratory testing rather than regression testing.

Just a short word about testers. QA professionals think differently and approach testing in ways that developers do not. I like getting QA involved with the developers, pairing. I like getting development involved with anything that QA is taking the lead on automating. And I really like getting QA involved with the Product Owner to help define good stories and good acceptance criteria.

Parting Note

I almost abandoned this series of posts. There’s really nothing new here. I’ve said nothing better than it has been said before. I’ve just said it differently. Sometimes ‘different’ resonates with someone who hadn’t understood the way it was put to them before. So, I’ll press on with these posts. Unless I change my mind.

Scrum Fundamental: User Stories

In my line of work as an agile coach I get to help people who are just starting to use Scrum. Some common problems crop up with many of these teams. If often comes down to some Scrum fundamentals. Fundamental doesn’t mean easy. Professional athletes work at the fundamentals and so must we.

If I had a short list of common problem areas, the neglected fundamentals might be: good stories, “keep it working”, getting to done, collaboration, and planning. That’s a bit much for one blog post, so, maybe over the next few days I can address these in a series of posts. Here’s the first.

User Stories

Analysis of a business problem yields features that may either be “stories” or that may be broken down into stories. The first pass will have stories that have some very important parts as well as some less important parts. Identify and implement the most important stories first and address risk first. Defer as much as possible to the end of the project.

That sounds obvious, but it’s so easy for a developer to “just add this in while we’re working in this area.” It’s seductive to the Product Owner to want to “round out” some feature set. So many useless features and dead code gets bought and paid for that way.

Maximize the amount of work not done. Be ruthless in deferring what is less essential until later; you might not ever have to do it. Don’t build frameworks in advance or build support in for features you plan to work on later because You Aren’t Going to Need It – the well known YAGNI principle.

Split Stories

The approach here is to split your stories. The “User Management” story becomes “Add Users”, “Delete Users”, and “Modify Users”. You might never implement Delete, and Modify isn’t needed in the first release. You may further split Add User into basic and wiz-bang versions. This can likely be split further to deal with different types of users, assorted permissions, error cases, cascading, and more complex attributes of the user. After splitting, implement the most valuable stories first, of course.

Each story should have some value to your business/client/customer, particularly to an end-user. Each story should deliver some function from top to bottom. It should be a complete vertical slice through your application. Don’t divide stories along architectural lines. Avoid technical/horizontal slices (the DB framework, the complete object model, all the UI components). For the “user management” story, don’t have a story for the UI and a story for the backend.

Spike Stories

Use spike stories when you need to figure out how to implement or estimate something. A spike is a short time-boxed experiment or research, with a specific objective to address unknowns and risks. The conclusion of a spike may beget more stories and spikes.

Rules of Thumb

Some rules of thumb for story size:

• One story should not take the whole team more than half the iteration.
• The team should be able to finish any story in one to two days for a one-week iteration.
• The team should be able to finish any story in one to four days for a two-week iteration.
• Aim to have 3 to 8 stories for a one-week iteration.
• Aim to have 3 to 8 stories for a two-week iteration.
• If most of your stories can be done by a person in half a day, they are too small.
• If your team is doing more than 8 stories per iteration, your stories are too small.
• If your team is doing more than 9 stories in a three-week iteration, your iterations are too big.

Remember, these are just my rules of thumb. There are valid exceptions.

Summary

Poor stories will make Scrum painful. I’m not the first to write about good user stories, not the most eloquent, and won’t be the last. If what I’ve written describes your problem, read more about it or find a good coach. I’m available.

Question for the day

Disagree with my recommendation? Have a better way to word this? Hate my rules of thumb? Have a better blog post on user stories? Post your thoughts or post a link.

You break your stories down /when/ ?

http://www.flickr.com/photos/pointshoot/ / CC BY 2.0

I've had a revelation. The other day I was talking to Mike Cottmeyer about Scrum and how I don't like the idea of tasking out and estimating all stories at the start of the sprint and then signing up for tasks, all before committing to the Sprint backlog.

When I first started doing XP in early 2000 we tried that and it was painful. Painfully long. Painfully tedious. And it felt painfully like Big Design Up Front even though it was for one or two weeks worth of work.

One team I was with for a handful of years broke our stories down into small 1 to 6 hour sized tasks and we pair programmed (up to 10 of us) on a limited number of stories (preferably 2, normally 3-4, but it could get up to 6-8 near the end of the iteration if something got blocked and if we had lots of abnormally small stories in the iteration). One person with an optional pair would design a story just in time and then walk the team through the details and proposed tasks and get their input. All the tasks were written on the whiteboard and this drove all development. Tasks were small and explicit:

• extract foo.class from fred.class
• extract bar#makeGenerous from #makeMagnanimous
•  introduce strategy pattern to someOther.class
• create someNew.class
• domain/persistance/patch for someNew.class
• write the acceptance test
• etc.

That design work could take up to a day or two to do for some of our stories. There is no way it would make sense to design all of our stories at the start of each iteration before committing to the iteration. That's how we did it using XP.

Tasks in Scrum, however, seem to be around 8 to 16 hours long, according to what I'm reading. And that was the new insight I gained. Some teams do things differently. (I'm slow sometimes, but I usually catch on.) With tasks that size, you could easily identify the tasks and estimate them before committing to the sprint. Further, teams that don't pair-program or that have too much specialization or too much code ownership need to break their tasks down by specialty and assign each task to the person that can implement it. Such teams would need to do this step before committing to the sprint.

I was blinded by my experiences being so similar to one another for such a long period of time. Help me see. How big are your story's tasks and how and when do you estimate them?

Printing Story Cards from VersionOne

VersionOne doesn't provide a direct way to print your stories on index cards, but it can be done. I'm going to tell you about the browser approach and the export approach. For the browser approach, getting set up to print your stories on index cards is a matter of figuring out what URL to hit to get just the stories you want to print. Then you can bookmark that URL for later reference and print the cards directly from the browser. The disadvantage is that it's a few steps to change your browser's print setup each time and you'll want to reset your browser's print settings afterward. But you can print on standard stock index cards if your printer will allow. I'll give you the details in a moment.

The export approach involves downloading Avery's print wizard and setting up a template. Then each time you want to print your stories, go to the view in V1 that has the stories and columns you want to see on cards. You'll then do a few steps to export the stories into a spreadsheet and use the Avery tool to format the stories (into another document) and then print them on special Avery stock. The advantage to this approach is being able to select which values you want on the card and where you want them. That's all I'm going to say about this approach.

The Browser Approach

To keep this from getting too complex, I'm going to describe what you need to do if VersionOne is hosting your instance. If you are hosting it yourself you should be able to figure out what to do by reading the original post.

Generally, we are going to get all the stories formatted as cards on a web page, then we're going to print from the browser onto cards. To see all of the stories in the system, use this URL:
https://servername/v1instance/rest-1.v1/Data/Story?xsl=/s/custom/storyCards.xsl
For this and all the URLs below, you will need to replace "servername/v1instance" with your actual server and instance name. (Hint: look a the URL you use to access V1.)

Of course, you most likely will rarely want to print every story card. Thankfully, you can add some filtering. I'll provide some examples. You may have to do a little guessing, experimenting and URL manipulation, but I'll tell you how to approach that.

You can control which stories are returned by adding some query strings to your URL. For example, to see the stories for "Sprint #1", use the URL
https://servername/v1instance/rest-1.v1/Data/Story?where=Scope.Name='Sprint %231'&xsl=/s/custom/storyCards.xsl
Note the %23 in place of the hash symbol. You'll need to handle special characters in your URL appropriately.

To see the stories for "Iteration 7":
https://servername/v1instance/rest-1.v1/Data/Story?where=Timebox.Name='Iteration 7'&xsl=/s/custom/storyCards.xsl

To see the stories for "Sprint #2" that are OPEN (64):
https://servername/v1instance/rest-1.v1/Data/Story?where=Scope.Name='Sprint %232';AssetState='64'&xsl=/s/custom/storyCards.xsl

To see the stories for Iteration 6 that are OPEN (64):
https://servername/v1instance/rest-1.v1/Data/Story?where=Timebox.Name='Iteration 6';AssetState='64'&xsl=/s/custom/storyCards.xsl

To see the stories for Iteration 6 that are CLOSED (128):
https://servername/v1instance/rest-1.v1/Data/Story?where=Timebox.Name='Iteration 6';AssetState='128'&xsl=/s/custom/storyCards.xsl

You may want to do more filtering than in the examples here. To see what all the attributes and values are, leave the xslt out of the URL as follows:
https://servername/v1instance/rest-1.v1/Data/Story
There might be something useful to you in the Meta:
https://servername/v1instance/meta.v1
And you may need to refer to the Meta API and the ReST API documentation for V1's sdk as well as the article Getting Started with the API.

Now for Printing

Now that you have the stories you want to print formatted as story cards in your web browser, it's time to print them. This xslt we are using is set up to print two stories per page on letter size paper. If this is acceptable to you, you'll likely need to scale the print job to 80% or 90% when you print. Alternatively, you could go the extra mile and set up your browser to have no margins and no header or footer. But if you do that, you'll likely want to reset all of that once you are done printing.

That is certainly the easiest approach if you have a paper cutter handy, but with a little more effort we can print those directly onto stock index cards. To print those on a 3x5 card, set your browser as follows:

• page setup: landscape, no margins, no header or footer
• print, properties, paper: 3x5
• print, properties, effects: leave set to actual size
• print preview: scale 50%

Put the index cards in the printer in portrait orientation. And print.

Question of the Day

Have you found any other approaches to print story cards from VersionOne?

Iteration Planning – Story Breakdown

There is a certain set of things you should consider during Iteration Planning when breaking stories down into tasks. Here's a short check-list I created a while back to post in the XP-lab as a reminder. Other teams have done the same but with different items to consider. In our retrospectives we notice things we should have thought of and those items get added to this list.

Consider:

• Acceptance tests
• Risks
• DB impacts
• Design sessions needed
• ...
• Error conditions
• Boundary conditions
• Time-zero
• Front-end/back-end impact
• Permissions
• Audit logs
• ...