Wednesday, August 06, 2014

Team Estimation
by Ron Lichty

The fundamental problem with estimating is that, no matter how good our estimates are, they won't take into account all the gotchas that arise when doing actual development. So our goal needs to be: give ourselves a reasonable range that represents best and worst cases - and not spend more time estimating than the exercise is worth.

Estimating is hard, in part, because we developers tend to deep-dive into detailed analysis even though we don't know enough for most or even any of the detail to be valid.

Agile has an answer: we use relative-sizing techniques - how this task compares to a task that we already know the size of, rather than trying to "measure" tasks in isolation - and non-time units, typically abstract Fibonacci-numbered "points”. By keeping point-estimates a measure of effort (not time) and tracking points per sprint, after a few sprints we can begin to leverage the team's history to calculate velocity - and for planning purposes to apply velocity to get rough date ranges.

The most commonly used relative-sizing technique - and the one scrum trainers have traditionally taught - is Planning Poker. There are plenty of Planning Poker descriptions out there - I won’t repeat them. In part, because…

A half dozen years ago I learned another technique from Chris Sims that he learned from its originator Steve Bockman that we all think is better. Perhaps I'm just not skilled enough with planning poker, but in my experience, story sizing is both richer and faster. It’s a technique that has, for me as well as many others, yielded better results in less time with more positive, bang-for-the-buck side benefits.

Generally just called Team Estimation or Story Sizing (or occasionally Rapid Release Planning or The Team Estimation Game), the technique is a two-step process: first the entire team orders the stories by size on a table or a wall; then, after labeling the smallest card a '1', the facilitator moves his hand along the card line until there's a card that's clearly no longer a 1 but twice that, so labels it a '2'; and so on.

I modified the method a bit and I think I improved it, but my changes are nuance to a technique this good already.

Sizing a 120-story backlog
I've worked with teams unfamiliar with agile and not comfortable with assigning "story points" or any other arbitrary value system who were nevertheless quite willing to compare each card with the next to sort them and then size based on '1' being the duration of the smallest card. (Midway through, I heard one say, "This seems like five of the first task.")

There's something very tactile about turning off all the computers and getting everyone comparing 3x5 cards on a big conference table, comparing this card to the others, asking whether longer or shorter. Things get very collaborative very fast - not only efficient sizing, but it generates team-talk as a highly productive side effect.

To my contention that Steve’s method is faster, I took a team unfamiliar with agile through planning their 120-story next-version release in less than two hours. That was both creating a snake of cards from smallest to largest on a conference table, then putting down markers for where 1, 2, 3, 5, 8 and so on started.

To my contention that Steve’s method is richer, there was discussion around every card as it came up ("Why is this card bigger than that one?"), not just where disparities might have arisen in poker "hands".  The team had previously been focused on v.1 and was not familiar with the v.2 roadmap, but by the end of two hours everyone had a sense of the entire project. I don't want to say they "knew" v.2, but I will say I've never seen anything like it for bringing an entire team up to speed fast.

One of the things that struck me, then, was how the shy-est developer in the room was soon up and negotiating difficulty, ease and complexity based on the code she knew. The teamwork and learning, in just two hours, was phenomenal.

I'm told by Scrum conference attendees that Team Estimation is getting more and more shrift these days and may be on the verge of overtaking Planning Poker as the preferred method, at least in this region.

Here's a blog post by another of my colleagues, Chris Sims - how he described it.

Chris has since written a core scrum book and posted this description of the technique as he now practices it - quite a bit more structured than the original.

Better yet, Steve Bockman has written a novel, Predictability, to evangelize good estimating and burndown practices. A short and highly enjoyable read, Predictability resembles The Phoenix Project for software development. Via the narrative, it easily answers the question, "Why do we estimate in points?" It captures the crazy workarounds and multipliers we used pre-agile as a work-around to attempt sane estimates. And its chapter 12 is a compleat guide to Team Estimation.

Your biggest challenge, when you finish reading Predictability, will be realizing that its simplicity actually applies to your project. Trust me, it does. Go back and re-read chapter 12 with your team and apply it.

11 Comments:

At 8:56 PM, August 06, 2014, Blogger Steve Bockman said...

Hey, Ron. Your experience regarding how long it takes to estimate a bunch of stories correlates with mine. I've seen teams estimate stories at the rate of between 1 and 3 minutes each, regardless of the complexity of the stories.

 
At 11:08 PM, August 06, 2014, Blogger Scott Hurlbert said...

One of the issues I have with estimating user stories is that scrum has no feedback mechanism for monitoring whether a team is getting any better at estimating or not. This effects the ability to effectively plan and forecasting.

I find it odd that we can't estimate user stories in hours with any precision, but when we task out the user stories we are willing to then estimate the tasks in hours. I am familiar with all the arguments about tasking being done just in time so that you have as much information as you are ever going to have, but I'm not buying it. I do agree that there is too much chaos in place for us to estimate user stories, but I believe that it is primarily chaos of our own making.

One of the challenges with Agile estimating software is that it's user story focused and not construction focused so there's a mis-match between what is being built and what is being estimated. This practice of functional decomposition has never been a good way to build, design, or estimate and user stories force us down that path. All but the most experienced of teams fall into this trap.

I've been working on trying to identify this boundary - between the functional and the structural. I agree, that to some degree the functional will stay "soft." But the structural should be as estimate-able as the tasks (the unknown unknowns not withstanding).

In this regard, Agile not placing a higher value on the architecture of the solution is troubling. It is in the architecture that we capture and isolate those parts of the system which can be estimated, which control the chaos through limited inter-connections, and which the user stories should sit on top.

I agree with your post, if you buy the premise that we should be working primarily via user stories. I think that gets the situation half right. Our user stories should be enacted on a landscape well built and architect-ed.

When we do create solid structural designs, we control for (most) chaos, dependencies are easier to identify, and estimates become much easier. Estimation of the users stories is easier too.

Thanks for the article. Good stuff.

 
At 11:10 PM, August 07, 2014, Blogger Ron Lichty said...

Steve, thanks! Scott, you raise several good questions; let me try to take them in order.

Scrum has no feedback mechanism for monitoring whether a team is getting any better at estimating or not.

I concur wholeheartedly to there being value in an estimation feedback-and-improvement mechanism. Enough so that, while I haven't run across others teaching or discussing it (though there undoubtedly are), I've given it thought myself. Scrum does, in fact, have a general purpose, every-iteration feedback-and-improvement mechanism. My recommendation to teams I train and coach: include estimating as a recurring topic in retrospectives.

This effects the ability to effectively plan and forecasting.

Ahhh, planning and forecasting. I think by "planning and forecasting" you're referring to the bane of software development that all of us have nonetheless been subjected to, "Tell me when you'll have it all done." Steve's novel opens with a best guess in a hallway conversation suddenly communicated company-wide as a firm date - just like projects we've all been part of. Scott, see my previous blog post, Estimation Has a Cost (and you may want to follow the link from it to Mike Cohn's newsletter article that drove my thinking).

I find it odd that we can't estimate user stories in hours with any precision, but when we task out the user stories we are willing to then estimate the tasks in hours.

Again, see my previous post, Estimation Has a Cost. As Mike points out, we can estimate user stories with precision - but the cost of estimating is directly proportional to the precision requested. Given most waterfall projects delivered only 10 to 45 percent of their hundreds of pages of written requirements, I'm not a fan of spending a lot of time estimating entire backlogs. That would (and in Waterfall did) constitute massive waste. That said, there's considerable value to taking just two to three hours to relative-estimate every story in the backlog: we can achieve a level of accuracy as good as we did when we used to spend days at it - a level of accuracy adequate to inform both velocity and, for our product people, the "I" of "ROI" (the "buck" of "bang for the buck").

I am familiar with all the arguments about tasking being done just in time so that you have as much information as you are ever going to have, but I'm not buying it.

I think we'll have to disagree on this one, Scott. (But see my point about waste above if you want to know where I would start in disagreeing.)

Agile not placing a higher value on the architecture of the solution is troubling.

Despite what we hear from some trainers and coaches to the contrary, agile and scrum are not prescriptive. They advise us to embrace the values and principles to guide us in what we adopt. What they advise against is waste. It's why we do tasking just in time. It's why we don't architect for the entire backlog up front. (Which is not to say we don't architect!) And it's why we don't do waterfall. There was so much waste in waterfall. That said…

I've been working on trying to identify this boundary - between the functional and the structural.

Keep working on that, Scott! It will benefit us all!

 
At 9:23 AM, August 08, 2014, Blogger Scott Hurlbert said...

Ron, thanks for the response. What if it could be shown that the least wasteful way to deliver a project were to plan out it's architecture? The (historic) argument against this has been waste, but that dates back to a misunderstanding that conflates waterfall practices with architecture.

I suggested that functional decomposition is death. If not death, it certainly is a primary source of waste. That I can demonstrate easily and repeatedly. Functional decomposition is the antithesis of efficiency and denies us the opportunity to create a landscape into which we can play out our use cases. You seemed to have skipped this point in your reply.

Let me give a very simple example. I have a use case that says, "As a shopper I want to be able to map a store's location so that I can see where it's located." One of the first architectural decisions I would make would be to decide how to handle mapping. Do I use Google maps, some other service, or build my own?

After some examination I choose Google maps and implement the user story. Things are going swimmingly. Then I get the next user story, "As a shopper I want to be able to see the shipping of my product to my store." For whatever reason this can't be done on Google maps (perhaps something about it violates the licensing agreement, who knows - for our purposes, I'm stipulating that Google maps won't work).

Had we viewed these two user stories together, and decided how to handle this architecturally, we probably would have had less waste. Now we have to replace Google maps with something that will work in both scenarios. Lets say, for the sake of argument, that Bing maps will do the job. We would now need to go back and rip up the work done in story 1, make it work with Bing maps (gotta be consistent) and implement story 2.

Now we start work on user story 3 that says, "As a shopper, I can see a map of the inside of the store and place social comments next to my favorite items." Gosh dangit!! Another mapping story that changes mapping. - continued...

 
At 9:23 AM, August 08, 2014, Blogger Scott Hurlbert said...

...continuation

If only I had read all three stories and created a landscape that included a mapping service that could handle all 3 user stories. Wow, the waste of this process is really starting to pile up. "At least I'm not over architecting my solutions," I say, to myself jokingly and ironically with a wry chuckle.

Here's the rub. Most Agile teams would blow right by this as "well, stuff changes." And they would happily make these "changes" and their velocity would suffer.

In practice, his happens at many touch-points in ways that are much sutler. But this is exactly how most Agile work is done. This is called functional decomposition. By breaking things down function by function, we fail to see the needs of the system until we get to the function that expands it. Reuse is low, and hard to come by.

This is WASTE. This is common. So common that it's the rule more than the exception.

Now imagine estimating in this process. Sure you could estimate that story 1 will take 2 days (or 5 points, or whatever) but you won't be able to estimate that story 2 will have you ripping up all that work and fixing it for the new implementation using Bing maps. That just falls through the cracks. Velocity slows.

But you might say, well we *do* architect some things. We would see this coming and do the mapping service correctly. Okay, let's roll with that. You identify the need for a mapping service and you understand the API you need and you have an activity (not really a story) that says, "Build a service to support StoreMappingApi." You place this in the backlog as a backlog-item (lets assume your tool allows you to do this as many wrongly restrict the backlog to just user stories, though as Bockman pointed out to me the backlog is just "items").

Now you've got to get your product owner to prioritize this item above any dependent user stories. And any other items upon which StoreMappingApi depends need to go above it. This is a very hard thing to communicate to many product owners and the tools offer few ways of managing this ordering issue.

It's also likely that if you're doing very short sprints (a solid practice in my mind) that implementing the StoreMappingApi will take several sprints. Time well spent, but it's a good chunk of work. You'll start to get pressure to break this work up into chunks when there is absolutely no reason to do so. Sure, you can do that, but again - more waste, if you're not careful.

We know how to build software. We just don't do it. Agile works best when put on top of this knowledge. We've thrown the baby out with the bath water. There are areas that work with functional decomposition and areas that work with thought and planning. Many people doing Agile act as if those are the same thing. I believe this happens because Agile is broken (in this respect) and needs to be fixed. Along the way it would largely fix the apparent estimation problem - which is really mostly an illusion.

 
At 11:39 AM, August 09, 2014, Blogger Ron Lichty said...

Scott,

You missed the point of Team Estimation entirely (well, one of them).

Team Estimation, in just 2-3 hours, not only provided the product owner with rough relative estimates for the entire backlog, but the team came away from the exercise with shared understanding of the project roadmap and vision.

It is, in fact, because the team did Team Estimation of the entire backlog, that at the point at which the team asks for detail about the first user story in your example, the team knows not just about the one but about all of the project's user stories. The feature's / project's architect is part of the team. He or she, too, participated in Team Estimation and knows about all of the user stories.

The team also came away with shared understanding because the requirements were phrased as user stories. Unlike the usual decomposition to "features", user stories carry not just "what" but also "who" and "why". Both, particularly the latter, are crucial information to architects as well as to developers, testers, and all other members of the team, and are all too seldom provided outside user stories.

But… you did seem to catch, in your continuation, that the team had knowledge of the entire project, so perhaps you're just describing the behaviors in one or more of the dysfunctional teams you've been part of.

I know you've been part of a succession of not-so-agile "agile" teams, Scott. I've seen my share of them, too. It's what motivated me to write Agile Is Not a Magic Pill, which I was torn to instead name, simply, Agile Is Hard. Teams (human beings?) seem to center in on practices and miss the vales, the principles, the spirit of agile.

One of the not-so-agile practices you seem to have encountered has apparently been consistent communication breakdown between product owners and the rest of the team. This occurs frequently enough that Greg Cohen and I presented an AIPMM webinar four years ago, Prioritizing the Product Backlog, to drive home the collaboration required for good backlog grooming - collaboration we should all expect - and collaboration I train and coach teams (including product owners, who are part of teams) to deliver.

Finally, you refer several times to tool deficits. Scott, stop blaming problems on deficient tools. When tools are deficient, stop using them. I work with teams to start with a card wall. Yes, it has its limitations, too, particularly for distributed teams, but unlike any online tool I've seen, it's utterly malleable. And for colocated teams, it's a much better information radiator. Figure out your process with the card wall. Then, if you must use a tool, find one that serves your process. Don't make your process subservient to any tool's limitations.

Agile is hard because it asks us to work differently - not just to be responsive - not to check our brains at the door (as some previous approaches have sure felt) - but rather to engage our brains and our selves every minute. It does not ask teams to carry out prescriptive rote practices - but it asks teams to self-organize around agile's values and principles for the purpose of embracing agile's spirit to deliver better software sooner and to delight customers.

We know how to build software. We just don't do it. Agile works best when put on top of this knowledge.

True. But not by throwing out Agile with the bath water. Agile works best not by doing Agile but by actually being Agile.

(Have you read Steve's book? Go there now! purchase! read!)

 
At 12:03 PM, August 09, 2014, Blogger Scott Hurlbert said...

Ron,

I did not miss the point of team estimation, I dismissed it entirely as broken before it starts. Therein lies our difference.

If you get a chance, Juval sums it up much better than I can in this video: https://www.youtube.com/watch?v=VIC7QW62-Tw

In my previous post I used the word "illusion" and what I meant by that word was that the process you describe with Team-Estimation gives the team the illusion that they have gone over the entire backlog.

What they have gone over is the functionality required. Estimating based on functionality is like reading a book and then estimating how long it took the author to write it. You know nothing of the author's original knowledge of the topic, what research they did, how long it took to get a publisher, how long their book signing tour lasted - if there even was one, and on and on. This is because - if we are using the contents of the book to represent implemented functionality - none of the above shows up in the book itself.

And so it is with software. On any given project history shows us that the software itself is AT MOST 25% of the work of the project (the actual metrics say that it's around 15%). This means that AT MOST the folks sitting in your Team Estimation session likely considered about 25% of all the work that is to be done.

Is it any wonder that estimation is off by so much?

Here's a real life example from a recent project.

We needed some videos and images - these were key to the copy that was going to next to them - and the copy was key to the page layout, both because the site was adaptive and because of flow control and snap, etc.

So, the software had a dependency on the digital assets. To get those assets it was decided we would do a photo shoot. This was a world class client, so they decided to do a photo shoot with one of the world's most famous photographers. That meant booking a studio, but the product was a car (well - dozens of cars and SUVs) and that meant the studio was really a warehouse - a world class photography studio warehouse.

It meant hiring models. Writing scripts for the shoot and setting up the stage to match the scripts. Contract negotiating with this world famous photographer took weeks. As did booking hotel reservations and flying everyone in. In short, a massive undertaking.

From the developer's point of view - "We need some images and some copy so that the designers can layout the page so that we know how to code it." Tip of the freaking iceberg.

This is how projects work. It's how software projects work. The idea that implementing the functionality is the project must be expunged.

This is not a communication problem.

This is not being "not so agile."

This is not blaming the tools.

I'm enjoying this dialog immensely. I am not disagreeing with the practice of team estimation if you need to do it, but rather the illusion that the Agile teams I've seen think that estimating the functionality is in anyway estimating the work involved in the project.

I hope my example - a real life one - illustrates this point.

 
At 12:08 PM, August 09, 2014, Blogger Scott Hurlbert said...

As for your suggestions that if the tools don't support what you need, don't use them, that's interesting.

I'd love to see how you go into a Kaiser, or a GE, or an IBM and tell them what tools you're going to use with their in house team. I would love to see how you tell them that THEIR PEOPLE will get training (and who will pay for it) so that you can have the tool YOU want.

I've had mixed luck with this.

 
At 12:27 AM, August 13, 2014, Blogger Ron Lichty said...

Greg Cohen contributes this product management perspective:
"In regard to Scott's comments, if the team didn’t know what they were building in the release, why they were building it, who it was for, and how the success of the release was to be measured, the Product Owner or Product Manager failed. Estimating each story is a great way to get familiar with the contents of the release, but this should not be the primary way the Product Owner communicates a release to the team.

 
At 12:45 AM, August 13, 2014, Blogger Scott Hurlbert said...

I don't believe I ever said that the developers didn't know what they were building in the sprint. What I said was that without extracting common functionality into generic(ish) services to create a architect-ed landscape on top of which the user stories can be build, you may fall into the trap of doing very wasteful and brittle construction via functional decomposition.

Juval's explanation of this trap outlined in the video is not to be missed. Highly recommended.

My claim is that estimating each story is a great way to get familiar with about 25% of the actual work. And most of the project work I've seen done has illustrated this point and I suspect a lot of folks would find this to be a common experience.

Is it possible that there's a better way or is this just more of the "scrum is awesome" until we figure out that "kanban is awesome" until we figure out that "lean is awesome" until we figure out that we should quit defending Agile and talk about how projects really should get completed.

If there weren't a better way then TDD vs Scrum vs Kanban vs Lean wouldn't be an improvement.

Why isn't organizing and inspecting only 25% of the work for the project a problem? Is it just that we are so biased toward software that we can't see the work of fully doing the solution?

 
At 12:51 AM, August 13, 2014, Blogger Scott Hurlbert said...

I think my example of why functional decomposition is bad may have been conflated with my example of how the software is often less than 25% of the work.

Those were entirely different examples used to illustrate different points both related to project planning and management.

 

Post a Comment

<< Home