The most convincing reason to change from waterfall to agile by Ron Lichty

Given no one on product teams believes in waterfall - even those using waterfall (see my last post on The 2013 Study of Product Team Performance) - what to do about managers who insist upon seeing a waterfall project plan so they can "guarantee that a project will come in on time"?

There is, of course, the response I heard attributed to XP creator Kent Beck, "How's that working for you?" I've used it myself, since first hearing it. It certainly applies. And it's a question that does make managers think - at least some of them.

But I suspect there's a swath of senior managers who, despite all evidence to the contrary, believe if they just hired better project managers or got their developers to arrive at the office when their salespeople do, they would meet their budget and schedule targets with room to spare. Another swath of senior managers seem so desperate for an anchor to hold onto that they grasp for Industrial Revolution management style hoping against hope that this time it will work for them.

But as Mickey and I noted in our book, "…the Waterfall model - in which each phase must be completed before the next one begins - was long ago shown to be an inadequate match for the reality of how software needs are understood and specified. In both of our careers, of the hundreds of projects we have worked on, each of us has seen just one project that was fully and completely described before coding began. Both cases were over 20 years ago. It virtually never happens."

It was collecting data from large numbers of developers and other development managers that truly knocked my socks off - converted me from a fan to a believer.

My first response to waterfall adherence these days is recounting my data gathering story - in part because I find stories can be easier for disbelievers to hear.

The story I tell - and the data I continue to collect - comes from speaking frequently to groups of developers and development managers.

I often ask them, "How many of you have been given a 400-page requirements spec?"

Virtually every hand in the room will go up.

I then ask a second question, "What percentage of those 400 pages of requirements did you actually deliver?"

Short of the occasional ringer (I've had a couple NASA engineers say 95%!), the responses seldom exceed 45 percent - are most typically 15 to 25 percent - and range as low as 10 percent of spec requirements delivered.

If that's not enough, I then ask, "Who decided which 15-25% was delivered?"

One of the things I like about Agile is that the top of the backlog is ordered every sprint by product owners collaborating with development leaders to ensure that the team is always working on the highest value stuff.

The answer in a waterfall scenario, on the other hand, is almost never value. The answers I get to my question include, "the easiest stuff to code," "what was most interesting to us" (what was most shiny!), "what stood out in our minds from reading the requirements," or "what was the most fun." Prioritization in 400-page waterfall requirements is almost universally done by developers, not product owners.

(Think back to your own 400-page requirements document: did it tell you what 15 to 25 percent was most important?)

But what truly clinched it for me - what converted me from enthusiastic about agile to downright disenchanted with waterfall - is waste. Waste of resources, waste of development time, waste of effort. When projects deliver only 15 to 25 percent of requirements in the spec, it means that:
    ▪    75 to 85 percent of the requirements-writing effort by product managers and business analysts was waste (and delayed getting development started by 3-5 times for having to wait on writing up all the undelivered requirements).
    ▪    75 to 85 percent of the work technical architects and designers did to craft technical specifications was waste.
    ▪    75 to 85 percent of QA's time writing test plans for the whole spec was waste.
    ▪    75 to 85 percent of visual design of wireframes for the entire application was waste.
    ▪    Project managers created project plans to deliver 100 percent of the requirements by the deadline - 75 to 85 percent waste (not to mention irrelevant).
    ▪    Developers, who need to read and understand the context of everything that goes into an application, struggled to get the big picture of all 400 pages - 75 to 85 percent of which they would not deliver - and seriously compromised their ability to do the 15-25 percent well.
    ▪    Not to mention customers, who thought through everything they might want but who really wanted the most important parts often before product managers and BAs even started writing requirements.

Waste is what convinced me. And waste is what, in my experience, brings around the disbelievers.

There's a reason that, when we asked about methodology, in this year's Study of Product Team Performance, those groups using waterfall responded that any of the methods other than waterfall would make their product more profitable. They've seen waste up close and personal.