Wednesday, September 17, 2014

What Makes High Performance Product Teams Successful?
by Ron Lichty

What does it take for a team to achieve high performance?

What makes high performance teams stand out from the rest of the pack?

For three years we’ve been surveying teams to find that out. We looked to those teams that told us they’ve achieved high performance and asked them to identify their characteristics. We asked the same questions to the teams that told us they hadn't. Then we conducted regression analysis on the most intriguing data to identify what high performance teams had in common and what differentiated them from the rest.

That's what makes up the now-annual Study of Product Team Performance.

In 2012 my coauthors (I didn’t join the authorship team until 2013) discovered five correlative factors:  67 percent of high performance teams do all five. (Further validating the results, only 2% of teams lacking all five factors were able to rise to high performance). My coauthors also asked what one thing product team members would change about their team; the number-one response: "Better cross-functional collaboration, trust and communication.”

In 2013, we expanded our understanding of four of the five factors of high-performing teams (the fifth had been so strong it needed no clarification). And a sixth factor - and four landmines that can undermine teams - emerged. Equally intriguing, we cross-correlated each team’s methodology with the methodology that the team believed would make their products more profitable. The finding: regardless of methodology, team members identified waterfall as having an adverse affect on profitability; even most teams using waterfall would, for profitability, opt for anything but waterfall.

This year's study is just released. The results: an additional set of factors found in high performance teams:
  • Positive team culture (and 5 elements essential to optimizing culture)
  • Positive relationships with sales and marketing teams (and some insights into how organizations foster inter-team integration and health)
  • Positive relationships between product and project managers (and why they turn out to be so important)
  • Understanding and integrating with the sales cycle
We also looked, this year, at how organizations are using return-on-investment (ROI) – or not using it as turns out to be overwhelmingly the case. And we continued to trend product development methodology adoption rates for Agile, Waterfall, blended and Kanban - observing a very interesting pattern emerge from three years' data, with implications for how these methodologies are likely to evolve going forward.

Lest you think that these results were unique to new products versus enhancing existing ones, it turns out that respondents were almost evenly split between the two. Another demographic statistic: a fifth of respondents were working on entirely internal products.

This year’s study - and both previous years’ studies - are available from lead author Greg Geracie's Web site at:

Many thanks to my other Study co-authors:
  • David Heidt, Principal, Enterprise Agility and former President of IIBA Chicagoland
  • Matt Jackson, President, Jackson Consulting
  • Gary Pickens, VP and Chief Research Officer at Truven Healthcare Analytics

Wednesday, August 06, 2014

Team Estimation
by Ron Lichty

The fundamental problem with estimating is that, no matter how good our estimates are, they won't take into account all the gotchas that arise when doing actual development. So our goal needs to be: give ourselves a reasonable range that represents best and worst cases - and not spend more time estimating than the exercise is worth.

Estimating is hard, in part, because we developers tend to deep-dive into detailed analysis even though we don't know enough for most or even any of the detail to be valid.

Agile has an answer: we use relative-sizing techniques - how this task compares to a task that we already know the size of, rather than trying to "measure" tasks in isolation - and non-time units, typically abstract Fibonacci-numbered "points”. By keeping point-estimates a measure of effort (not time) and tracking points per sprint, after a few sprints we can begin to leverage the team's history to calculate velocity - and for planning purposes to apply velocity to get rough date ranges.

The most commonly used relative-sizing technique - and the one scrum trainers have traditionally taught - is Planning Poker. There are plenty of Planning Poker descriptions out there - I won’t repeat them. In part, because…

A half dozen years ago I learned another technique from Chris Sims that he learned from its originator Steve Bockman that we all think is better. Perhaps I'm just not skilled enough with planning poker, but in my experience, story sizing is both richer and faster. It’s a technique that has, for me as well as many others, yielded better results in less time with more positive, bang-for-the-buck side benefits.

Generally just called Team Estimation or Story Sizing (or occasionally Rapid Release Planning or The Team Estimation Game), the technique is a two-step process: first the entire team orders the stories by size on a table or a wall; then, after labeling the smallest card a '1', the facilitator moves his hand along the card line until there's a card that's clearly no longer a 1 but twice that, so labels it a '2'; and so on.

I modified the method a bit and I think I improved it, but my changes are nuance to a technique this good already.

I've worked with teams unfamiliar with agile and not comfortable with assigning "story points" or any other arbitrary value system who were nevertheless quite willing to compare each card with the next to sort them and then size based on '1' being the duration of the smallest card. (Midway through, I heard one say, "This seems like five of the first task.")

There's something very tactile about turning off all the computers and getting everyone comparing 3x5 cards on a big conference table, comparing this card to the others, asking whether longer or shorter. Things get very collaborative very fast - not only efficient sizing, but it generates team-talk as a highly productive side effect.

To my contention that Steve’s method is faster, I took a team unfamiliar with agile through planning their 120-story next-version release in less than two hours. That was both creating a snake of cards from smallest to largest on a conference table, then putting down markers for where 1, 2, 3, 5, 8 and so on started.

To my contention that Steve’s method is richer, there was discussion around every card as it came up ("Why is this card bigger than that one?"), not just where disparities might have arisen in poker "hands".  The team had previously been focused on v.1 and was not familiar with the v.2 roadmap, but by the end of two hours everyone had a sense of the entire project. I don't want to say they "knew" v.2, but I will say I've never seen anything like it for bringing an entire team up to speed fast.

One of the things that struck me, then, was how the shy-est developer in the room was soon up and negotiating difficulty, ease and complexity based on the code she knew. The teamwork and learning, in just two hours, was phenomenal.

I'm told by Scrum conference attendees that Team Estimation is getting more and more shrift these days and may be on the verge of overtaking Planning Poker as the preferred method, at least in this region.

Here's a blog post by another of my colleagues, Chris Sims - how he described it.

Chris has since written a core scrum book and posted this description of the technique as he now practices it - quite a bit more structured than the original.

Better yet, Steve Bockman has written a novel, Predictability, to evangelize good estimating and burndown practices. A short and highly enjoyable read, Predictability resembles The Phoenix Project for software development. Via the narrative, it easily answers the question, "Why do we estimate in points?" It captures the crazy workarounds and multipliers we used pre-agile as a work-around to attempt sane estimates. And its chapter 12 is a compleat guide to Team Estimation.

Your biggest challenge, when you finish reading Predictability, will be realizing that its simplicity actually applies to your project. Trust me, it does. Go back and re-read chapter 12 with your team and apply it.

Saturday, April 26, 2014

Estimation Has a Cost
by Ron Lichty

Mike Cohn asserted, in a blog post a couple years ago, that estimation is not by definition waste. When, to his astonishment, he ended up in a Twitter-storm, he followed up with a second post, arguing, "Estimating is a way of buying knowledge. If having the additional knowledge will lead to different decisions, then acquiring that knowledge might be a good thing to do."

Mike added a proviso: that the cost of acquiring that knowledge doesn't exceed the benefit.

That’s the problem. Valid estimates can be expensive. Estimation accuracy often doesn’t get much better as more time and effort is expended - I've even seen accuracy get worse with more effort. For most of us, accuracy doesn’t improve significantly until our teams are experiencing estimating as truly tedious.

Our agile training has taught us techniques and insights that help us with the rough estimates. We do relative sizing, we estimate in points, we approximate in ranges, we divine other techniques to estimate more quickly, and we acknowledge that velocity not only varies dramatically from team to team but relies on team stability (and in fact must be re-derived and earlier estimates re-done if team make-up changes). And even with all that, we recognize the likelihood that our estimates will go wrong, so we negotiate at the beginning which side of the software triangle we're all willing to let slip (from scope/resources/schedule, the answer usually but not always is scope).

But the cost and pain of estimating still make us loathe to estimate long-term. We try to convince our customers to value the working code they’re getting every sprint and let go of the need for estimates.

But what we too seldom do is put a cost to estimating with accuracy. We too seldom give our customers a price to weigh against estimating's benefit. As a result, we continue to get requests for highly accurate up-front estimates accompanied by expectations that they’ll be delivered at low- or zero-cost.

Mike in his newsletter this month writes about how he puts the cost to bosses asking for dates: "I have no problem with a boss who asks a team to provide an estimate for how long a huge set of product backlog items will take to deliver," he begins, "So long as the boss understands that information has a cost…. To add precision to an estimate requires time….

"If providing the estimate will cause all other work to stop for a week, the boss needs to understand that. If the boss has a true need for the estimate, he or she may be willing to let all other work stop. If the boss just wants an estimate to feel good or to hold the team accountable, the boss will probably settle for an easier-to-provide but less precise estimate. Information has a cost. If the person asking for that information is willing to pay that cost, you need to be willing to provide the information."

Mike concludes, "I don't always estimate. But it's a very helpful skill to become proficient at and is useful on many, perhaps most, projects."

We have to stop being reluctant to cite a cost for estimating, and instead educate our customers that estimating has a cost and ask whether the outcome justifies it. If it does, then we all need to negotiate what we’re taking off the table to make the time that accurate estimates require.

(Mike Cohn is the author of User Stories Applied for Agile Software Development, Agile Estimating and Planning, and Succeeding with Agile. Mike is a co-founder of the Agile Alliance, and a co-founder and board member of the Scrum Alliance.)

Tuesday, November 26, 2013

Why Is Programming Fun?
by Ron Lichty

Fred Brooks answered this question so well and so poetically, in his 1974 classic, The Mythical Man-Month, that when I read it during my first year programming, I framed it. I'm sure I'm not the only one.

The whole Why is programming fun? section can be found online in many places - and more importantly in his book, The Mythical Man-Month: Essays on Software Engineering. It's short, but to quote a few favorite lines,

 "Why is programming fun?"
     "First is the sheer joy of making things. As the child delights in his mud pie, so the adult enjoys building things, especially things of his own design. I think this delight must be an image of God's delight in making things, a delight shown in the distinctiveness of each leaf and each snowflake.
     "Second is the pleasure of making things that are useful to other people....
     "Third is the fascination of fashioning complex puzzle-like objects of interlocking moving parts and watching them work in subtle cycles....
     "Fourth is the joy of always learning....
     "Finally, there is the delight of working in such a tractable medium. The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures....
     "The magic of myth and legend has come true in our time....
     "Programming then is fun because it gratifies creative longings built deep within us..."

Sunday, October 13, 2013

Good Product Management Matters!
by Ron Lichty

As a consulting CTO, I'm on the lookout for the knots in clients' software development; untangling them has been the key to transforming chaos to clarity and making product delivery hum. As an advocate for best practices, amplified by publication of our book Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams, I'm delighted to find (and become part of authoring) studies that document what makes product teams perform.

It came as no surprise when, among practices endemic to product team success documented by The 2013 Study of Product Team Performance, was:  how well the product manager role is defined.

Over and over, I've experienced the value that good product managers bring to development teams. They talk to customers. They talk to prospects. They get insight from sales and business development. They analyze the competition. They synthesize all that complexity (and more!), aligning and integrating it with corporate strategy, to formulate clear roadmaps that communicate both product strategy and tactics to everyone on their teams. And they collaborate with technical leaders to order the work that needs doing based on highest customer value incorporating technical factors like reducing risk, coordinating dependencies, and paying down technical debt.

I don't know how any organization can find success with its technical efforts without someone who can do all the above well.

Neither do the folks on product teams.

Last year's Study of Product Team Performance revealed five factors that, when weak or missing, almost guarantees low product team performance, but when all are delivered give teams a two-out-of-three likelihood of high performance.

This year's study revealed this sixth success factor: product managers working in well-defined product management roles.

Graph showing how teams perceive the role of product manager in their organizations
The good news is that 60 percent of respondents believe the role of the product manager is well defined in their organizations. The bad news is that a quarter told us that product manager roles in their organizations were not well defined. Worse, a sixth told us they have no product managers at all.

As we noted in the study, "Mapping out exactly what the product manager is responsible for, when they are responsible for delivering it, and who they need to interact with across the product lifecycle is required for any product initiative to operate at a level of high performance. The regression analysis clearly shows that the more well-defined the product manager’s role is within the organization and the product team, the more likely the team is to be high performing."

Sunday, September 29, 2013

Onboarding: (mostly overlooked) Critical Success Factor
by Ron Lichty

It wasn't a surprise to me to find, for two years in a row, that our Study of Product Team Performance identified that onboarding new team members highly correlates to high product team performance.

Last year's study revealed five factors that, when all are missing, is almost a guarantee of low product team performance, but when all are delivered give teams a two-out-of-three likelihood of high level team performance. Onboarding members into product teams was one of those five factors.

In this year's study, we looked at how teams are doing at onboarding. Here's what this year's study found:
    ▪    75% of responding organizations underestimate the importance of establishing streamlined processes that enable product team members to quickly become contributing members to the product team's success!
    ▪    Only 4% of organizations onboard product team members well enough to claim best practice

Most organizations apply a sink-or-swim approach to bringing new members onto their teams.

Based on our own experience, Mickey and I considered onboarding so important that, as we developed the outline for our book, Managing the Unmanageable, we realized we needed not only to devote a chapter to recruiting and hiring, but one to onboarding, or as we put it in the chapter title, "Getting New Programmers Started Off Right."

Good team onboarding practices were a lesson I learned early in my programming management career. I inherited Apple's Finder team just after they'd done a ground-up re-write - and just after most members of the team had departed for some equally ambitious project elsewhere rather than sticking around to enhance and evolve the result. I had a tiny nucleus of a team that I needed to rebuild. And with my first hire, I discovered I had a problem.

It's common to give new programmers a first task to set up their programming environments and sandboxes, install their tools, pull source code, compile, and check their result against the team's official last build to confirm everything. But when I gave that assignment to my first new hire, our systems turned out to be so convoluted that it took him a week to get everything in place and get the entire Finder built and validated. So the second task I gave him was to measurably shorten that for my next hire. I gave that same second task to every subsequent hire. Three years (and a number of hires) later, downloading the source and tools and creating and validating the build could be accomplished by a green new hire in less than 30 minutes.

That's just one tiny part of making onboarding effective and useful. We cover lots, in our chapter on the topic. Most readers tell us that it's one of the most valuable chapters in the book - in part, because it's so seldom addressed - or even considered.

While H.R. departments in many companies establish processes that successfully onboard new employees to the organization, that's simply not enough. It's important to get them through the benefits part, and to make sure they get a corporate meet-and-greet. But if they aren't brought onto their team effectively, their team will suffer for a long while after.

Managers and teams need to establish a broad set of processes that successfully onboard not only new hires but new contractors and transferring employees onto their teams. As the 2012 Study of Product Team Performance showed, it's one of five factors that make the difference between team failure and team success.

As our 2013 Study of Product Team Performance showed, teams have a lot of room to improve.

Monday, September 23, 2013

Agile: Not a Magic Pill
by Ron Lichty

No one on software teams believes in waterfall any longer. That's what The 2013 Study of Product Team Performance revealed. As the correlation chart shows at a glance, agile, blended and lean teams each correlate their own method with increased profitability, but waterfall teams believe that any method other than waterfall would make their product more profitable.

Table correlating methods teams use with those they associate with better product profitability

That doesn't mean agile, blended and lean are magic pills. Agile is not easy to do well. But waterfall is almost impossible to do well.

Mike Cohn notes, "Becoming agile is hard. It is harder than most other organizational change efforts I've witnessed or been part of [for reasons] including the need to change from the top-down and bottom-up simultaneously, the impossibility of knowing exactly what the end state will look like, the dramatic and pervasive changes caused by Scrum, the difficulty adding more change on top of all that is already occurring, and the need to avoid turning Scrum into a list of best practices."

To that last point, despite the efforts of trainers and coaches to inculcate agile's values and principles, human brains latch onto the concrete: practices, rituals and customs.

When software development gets hard, as it almost always does, instead of clinging to practices and rituals, we should be realigning our practices with the spirit of agile. Just as going to class doesn't make us educated, performing practices and rituals won't make our teams agile. Agile means embracing true change, not just tweaking the process at the outside corners, where it's most comfortable.

But practices do have purpose. They support us in living agile's values. That means we can't let centuries of top-down management and waterfall thinking erode our organizations' practices beyond recognition, as we've all seen happen.

I can't count all the times I've heard "we do agile" - only to discover that "we do standups" translates to "we do them sitting down, they take an hour, and… oh, what was that about there being three questions?"

In a long thread on agile and waterfall I was following earlier this year, a contributor named Greyfox observed, "I often see inept upper managers latch onto agile as the latest magic bullet which will solve all their problems with no other changes on their part. Except they keep all the micromanagement bits, discard all the engineer empowerment bits and hand their scrum team a year's worth of priority 1 stories to implement in the next sprint."

It's agile's values, supported by well-formed practices, that ultimately nurture frictionless, high performance delivery.

And as team members know, it beats waterfall.

Sunday, July 14, 2013

The most convincing reason to change from waterfall to agile
by Ron Lichty

Given no one on product teams believes in waterfall - even those using waterfall (see my last post on The 2013 Study of Product Team Performance) - what to do about managers who insist upon seeing a waterfall project plan so they can "guarantee that a project will come in on time"?

There is, of course, the response I heard attributed to XP creator Kent Beck, "How's that working for you?" I've used it myself, since first hearing it. It certainly applies. And it's a question that does make managers think - at least some of them.

But I suspect there's a swath of senior managers who, despite all evidence to the contrary, believe if they just hired better project managers or got their developers to arrive at the office when their salespeople do, they would meet their budget and schedule targets with room to spare. Another swath of senior managers seem so desperate for an anchor to hold onto that they grasp for Industrial Revolution management style hoping against hope that this time it will work for them.

But as Mickey and I noted in our book, "…the Waterfall model - in which each phase must be completed before the next one begins - was long ago shown to be an inadequate match for the reality of how software needs are understood and specified. In both of our careers, of the hundreds of projects we have worked on, each of us has seen just one project that was fully and completely described before coding began. Both cases were over 20 years ago. It virtually never happens."

It was collecting data from large numbers of developers and other development managers that truly knocked my socks off - converted me from a fan to a believer.

My first response to waterfall adherence these days is recounting my data gathering story - in part because I find stories can be easier for disbelievers to hear.

The story I tell - and the data I continue to collect - comes from speaking frequently to groups of developers and development managers.

I often ask them, "How many of you have been given a 400-page requirements spec?"

Virtually every hand in the room will go up.

I then ask a second question, "What percentage of those 400 pages of requirements did you actually deliver?"

Short of the occasional ringer (I've had a couple NASA engineers say 95%!), the responses seldom exceed 45 percent - are most typically 15 to 25 percent - and range as low as 10 percent of spec requirements delivered.

If that's not enough, I then ask, "Who decided which 15-25% was delivered?"

One of the things I like about Agile is that the top of the backlog is ordered every sprint by product owners collaborating with development leaders to ensure that the team is always working on the highest value stuff.

The answer in a waterfall scenario, on the other hand, is almost never value. The answers I get to my question include, "the easiest stuff to code," "what was most interesting to us" (what was most shiny!), "what stood out in our minds from reading the requirements," or "what was the most fun." Prioritization in 400-page waterfall requirements is almost universally done by developers, not product owners.

(Think back to your own 400-page requirements document: did it tell you what 15 to 25 percent was most important?)

But what truly clinched it for me - what converted me from enthusiastic about agile to downright disenchanted with waterfall - is waste. Waste of resources, waste of development time, waste of effort. When projects deliver only 15 to 25 percent of requirements in the spec, it means that:
    ▪    75 to 85 percent of the requirements-writing effort by product managers and business analysts was waste (and delayed getting development started by 3-5 times for having to wait on writing up all the undelivered requirements).
    ▪    75 to 85 percent of the work technical architects and designers did to craft technical specifications was waste.
    ▪    75 to 85 percent of QA's time writing test plans for the whole spec was waste.
    ▪    75 to 85 percent of visual design of wireframes for the entire application was waste.
    ▪    Project managers created project plans to deliver 100 percent of the requirements by the deadline - 75 to 85 percent waste (not to mention irrelevant).
    ▪    Developers, who need to read and understand the context of everything that goes into an application, struggled to get the big picture of all 400 pages - 75 to 85 percent of which they would not deliver - and seriously compromised their ability to do the 15-25 percent well.
    ▪    Not to mention customers, who thought through everything they might want but who really wanted the most important parts often before product managers and BAs even started writing requirements.

Waste is what convinced me. And waste is what, in my experience, brings around the disbelievers.

There's a reason that, when we asked about methodology, in this year's Study of Product Team Performance, those groups using waterfall responded that any of the methods other than waterfall would make their product more profitable. They've seen waste up close and personal.

Sunday, July 07, 2013

No One Believes in Waterfall
by Ron Lichty

The most startling result coming out of this year's just-released Study of Product Team Performance came in response to two questions we asked:
    ▪    what methodology does your team use?
    ▪    what method do you associate with increased profits?

The cross-correlation of responses to those two questions revealed the startling result:
    ▪    each self-identified methodology group - with the exception of those using waterfall - associated increased profits with the method their group currently uses

In fact, the data showed that those groups using waterfall believe that any of the methods other than waterfall would make their product more profitable.

If it were just that those of us who teach or lead agile and lean transformations were associating the methods we advocate with increased profits, that would not be startling.

In fact, the data is quite gratifying - to this agile coach, at least - that members of teams using agile and lean and blends of waterfall with agile and lean, across the board, associate their methods with increased profits. As challenging as agile transformations can be, agile users are not opting to return to waterfall.

What's startling is that among teams still using waterfall, they just don't believe in it. Doubly startling given users of every other methodology rate their own method most highly.

You can take a look at the data yourself: the census (who is using what) is summarized in Table 2 on p.16 of this year's study. If you compare with last year's 2012 Study of Product Team Performance, you'll see a strong surge in agile adoption, year over year. (While our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams, is not specific to agile or blended methodologies, we didn't hesitate to call out the value of agile practices.)

The data that surprised me is in Table 3 on p. 17: the correlation between methodology use and association with profit.

No one (on product teams) believes in waterfall!

The question, of course, is why waterfall is still in use. I would surmise that the answer in some organizations has to do with desire by management to command-and-control (regardless of its lack of effectiveness) - I've seen that firsthand - perhaps most of us have - combined with how hard it is to overcome inertia - the effort required to change a long-lived waterfall pattern of behavior that has been passed along for millennia from directing manual laborers. Whether from managers used to giving orders, or developers used to taking them, the habit is hard to change.

But clearly, no one on product teams thinks waterfall is effective.

Sunday, June 30, 2013

The 2013 Study of Product Team Performance
by Ron Lichty

The 2013 Study of Product Team Performance is out! The results are not quite as across-the-board extraordinary as last year's study - that would have been hard to match - but it offers individual results that are every bit as eye-opening - and overall it's quite revealing.

What startled me in last year's study was that it named just five things that:
    ▪    if you don't do them, you have only a 2% chance of high level team performance
    ▪    if you do all of them, you have a 67% likelihood of high level team performance

I thought that was pretty remarkable. I blogged about the 2012 Study's five factors for success.

I also blogged about how the change 2012 survey respondents most wanted to make in their teams rang true in my experience - and how one team had achieved that challenging leap forward.

But I was struck by the fact that the 2012 Study's then-three authors had product and project and requirements backgrounds - but no developer viewpoint. When I pointed that out, they recruited me to be a fourth coauthor of this year's study.

We targeted this year's study to dig deeper into the factors that affect team performance and product profitability. We partnered with 13 professional associations and companies to recruit and survey almost 1500 members of product teams over 90 days at the beginning of this year. We provided the resulting data to statisticians.

The regression analysis the statisticians delivered deepened or expanded our understanding of four of the five factors of high-performing teams that emerged from last year's study (the fifth had been so strong it needed no clarification):
    ▪    An Aligned Strategy: the significance of the product team aligning with the company’s overall goals and objectives
    ▪    The Importance of Product Launch: successfully staffing and resourcing product launch activities
    ▪    Onboarding Processes: the surprising impact that emphasizing effective product team member onboarding has on the overall performance of the product team
    ▪    Executive Team Support: not only is unwavering executive team support linked with product team success, but so is close connection with the CEO or general manager.

A new factor emerged from the analysis:
    ▪    Product Manager Role Definition: the favorable impact that a well-defined product management role plays in product team success

The study also revealed four landmines: four statistically relevant factors that can undermine a team's ability to achieve high performance.

Remarkably, it also showed that members of teams using agile and lean and blends of waterfall with agile and lean, across the board, associate their methods with increased profits - but members of teams using waterfall believe that approach is costing them profits!

I'll blog about some of these factors (and at least one other intriguing result) in coming weeks, but you may want to read the 2013 Study of Product Team Performance white paper now (and the 2012 Study, as well, if you haven't already!).

Monday, June 03, 2013

Colleges: Teach the "How"!
by Ron Lichty

We're at an inflection point.

Agile practices transitioned over the last couple years from "emerging" to "predominant". Companies are sampling if not wholesale adopting agile, from scrum and extreme programming to lean and kanban. Our study of product team productivity due out later this month documents an increase to 76% of teams incorporating at least some agile methods, up from 71% last year. Even more significant, 84 percent of teams still using waterfall think they could increase the profitability of their product by adopting agile practices.

But agile practices are not easy. Agile has a reputation as "loose" but, in my experience, the various forms of agile represent the most rigorous methodologies, frameworks and practices I've come across in my 30 years in software development.

Companies adopting agile soon discover what those of us using agile for a decade already know: there are all too few developers (and managers and scrum masters and product owners) with any significant amount of training and experience in agile methods and practices. Too few developers have practiced pair programming. Too few developers have tried test driven development (TDD). Too few shops leverage practices that support continuous deployment. Too few teams practice team "ownership." Too few developers and testers and scrum masters and product owners have spent time as part of successful scrum teams. And too few managers have insight into the critical contribution they could make to the success of their companies' transitions.

There's good training out there - I've taken some of it and my clients tell me I've given some of it. But companies sign up for too little training. Dean Leffingwell reports that factories don't even begin a transition to lean without having first sent their managers and directors off to lean bootcamp for a minimum of two weeks and often six. And companies sign up for too little coaching. Agile's wins don't come from learning drills and nomenclature in meaningfully applying them - which requires understanding the meaning behind the drills - application of agile's principles to the unique culture and challenges each team and each company faces.

I think the place to set meaning and understanding in place is much, much earlier: when students first learn to code.

But the education community has done little to incorporate agile into curricula. I was shocked when I interviewed Stanford computer science students for internships two years ago to find, to a one, that they had heard not a mention of TDD or continuous deployment or scrum and had only a vague idea of pair programming.

Earlier this year, I pitched community college educators, at the ICT Educator Conference in San Francisco, on the value of teaching not just programming languages, algorithms, and logic (the "what") but also agile methods and practices (the "how" now coveted by hiring managers). I titled my talk "Win / Win / Win with Agile" because this inflection point and the dearth of developers with agile experience provide unprecedented upside for educators, students and hiring managers alike. A video interview I did with the conference organizer just posted this week on the wins for colleges and educators and their students.

Teaching test driven development is at the top of my list. We called out TDD in our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams. Valuable for the students? I can't express enough how highly I value resumes from programmers who can say they practiced TDD with every line of code they wrote.

flow of test driven development
TDD turns coding inside out: programmers are responsible for unit testing their code; before they write a single line of code, they write the tests that will prove the line works; they then run that test demonstrating that the test fails without the code; they then write the code and run the test again until the test succeeds. That cycle should take an experienced TDD coder between 30 seconds and two minutes. Rob Myers, who trains teams in TDD, describes a two-year effort writing a life-critical application yielding 9,000 unit tests that together made up a comprehensive automated regression test suite; the entire suite could be run in 10 minutes; and the team could make a change to a life-critical application and deploy the change, with confidence, within an hour.

I've found converting an entire team that has never used TDD is challenging. In my experience, no more than 1 in 5 programmers "gets it" - sees writing tests before writing code as an obvious way to improve quality, to ensure requirements are understood, and to enable later refactoring with impunity - with confidence in the outcome. The remaining 4 of 5 programmers find it not "natural" to write tests before writing code.

Unless they were taught from the beginning that writing tests first is the natural order of things.

As a hiring manager, I expect students who test first - who have incorporated TDD into their way of working so thoroughly that they do it naturally - will deliver a level of quality that matches or exceeds the best of what I can get from experienced programmers who have no experience with TDD. Students who can put TDD on their resumes - and speak to the practice with a level of confidence deeper than most experienced programmers - will win jobs.

But TDD is only the beginning. Bringing programmers (and tech leads and architects) into pair programming can be even more of a challenge. While pairing has been shown to result in higher quality code in comparable time than two programmers working separately, for many programmers it's as counter-intuitive as TDD - maybe more so.

Programmers need practice working in pairs and in teams to feel comfortable doing it. The good news - for both online learners and distributed teams - is that pair programming can be practiced by remote students and teammates, not those who meet to pair. It's slightly easier, in fact. In physical space, the two programmers share a single monitor and pass a single keyboard and a single mouse back and forth. In virtual space, they're connected via audio link so their conversation is the same, and they link their monitors so they're always looking at the same thing but each has their own keyboard and monitor - it's control of the cursor that, using conventions, gets passed back and forth.

Students who leave college having had to pair on assignments and having had to write tests first (TDD) would be dramatically more hirable. Industry would not only snatch them up but be enormously grateful to the colleges that taught students not only the what but the how. Such curricula would change the tenor of programming for the better - with regard to both quality and productivity - across their region. Institutions where curricula required students to learn Agile practices would see appreciation from students and hiring managers alike.

Colleges can deliver what we hiring managers are looking for, just by expecting students to:
    •    work in pairs (pair programming)
    •    write tests first (test driven development)
    •    partner in requirements elicitation (scrum)
    •    retrospect regularly (continuous learning)
    •    demonstrate facility with pomodoros (agile time management)
All the stuff that's so hard to teach and makes it so hard to transform organizations.

It's time for educators to step up, teach not just the "what" but the "how", and help transform this hard thing we do called software.

Sunday, April 14, 2013

Remote Pair Programming
by Ron Lichty

I've been enamored of remote pair programming since briefly leading development at Socialtext, where no three developers were in the same locale. Socialtext was committed to agile approaches including pair programming (but not including colocation, I might note!). Being distributed, remote pairing was the way forward.

Pair programming is controversial in being possibly the most difficult agile engineering practice to inculcate into development teams - we've heard of programmers quitting companies rather than signing on. But there's also plenty of evidence that it's well worth doing: Mike Cohn, in Succeeding with Agile, says, "if you're in a hurry this is the time you need pair programming the most… Although most studies show a slight increase in the total number of person-hours used when pairing, this is offset by a decrease in the total duration of the effort. That is, while pairing takes more person-hours, fewer hours pass on the clock…. Pair programming has also been shown to improve quality…. Additionally, pair programming facilitates knowledge transfer and is an ideal way to bring new developers up to speed on the application. It is also an effective practice for working in uncharted territory or solving difficult problems in known parts of the system."

photo: pair programming
In our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams, we note a less obvious advantage of the technique. We share David Vydra's interviewing rule of thumb, which states, "Pair programming for half an hour during an interview will save everyone's time."

Of remote pair programming, we also share the pleasure I saw programmers at Socialtext experience transitioning from physical to virtual pairing. (pp. 133-4)

While proximal pairs typically share a single screen and often a single keyboard and single monitor, virtual pairs each sit square to their own monitor, keyboard and mouse, sharing a virtual single screen. For screen sharing, developers at Socialtext typically used a version of VNC (e.g., Chicken or RealVNC).

As a co-chair of the Emerging Technology SIG of SVForum, I've recently been looking at collaboration tools of various kinds, led there in part by the amazing advances in sharing Web research that Vivek Agarwal at Surfmark is doing. That led me to consider that screen sharing in general might have taken a leap forward.

I didn't find anything startling, when I went looking for an update on remote pairing tools, but David Vydra, who seems to know and have worked with all the cool kids, pointed me to a wonderful resource, Joe Moore's blog, which is focused on remote pairing tools. Not surprisingly, Joe is part of Rob Mee's Pivotal Labs, known for its XP engineer practices.

The blog covers a lot of great stuff, including updates on tmux, Google Hangouts and MadEye, SublimeText 2, solving Skype challenges, and more. In his two-years-ago assessment of remote pairing technology, Moore concluded, "using Apple’s built in Screen (aka SSA) combined with Skype has proved to be the best remote paring technology combination."

For those who are distributed - or just working from home - remote pairing is alive and well.

Saturday, April 06, 2013

Keep Your Culture Positive!
by Ron Lichty

Which is more effective to improve team performance: positive feedback or constructive criticism? a positive culture or a negative one?

The answer: both.

The real question: in what proportion?

The answer to the proportion question, from new research by Michigan doctoral student Emily Heaphy and team productivity consultant Marcial Losada:

The ratio of positive-to-negative statements is directly correlated to "strikingly different results for each performance category" of teams:
* The highest-performing teams expressed 6 positive comments for every negative one
* Medium-performing teams expressed 2 positive comments for each negative one
* Low-performing teams swung the other way, expressing almost 3 negative comments for every positive one

"…in order to predict team performance, one only has to know the ratio of positive to negative interactions…", Heaphy and Losada concluded.

Jack Zenger and Joseph Folkman gave it a giving-feedback spin in their Harvard Business Review blog post, but nonetheless correctly summarized:

"The factor that made the greatest difference between the most and least successful teams: the ratio of positive to negative comments that participants made to one another."

Heaphy and Losada were studying leadership teams - 60 of them - and not development teams, but given they're measuring the nature of human satisfaction and response, development teams can't be far off.

They were measuring approving versus disapproving statements (not just feedback) - expressions of support, encouragement or appreciation (e.g., “that’s a good idea”), versus sarcasm, cynicism, or disapproval (e.g., “that’s about the dumbest thing I ever heard”).

Their finding may apply to feedback but the overall truth is about culture.

photo: working together positively and productively
"Qualitative observations of the teams showed that high performance teams were characterized by an atmosphere of buoyancy that lasted during the whole meeting. By showing appreciation and encouragement to other members of the team, they created emotional spaces that were expansive and opened possibilities for action and creativity as shown in their strategic mission statements. In stark contrast, low performance teams operated in very restrictive emotional spaces created by lack of mutual support and enthusiasm, often in an atmosphere charged with distrust and cynicism."

Heaphy and Losada noted validation of prior research by other teams that showed positivity opens opportunities for action while negativity closes them down. (Interestingly, they also noted prior research into married couplehood that correlated remarkably similar ratios of positive-to-negative comments to stable marriages (a 5-1 ratio) and negative ratios to divorce. "Gottman’s research on married couples has shown that the best predictor of stable marriages is the ratio of positive to negative interactions…. Where his “performance” variable was the sustainability and quality of a marital relationship, we found that this same ratio of positive to negative interactions is the critical differentiator between high-, medium-, and low-performing teams.")

Negative comments must clearly be part of the mix. Pollyannaism - positivity alone - is no answer. It's just that most of us don't want to work in a culture that swings negative. What we all need to note is that the swing negative starts from a 6-to-1 positive-to-negative ratio!

Throughout our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams, we devote pages and pages to the value of praise, recognition and having fun with your teams - all key components to building a culture of positivity. "Say 'thank you' at least ten times every week," we counsel. "Never pass up an opportunity."

We just didn't have the researchers to measure it - proof positive.

Heaphy's and Losada's conclusion: "We need to have organizations with teams where the abundance of positivity, grounded in constructive negative feedback, can generate the state of realistic enthusiasm that can propel organizations to reach and uphold the heights of excellence."

Sunday, March 31, 2013

Drawing in 3D
by Ron Lichty

The way history worked, pens and brushes were invented millennia before the printing press - artistry before printing.

Which got us into the modern age, where 3D has been a different story.

Now, not only can 2D be printed - books and copies and newspapers - but 3D things can be created and manufactured based on plot points and axis control and software.

I don't want to suggest that they're not artistry, but they're repeatable and (depending on the specs) perfect. So where's the 3D paintbrush, the 3D pen, that one might have expected to come before? History is reversed in our modern age. Manufacturing before artistry.

It was fixing 3D printing that got designers Peter Dilworth and Max Bogue to realize what our age was missing. So they invented it.

Imagine a glue gun that extrudes a "line" of plastic in the air. Their 3D pen can draw in all dimensions in multiple colors in free-space.

You have to see it:

Friday, March 08, 2013

Motivating (and not de-motivating) Programmers
by Ron Lichty

One of the great "aha"s for me as a manager was realizing that motivating people - and not de-motivating them - are two different things.

It was years ago - I think I was at Apple. One of my colleagues shared a Harvard Business Review article from the 1980s that recounted Frederick Herzberg's work in the 1950s. Titled "One More Time: How Do You Motivate Employees?", when it was republished in 1987 it had already sold more than 1.2 million reprints since its first publication 20 years earlier - 300,000 more than any other of the thousands of articles HBR had published.

Herzberg posited that motivation isn't so simple:
    ▪    there are a lot fewer factors that motivate people than we think ⁃ a lot fewer
    ▪    but there are an equal number of factors that, while they don't motivate, they'll de-motivate when they're lacking - when they go negative

By example, company policy / administration is just not a motivator - no one joins a company because of its administration. But it can sure be a major de-motivator. Almost all of us have seen companies with policies and administration that could deflate the morale of anyone.

While we summarize Herzberg's findings in our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams, and the article itself is well worth a read - his findings of what motivates and what de-motivates are general to the entire workforce population.

That led my co-author and me to adjust the categories and the weightings slightly, based on our own decades of experience managing software teams, to target those factors we believe motivate and de-motivate programmers.

Here's the chart we came up with: what we believe Herzberg would have found if he had applied his research just to programmers:

graph: Herzberg's motivators and hygiene factors as we believe they apply to programmers
Chart showing the factors that motivate and demotivate programmers

The chart is ordered by the motivators - the blue vertical lines; these are the causes of satisfaction and thus what motivate. The red vertical lines, on the other hand, indicate the extent to which the same categories are de-motivators - causes of dissatisfaction. The height of each line, blue or red, is its statistical importance.

Notice that motivators and de-motivators are not aligned!

Stepping through the Motivators (the blue) is easy, given the chart is ordered by them. Here are the motivators in the order that we believe drive programmers to do great work:
    ▪    Making a Difference in the World
    ▪    Learning and Growing
    ▪    Toys and Technology
    ▪    Recognition and Praise
    ▪    Having fun
    ▪    Upside
    ▪    Interpersonal Relationships
    ▪    Compensation

We devised the first of these, "Making a Difference in the World," to both take in Herzberg’s category of Achievement and take it a step further to capture the reason many if not most of us started programming – the opportunity to make a difference. I remember showing off my first programs to my wife and feeling the pride of achievement. But even with those first small efforts I could see the power of the craft to positively affect the world – what a motivator that was!

It's worth noting that the 6th category, "Upside" - that would be stock, stock options, bonuses, that sort of thing - is the first mention of money, and it's 6th down the order! And "Compensation" doesn't come in until 8th! (That's about where Herzberg puts "Compensation" for the general population, too, by the way.) Once you're paying people enough - adding more comp is just not that significant a motivator.

Stepping through the De-motivators (the red) (lack of these are the reasons programmers quit) is a bit harder. Starting with the highest red line, here are the factors in the order we believe they de-motivate programmers:
    ▪    Lack of respect for supervisor
    ▪    Lack of having fun
    ▪    Lack of learning and growing
    ▪    Working conditions
    ▪    Company policies and administration
    ▪    Lack of ethical management
    ▪    Compensation

Notice the first of those is out-and-out us - poor performance by their manager! Programmers may not hire into a company out of respect for their new manager, but they'll surely consider leaving if they don't respect their manager. In fact, there's a rule of thumb in H.R. that "People don't leave their companies. They leave their managers."

"Having Fun" and "Learning and Growing" appear on both lists - motivators and de-motivators. Contributing to making your environment a fun place to work and ensuring that staff have opportunities to learn and grow are enormously important both to retention (keeping your people from leaving) and to keeping your people engaged and productive. And then come "Working Conditions" and "Company Policies and Administration". Fighting for your people and their needs - and shielding them from corporate politics and inanities - are critical contributions managers must make. Again, people don't hire in for the working conditions or the company policies, but they'll surely leave because of bad ones. And then there's "Ethical Management" - whether from a team's direct manager or all the way to the execs, lack of ethics will squeeze the life out of a programming team.

Much as for the motivators, it's only then, this time 7th on the list of de-motivators, that we come to "Compensation"! (Also near where Herzberg placed it for the general population.)

In fact, my coauthor and I issue warnings - in our discussions of motivating programming teams - about attempting to motivate with money, and the studies and experience that lead us to issue those warnings.

There's not room in a single blog post to discuss every factor, show you where our chart differs from Herzberg's observations of the general populace (and describe why), flip the chart to order it by the de-motivators, share with you the other two motivation theorists we find incisive and wrote about in Managing the Unmanageable, or delve more deeply into why, in general, we caution against motivating with money.

But I'm hoping it's enough to clearly show the tightrope that programming managers walk to motivate their teams and the individual programmers that comprise them - and not deflate their motivation.

And I'm hoping it will suggest how project managers, program managers, product managers and other team leaders, by being aware, can collaborate with programming managers in motivating (and not de-motivating) programming teams.

Wednesday, January 30, 2013

What makes product teams great?
by Ron Lichty

What makes product teams great?

What if you were able to name just five things that:
    ▪    if you don't do them, you have just a 2% chance of high level team performance
    ▪    if you do all of them, you have a 67% likelihood of high level team performance

Last year's Study of Product Team Performance found just that. Five factors to a 65%-more-likely high performance team:
    ▪    unwavering executive team support
    ▪    strong team alignment w strategy
    ▪    post-production development focus and accountability
    ▪    effective on-boarding of new team members
    ▪    assigning core team members based on skills needed

One of the reasons the study is so useful is that several of those require getting your senior execs on board and the rest require getting your peers on board. And one of the hard truths - about senior managers, at least - is that they tend to believe studies and consultants more than their own people. (And that's one reason why my co-author and I collected 300 Rules of Thumb for managing software people and teams that we share as a quickly thumb-able center section of our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams (Addison Wesley, Oct. 2012). And why studies like this one are so important.)

As for those five factors, you can download last year's study (including some of the comments behind the data).

(For my blog post on another finding of last year's study, see my previous post, "Better cross-functional collaboration, trust and communication")

Friday, January 11, 2013

Better cross-functional collaboration, trust and communication
by Ron Lichty

Coming soon: 2013 results:
what makes product teams great?
In 2012's Study of Product Team Performance, respondents were asked what they would change about the core product team.

"Better cross-functional collaboration, trust and communication" was their number-one response.

I was struck by how closely that aspiration aligns with a step that management took 15 years ago when I was at Schwab, a step I was convinced was responsible for my team becoming one of the highest performing I've had opportunity to manage.

The time was February 1997. The team was embarking on building Schwab's first web-based customer applications in Java. At the beginning of 1997, Java was mostly being used for eye candy: animated thermometers, dancing fonts, blinking messages, that sort of thing. Java applets didn't even print. Not that there were a lot of alternatives: HTML was flat and lifeless; Javascript wasn't yet built into a single browser; the only alternative was COM, also very early as a platform. To build business applications in Java at the beginning of 1997 was not just leading edge, it was bleeding edge.

To support the effort, senior management hired a team-building h.r. group - they go by names like "organizational development" - "OD" - but they're all about building collaboration, trust and communication - soft skills - emotional intelligence - that sort of thing.

As keyed up as management was to prove to the execs that we could do something new, something breathtaking, something never done before - they arranged for the entire product team to take entire afternoons offsite multiple times during our short four-month development cycle. Developers, testers, project managers, product managers, business analysts, architects, instructional designers, the whole bunch of us.

That project progressed from research and ideas to working product - an asset allocation toolkit - with functionality that had never before been built even in the client-server world - in a few short months. The team won awards; the product was lauded; Schwab was lionized.

At the time, I remember thinking that I never wanted to work on a product team again that did not undertake significant cross-functional trust and collaboration team-building. I recounted the experience in our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams. Sadly, I've never worked on one since that did.

The 2012 study reminded me of what gave that Schwab team greatness. The study examined the interactions of Product Managers, Project Managers, Program Managers, Business Analysts, Developers and others actively involved in product development projects.  "The goal of our research was to better understand the dynamics of product team performance and to uncover the practices that make these teams successful. What makes this survey unique is that it enjoys the support of various industry associations and market players — groups and individuals that don't generally work together."

Wednesday, December 26, 2012

Could we make managing programmers a little easier?
by Ron Lichty

Programming managers have typically had years of training in programming…

And no training in management. 

I've been taking a fascinating census of programming managers this fall that has shown me just how true that is. As I talk about our book Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams to groups that include software development managers, I start by asking:

"Stand up if you are now or have ever been a manager of programmers and programmer teams."

I continue:
"Stay standing if you have ever received management training
- let's say a day or more of training -
whether from your company or in college or on your own.

Almost to an audience, half of managers sit down! 

Half of programming managers have never had a single day of management training in their careers!

To the people still standing, I ask:
"Stay standing if you got any of that training before you first became a manager."

The result: 
Of all the programming managers I talk to, only 8-15 percent have received any management training before becoming a manager.

Almost all the training for managing programmers that exists.
There are only two others that are missing from this picture!
Equally telling: there are a ton of books on project management. In fact, there are scores of books on each different flavor of project management.

But the number of books on managing programmers won't use up the fingers on both hands. (It's one of the reasons that Mickey and I wrote Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams.)

Isn't it odd?:
* how long we expect you to have studied the art of programming before we hire you to be a programmer?  languages, libraries, frameworks, tools, techniques, ...?
* the growing expectation that not only have you studied to be a project manager, but you've become certified to do so?

Isn't it odd how little we expect you to have studied the art of managing before we tap you to become a manager?

Tuesday, December 18, 2012

Orchestrating Software Development
by Ron Lichty

For years, I've been pointing development and project managers to the full-length documentary, Tom Dowd & the Language of Music. (You can rent via Netflix, purchase via Amazon or Barnes & Noble.)

Every time I watch Tom Dowd, I'm struck by how closely producing music resembles leading software development.

Tom Dowd, an innovative music producer and recording engineer for over half a century, altered the course of contemporary music not only through his many technical achievements but also through his ability to coach some of the greatest musicians of our lifetimes, including Eric Clapton, the Allman Brothers, Lynyrd Skynyrd, Aretha Franklin, Ray Charles, Marshall Tucker, The Drifters and many, many more.

To accomplish with programming teams what Tom Dowd accomplished with rock and roll musicians is a goal I have long strived for.

Just as Dowd's goal was to stir musicians to together create music more thrilling than they thought possible, our goal as development leaders is to stir our programming teams to create software more thrilling and satisfying than any of us thought possible. 

As a bit more background, Tom Dowd was employed at 16 to do math for, unbeknownst to him, the WWII atom bomb project - the Manhattan Project - because they figured they'd get at least two years of engineering and math from his young mind before he risked being drafted. He was such a part of that project by the time he turned 18 that while he was sent off to basic training, he was immediately returned to NYC to continue calculating. But when the war ended, there was no segue to a nuclear physics career - Tom Dowd knew more than the academics he would have had to study from! In frustration, he turned his engineering ability to his other love, music. It was a time when groups went into the studio and laid down an entire album in an afternoon, each song a single monaural track. He was hired into Atlantic Records as a recording engineer. He continued to engineer, mix and produce music until his death in 2002.

His engineering prowess combined with his musical ear to give him a ticket to play. He was one of the first to lay down increasing numbers of separate tracks. (An actor portrays Tom Dowd in the Ray Charles movie: at the sound board, Dowd not only mixed but invented the overlays and the engineering that Ray Charles needed.)

As Dowd's coaching skills developed, he became responsible for some of the great collaborations in music. When Cream was having trouble bringing their individual parts together in "In the Sunshine of Your Love", it was Dowd who pointed them to the beat change that brought the piece to life and fame. And it was Tom Dowd who introduced Duane Allman and Eric Clapton, resulting in their classic Derek and the Dominos collaboration, Layla.

From the first days of programming, the crossover between programmers and musicians has long been noted. Mickey and I discuss it in our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams. (Mickey was a professional musician himself, once, and still makes command performances for family and friends; and I had years of training in piano, trumpet and guitar.) 

But the management crossover - the similarity between orchestrating programmers and orchestrating musicians - hasn't been as obvious, as written about or as noted.

Eric Clapton appears in the documentary to give voice to the contribution made by his producer. "The quality and the success of those recordings could mostly be laid at the doorstep of Tom Dowd." 

Butch Trucks, drummer of the Allman Brothers, says, "Tom had a way of making things work - had a way of pulling us together - being this father figure and this psychologist that would dig into the depths of us and find what we were capable of doing."

From one of the members of Lynyrd Skynyrd, "He could bring the best out of you without you ever knowing it and all of a sudden, wow, there it is."

Again, Eric Clapton: "I've always been riddled with self-doubt about my work, especially as a composer…. His role was making me feel comfortable and inspiring confidence in myself."

One of Dowd's colleagues said, "He's a coach. He's the absolute perfect coach."

For those of us eager to have that kind of impact in software, Tom Dowd and the Language of Music is a movie worth watching.

Monday, October 08, 2012

Managing the Unmanageable is in print!
by Ron Lichty

There have ever only been a handful of books on managing programmers and teams (while project managers have benefited from having a ton of books on project management of every kind). 

That's what led my coauthor and I to start writing. That and our wish to share the scores - now hundreds - of rules of thumb and nuggets of wisdom we'd been collecting from our peers. We'd been trading them for the decade we'd known each other. They seemed more broadly useful. 

Now all that's in a book. Rules. Nine chapters of our insights from a combined 70 years in software. And the tools we've used to manage. 

Addison Wesley sent us first copies off the press last week. Amazon is shipping!

Thursday, September 20, 2012

by Ron Lichty

I am repeatedly in conversations with someone who says, "Oh, PMs blah blah blah." 

And I have to figure out which kind of PM they're talking about. 

(It happened again yesterday afternoon. The person speaking was a product manager, which made it likely that the "PMs" to which he referred were other product managers, not project managers. Then this morning, I got an email with the subject line,  "The Evolving PM". Again I had to think. The sender was a project management association, making it likely that the "PMs" to which it referred were other project managers, not product managers.)

I'm not a PM. I'm not a PM of any kind. I'm not a Project Manager, I'm not a Product Manager, I'm not a Program Manager, I'm not a Publication Manager (engagement last year: Stanford subsidiary HighWire, which hosts web sites for 1400 of the world's most prestigious academic and scholarly journals from 150 publishers worldwide, and when I arrived, had 14 "PMs" for me to manage - who were account managers!).

I'm an interim VP Engineering. And a consulting CTO. And an Agile trainer and coach of Agile transformations. And an author of the Addison Wesley book (just released today!):
   Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams  

The common abbreviation terminology I've been pushing for, in my engagements:
PjM == Project Manager
PdM == Product Manager
PgM == Program Manager
They're all self-clarifying.

I give talks on Transforming Chaos to Clarity.

Project Managers and Product Managers and Program Managers are, to a one, charged with clarifying software development, not adding to the chaos.

"PM" adds to the chaos. 

I wish that were…

'nuff said.

Saturday, April 24, 2010

Software Engineering Radio
by Ron Lichty

Robert Blumen recorded the most recent one on Aspect Oriented Programming. He did an earlier one on Hadoop and has upcoming ones on Project Voldemort, NoSQL, workflow management, cloud architectures, high-performance web architecture and event-driven systems. (Robert was a senior systems architect at Morgan Stanley a decade ago when I was leading Schwab's Java initiative.)

Popular topic areas on Software Engineering Radio include languages, patterns, agile, process, SOA, and DSLs.

Looks like good stuff!

Friday, April 02, 2010

Book review: Succeeding with Agile
by Ron Lichty

The first half of Mike Cohn's book, "Succeeding with Agile: Software Development Using Scrum", isn't so much about how to be successful with agile as how to be successful replacing non-agile with agile, selling agile, and transitioning to agile. Mike Cohn is neither a wild-eyed dogmatist professing the one true way nor an ivory-tower philosopher; he's a pragmatist. He devotes entire chapters to helping scrum evangelists decide whether to start with a pilot or go all-in across the organization, whether to pilot publicly or in stealth, what factors might prevent success, how to swing things your way, how to approach a transition in steps, mechanisms he calls Improvement Communities that along with a more global Enterprise Transition Community can both spread scrum across your enterprise and debug adoption issues, ways to use scrum to manage the transition to scrum, picking the right first project and first team, and how soon to adopt agile technical practices like simple design, pair programming, TDD, continuous builds, refactoring, and automated testing.

He provides helpful anecdotes of two highly visible companies that, when waterfall stopped being effective, tried getting more rigorous about it, which turned out to just make things worse - then finally yielded and tried agile and got visible benefits almost immediately.

Yet he also notes from the first chapter that agile implementations are hard and don't always "take" - and explains why.

He also does us all the favor of reviewing and summarizing scads of studies and surveys that demonstrate why agile is worth the effort, with data on productivity gains, lower costs, improved employee engagement and job satisfaction, faster time to market, higher quality and, importantly, stakeholder satisfaction gains. (Have you seen the study of pair programming that shows it an effective way to counter Brooks' Law - the one about adding programmers to a late project making it later? Or the study of the negative impact of multitasking on productivity? Or the one showing productivity of teams larger than seven plummeting? Or the one that shows R&D teams do need new members to stay vibrant - but only at the rate of one very 3-4 years!)

About midway through the book, as you're reading insights into teamwork, you realize that this book is not just about transitioning to agile but about getting really good at agile - and that you're going to need to read it not just once but come back to it for new insights again and again. The fact is it's stuffed with tips, guides, data, approaches, failures, successes, and helpful hints of every kind. What's the role of a manager of a self-organizing team? One of them, agile gurus seem to agree, is exercising subtle control (as opposed to command-and-control) and Mike provides an entire framework for debugging team problems and influencing their success. This will be one of those chapters I know I'll revisit repeatedly.

It is good to have in print solutions to the many conundrums and traps that agile teams face:
* How to deal with work that seems indivisible into sprints
* How to make user experience, architecture and database design part of sprints, not separate waterfall efforts
* How to make testers part of the team and testing part of the team's work
* How to hold a hard line against mid-sprint change - and what to do when you can't ("abnormal termination", followed by immediate planning for a new sprint - never extend a sprint nor change its flavor!)
* Avoid the anti-pattern of doing design in one sprint, handing off to development in the next, handing off to QA in the third - designers, developers and testers should be working together in a sprint to solve, develop and deliver complete stories within a single sprint timebox
* How to deliver visible functionality every sprint even when you're mostly developing unobservable features (and why you want to)
* Figuring out how to get a slice of working application functionality every sprint (not just components, libraries or layers)
...and so many others.

Part of what makes this book important for me is repeatedly reading observations consistent with my own experience, but drawing on so many more experiences with agile that it results in emergent patterns, as well as in truths that are too often left unsaid:
* Agile is not successfully implemented by fiat, without support from the folks at the front lines, but similarly it's not successful as a groundswell without senior management support. Check. Seen both.
* Agile is not a set of rules to be followed to the "T" but has to be adapted uniquely to each organization where it's implemented. It requires an iterative "provoke and observe" approach. Check. Seen that one.
* Furthermore, attempting to turn software development into a set of rules is counterproductive for the very reason that you need to keep questioning rules to get continuous improvement. Check.
* Yes, agile teams plan (and often more accurately than teams using a sequential process)
* While enormously rewarding, it's also a real challenge for developers to take on and deliver working software by the end of each of Scrum's short, focused, timeboxed sprints. Yup.
* Scrum transitions almost all need not only an identified sponsor, but a champion who will allot it time.
* "Testers often must learn how to test a system without as much reliance on documentation." Ouch.
* When piloting an organization's first Scrum projects, the pace will initially be less predictable than with the organization's prior approach to software development (and you should be clear about that up front with stakeholders).
* Similarly, consider early calculations involving velocity to be suspect (and communicate that, too, to stakeholders).
* Most teams will overestimate how much they will achieve in their first sprint.
* A good ScrumMaster does not end very many days with impediments left unaddressed.
* It's not a slam-dunk to make a team lead or a project manager the scrummaster. Directing the team will fail; facilitation, not command-and-control, is the managerial behavior to reward.
* A team's sprints should all be the same length to let the team benefit from a regular cadence and make sprint and release planning easier.
* One of the most striking changes for programmers and testers on a scrum team is that they can no longer sit in their cubicles and wait to be told exactly what to do. The words "Hand me the perfect spec" or "Hand me the perfect requirements doc" represent an abdication of responsibility. Each programmer and tester needs to be thinking about the product and asking questions about each feature and how it adds to or detracts from the overall product. It's hard for testers to transition to testing a moving target on the fly. But they need to. Everyone on the team is responsible for the ultimate success of the project in the broadest sense.

And then there are the surprises that come from reading an author who is himself remarkably well-read across genres. In a chapter on Distributed Teams, he reprints part of a table that compares country cultures five ways - speaking truth to power, importance of team unity / comfort with individualism, need to achieve, tolerance for uncertainty, and long-term orientation - that gave me aha's just scanning the chart.

In another example, Mike remarkably provides what in essence is a crash course in change, and that chapter alone, while loaded with specifics about making a change to agile, could be used by any technical leader making a technology-and-culture change of any kind. Know why shopping carts were introduced and how shoppers were convinced to use them? Mike does, and effectively uses the example to teach change leadership. How does a leader deal with the skeptics and the saboteurs, the diehards and the followers certain to show up on your teams? How do you sway the uncertain to get on board?

Yet Cohn is sometimes unsatisfying as well - drilling down not quite far enough in sharing what's worked:
* "Hold individuals accountable... employees need to know they will be held accountable for applying the new skills the organization is paying them to acquire." Great, but what mechanisms have been most effective in doing so? And which that seemed like a good idea initially turned out to reward the wrong behavior?
* "Set reasonable targets... Rather than asking a team to 'start doing test-driven development,' the ScrumMaster should ask the team to develop one feature that way in the next sprint... encourage teams to select realistic, actionable targets." Been there. Ever watched a team select everyone else's feature as the one to develop in a new way? I have. The NIMBY response of coding. (NIMSOC? - Not in My Section of Code.) Again, a little more "how" would be useful.
* "Prepare in this Sprint for the next," he exhorts, but while giving a notion of what to accomplish pre-next-sprint, he fails to give a picture of what that looks like - the mechanics of interrupting this sprint for next sprint's business.
* To scale Scrum, he offers multiple approaches, one of which is sharing team members between feature and component teams, an intriguing and seemingly useful approach about which he writes only two short paragraphs. It desperately cries out for an example.
But in its 400-plus pages, it's remarkable how few of these unsatisfying examples there are.

He doesn't limit himself to the Scrum framework but devotes a chapter to the technical practices that arose out of XP: Test-Driven Development, Refactoring, Collective Ownership, Continuous Integration and Pair Programming.

Mike is practical right down to identifying the non-product groups that must nonetheless change or loosen their strictures for you to succeed at agile, from facilities letting you hang cards and charts on the walls, to h.r. valuing and rewarding teamwork over individual contributions, to finance and marketing's expectations regarding project delivery prediction. And to recognizing the likelihood that many agile transitions in the real world will face waterfall-driven expectations at the beginning (e.g., no funding without a project plan and specs) or at the end (the heave-ho to QA). Or even that agile will be coexisting with and collaborating with waterfall teams simultaneously. He charts courses for each. He clearly has had to resolve questions and objections from all quarters: he devotes a chapter to coexisting with other approaches, with regulatory standards and with customer mandates like ISO 9001 and CMMI. (Software house Systematic, "... after approximately two years at Level 5, decided to also adopt Scrum. It reports that [CMMI and Scrum] complemented each other well: 'Scrum now reduces every category of work (defects, rework, total work required, and process overhead) by almost 50% compared to our previous CMMI Level 5 implementation while maintain the same level of process discipline.' ")

Equally pragmatically, Mike doesn't just give the "what," but explains why his recommendations have value, and closes sections by walking through common objections and how he and others have countered them. Importantly, Mike includes examples from companies ranging from SalesForce, IBM and Google to lesser-known companies of how they have tackled their transitions to scrum. He's familiar enough with other process changes your company may have attempted to be able to compare and contrast and differentiate the substance and style of scrum from everything that came before.

Finally, I should point out that though it may not be immediately obvious, there's one more group of readers for whom this book has enormous value: Agile wannabes who know their current process isn't working. They frequently have heard of agile but have no idea what it looks like or how to sell it. If you're one of those, you also need to read this book. Mike Cohn over and over offers advice and examples and data that will give you aha after aha as you build a picture of what a truly effective process can look like - and how yours could become functional like his.