Colleges: Teach the "How"! by Ron Lichty

We're at an inflection point.

Agile practices transitioned over the last couple years from "emerging" to "predominant". Companies are sampling if not wholesale adopting agile, from scrum and extreme programming to lean and kanban. Our study of product team productivity due out later this month documents an increase to 76% of teams incorporating at least some agile methods, up from 71% last year. Even more significant, 84 percent of teams still using waterfall think they could increase the profitability of their product by adopting agile practices.

But agile practices are not easy. Agile has a reputation as "loose" but, in my experience, the various forms of agile represent the most rigorous methodologies, frameworks and practices I've come across in my 30 years in software development.

Companies adopting agile soon discover what those of us using agile for a decade already know: there are all too few developers (and managers and scrum masters and product owners) with any significant amount of training and experience in agile methods and practices. Too few developers have practiced pair programming. Too few developers have tried test driven development (TDD). Too few shops leverage practices that support continuous deployment. Too few teams practice team "ownership." Too few developers and testers and scrum masters and product owners have spent time as part of successful scrum teams. And too few managers have insight into the critical contribution they could make to the success of their companies' transitions.

There's good training out there - I've taken some of it and my clients tell me I've given some of it. But companies sign up for too little training. Dean Leffingwell reports that factories don't even begin a transition to lean without having first sent their managers and directors off to lean bootcamp for a minimum of two weeks and often six. And companies sign up for too little coaching. Agile's wins don't come from learning drills and nomenclature in meaningfully applying them - which requires understanding the meaning behind the drills - application of agile's principles to the unique culture and challenges each team and each company faces.

I think the place to set meaning and understanding in place is much, much earlier: when students first learn to code.

But the education community has done little to incorporate agile into curricula. I was shocked when I interviewed Stanford computer science students for internships two years ago to find, to a one, that they had heard not a mention of TDD or continuous deployment or scrum and had only a vague idea of pair programming.

Earlier this year, I pitched community college educators, at the ICT Educator Conference in San Francisco, on the value of teaching not just programming languages, algorithms, and logic (the "what") but also agile methods and practices (the "how" now coveted by hiring managers). I titled my talk "Win / Win / Win with Agile" because this inflection point and the dearth of developers with agile experience provide unprecedented upside for educators, students and hiring managers alike. A video interview I did with the conference organizer just posted this week on the wins for colleges and educators and their students.

Teaching test driven development is at the top of my list. We called out TDD in our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams. Valuable for the students? I can't express enough how highly I value resumes from programmers who can say they practiced TDD with every line of code they wrote.

TDD turns coding inside out: programmers are responsible for unit testing their code; before they write a single line of code, they write the tests that will prove the line works; they then run that test demonstrating that the test fails without the code; they then write the code and run the test again until the test succeeds. That cycle should take an experienced TDD coder between 30 seconds and two minutes. Rob Myers, who trains teams in TDD, describes a two-year effort writing a life-critical application yielding 9,000 unit tests that together made up a comprehensive automated regression test suite; the entire suite could be run in 10 minutes; and the team could make a change to a life-critical application and deploy the change, with confidence, within an hour.

I've found converting an entire team that has never used TDD is challenging. In my experience, no more than 1 in 5 programmers "gets it" - sees writing tests before writing code as an obvious way to improve quality, to ensure requirements are understood, and to enable later refactoring with impunity - with confidence in the outcome. The remaining 4 of 5 programmers find it not "natural" to write tests before writing code.

Unless they were taught from the beginning that writing tests first is the natural order of things.

As a hiring manager, I expect students who test first - who have incorporated TDD into their way of working so thoroughly that they do it naturally - will deliver a level of quality that matches or exceeds the best of what I can get from experienced programmers who have no experience with TDD. Students who can put TDD on their resumes - and speak to the practice with a level of confidence deeper than most experienced programmers - will win jobs.

But TDD is only the beginning. Bringing programmers (and tech leads and architects) into pair programming can be even more of a challenge. While pairing has been shown to result in higher quality code in comparable time than two programmers working separately, for many programmers it's as counter-intuitive as TDD - maybe more so.

Programmers need practice working in pairs and in teams to feel comfortable doing it. The good news - for both online learners and distributed teams - is that pair programming can be practiced by remote students and teammates, not those who meet to pair. It's slightly easier, in fact. In physical space, the two programmers share a single monitor and pass a single keyboard and a single mouse back and forth. In virtual space, they're connected via audio link so their conversation is the same, and they link their monitors so they're always looking at the same thing but each has their own keyboard and monitor - it's control of the cursor that, using conventions, gets passed back and forth.

Students who leave college having had to pair on assignments and having had to write tests first (TDD) would be dramatically more hirable. Industry would not only snatch them up but be enormously grateful to the colleges that taught students not only the what but the how. Such curricula would change the tenor of programming for the better - with regard to both quality and productivity - across their region. Institutions where curricula required students to learn Agile practices would see appreciation from students and hiring managers alike.

Colleges can deliver what we hiring managers are looking for, just by expecting students to:
    •    work in pairs (pair programming)
    •    write tests first (test driven development)
    •    partner in requirements elicitation (scrum)
    •    retrospect regularly (continuous learning)
    •    demonstrate facility with pomodoros (agile time management)
All the stuff that's so hard to teach and makes it so hard to transform organizations.

It's time for educators to step up, teach not just the "what" but the "how", and help transform this hard thing we do called software.

Remote Pair Programming by Ron Lichty

I've been enamored of remote pair programming since briefly leading development at Socialtext, where no three developers were in the same locale. Socialtext was committed to agile approaches including pair programming (but not including colocation, I might note!). Being distributed, remote pairing was the way forward.

Pair programming is controversial in being possibly the most difficult agile engineering practice to inculcate into development teams - we've heard of programmers quitting companies rather than signing on. But there's also plenty of evidence that it's well worth doing: Mike Cohn, in Succeeding with Agile, says, "if you're in a hurry this is the time you need pair programming the most… Although most studies show a slight increase in the total number of person-hours used when pairing, this is offset by a decrease in the total duration of the effort. That is, while pairing takes more person-hours, fewer hours pass on the clock…. Pair programming has also been shown to improve quality…. Additionally, pair programming facilitates knowledge transfer and is an ideal way to bring new developers up to speed on the application. It is also an effective practice for working in uncharted territory or solving difficult problems in known parts of the system."

In our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams, we note a less obvious advantage of the technique. We share David Vydra's interviewing rule of thumb, which states, "Pair programming for half an hour during an interview will save everyone's time."

Of remote pair programming, we also share the pleasure I saw programmers at Socialtext experience transitioning from physical to virtual pairing. (pp. 133-4)

While proximal pairs typically share a single screen and often a single keyboard and single monitor, virtual pairs each sit square to their own monitor, keyboard and mouse, sharing a virtual single screen. For screen sharing, developers at Socialtext typically used a version of VNC (e.g., Chicken or RealVNC).

As a co-chair of the Emerging Technology SIG of SVForum, I've recently been looking at collaboration tools of various kinds, led there in part by the amazing advances in sharing Web research that Vivek Agarwal at Surfmark is doing. That led me to consider that screen sharing in general might have taken a leap forward.

I didn't find anything startling, when I went looking for an update on remote pairing tools, but David Vydra, who seems to know and have worked with all the cool kids, pointed me to a wonderful resource, Joe Moore's blog, which is focused on remote pairing tools. Not surprisingly, Joe is part of Rob Mee's Pivotal Labs, known for its XP engineer practices.

The blog covers a lot of great stuff, including updates on tmux, Google Hangouts and MadEye, SublimeText 2, solving Skype challenges, and more. In his two-years-ago assessment of remote pairing technology, Moore concluded, "using Apple’s built in Screen Sharing.app (aka SSA) combined with Skype has proved to be the best remote paring technology combination."

For those who are distributed - or just working from home - remote pairing is alive and well.

Keep Your Culture Positive! by Ron Lichty

Which is more effective to improve team performance: positive feedback or constructive criticism? a positive culture or a negative one?

The answer: both.

The real question: in what proportion?

The answer to the proportion question, from new research by Michigan doctoral student Emily Heaphy and team productivity consultant Marcial Losada:

The ratio of positive-to-negative statements is directly correlated to "strikingly different results for each performance category" of teams:
* The highest-performing teams expressed 6 positive comments for every negative one
* Medium-performing teams expressed 2 positive comments for each negative one
* Low-performing teams swung the other way, expressing almost 3 negative comments for every positive one

"…in order to predict team performance, one only has to know the ratio of positive to negative interactions…", Heaphy and Losada concluded.

Jack Zenger and Joseph Folkman gave it a giving-feedback spin in their Harvard Business Review blog post, but nonetheless correctly summarized:

"The factor that made the greatest difference between the most and least successful teams: the ratio of positive to negative comments that participants made to one another."

Heaphy and Losada were studying leadership teams - 60 of them - and not development teams, but given they're measuring the nature of human satisfaction and response, development teams can't be far off.

They were measuring approving versus disapproving statements (not just feedback) - expressions of support, encouragement or appreciation (e.g., “that’s a good idea”), versus sarcasm, cynicism, or disapproval (e.g., “that’s about the dumbest thing I ever heard”).

Their finding may apply to feedback but the overall truth is about culture.

"Qualitative observations of the teams showed that high performance teams were characterized by an atmosphere of buoyancy that lasted during the whole meeting. By showing appreciation and encouragement to other members of the team, they created emotional spaces that were expansive and opened possibilities for action and creativity as shown in their strategic mission statements. In stark contrast, low performance teams operated in very restrictive emotional spaces created by lack of mutual support and enthusiasm, often in an atmosphere charged with distrust and cynicism."

Heaphy and Losada noted validation of prior research by other teams that showed positivity opens opportunities for action while negativity closes them down. (Interestingly, they also noted prior research into married couplehood that correlated remarkably similar ratios of positive-to-negative comments to stable marriages (a 5-1 ratio) and negative ratios to divorce. "Gottman’s research on married couples has shown that the best predictor of stable marriages is the ratio of positive to negative interactions…. Where his “performance” variable was the sustainability and quality of a marital relationship, we found that this same ratio of positive to negative interactions is the critical differentiator between high-, medium-, and low-performing teams.")

Negative comments must clearly be part of the mix. Pollyannaism - positivity alone - is no answer. It's just that most of us don't want to work in a culture that swings negative. What we all need to note is that the swing negative starts from a 6-to-1 positive-to-negative ratio!

Throughout our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams, we devote pages and pages to the value of praise, recognition and having fun with your teams - all key components to building a culture of positivity. "Say 'thank you' at least ten times every week," we counsel. "Never pass up an opportunity."

We just didn't have the researchers to measure it - proof positive.

Heaphy's and Losada's conclusion: "We need to have organizations with teams where the abundance of positivity, grounded in constructive negative feedback, can generate the state of realistic enthusiasm that can propel organizations to reach and uphold the heights of excellence."

Drawing in 3D by Ron Lichty

The way history worked, pens and brushes were invented millennia before the printing press - artistry before printing.

Which got us into the modern age, where 3D has been a different story.

Now, not only can 2D be printed - books and copies and newspapers - but 3D things can be created and manufactured based on plot points and axis control and software.

I don't want to suggest that they're not artistry, but they're repeatable and (depending on the specs) perfect. So where's the 3D paintbrush, the 3D pen, that one might have expected to come before? History is reversed in our modern age. Manufacturing before artistry.

It was fixing 3D printing that got designers Peter Dilworth and Max Bogue to realize what our age was missing. So they invented it.

Imagine a glue gun that extrudes a "line" of plastic in the air. Their 3D pen can draw in all dimensions in multiple colors in free-space.

You have to see it:

Motivating (and not de-motivating) Programmers by Ron Lichty

One of the great "aha"s for me as a manager was realizing that motivating people - and not de-motivating them - are two different things.

It was years ago - I think I was at Apple. One of my colleagues shared a Harvard Business Review article from the 1980s that recounted Frederick Herzberg's work in the 1950s. Titled "One More Time: How Do You Motivate Employees?", when it was republished in 1987 it had already sold more than 1.2 million reprints since its first publication 20 years earlier - 300,000 more than any other of the thousands of articles HBR had published.

Herzberg posited that motivation isn't so simple:
    ▪    there are a lot fewer factors that motivate people than we think ⁃ a lot fewer
    ▪    but there are an equal number of factors that, while they don't motivate, they'll de-motivate when they're lacking - when they go negative

By example, company policy / administration is just not a motivator - no one joins a company because of its administration. But it can sure be a major de-motivator. Almost all of us have seen companies with policies and administration that could deflate the morale of anyone.

While we summarize Herzberg's findings in our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams, and the article itself is well worth a read - his findings of what motivates and what de-motivates are general to the entire workforce population.

That led my co-author and me to adjust the categories and the weightings slightly, based on our own decades of experience managing software teams, to target those factors we believe motivate and de-motivate programmers.

Here's the chart we came up with: what we believe Herzberg would have found if he had applied his research just to programmers:

Chart showing the factors that motivate and demotivate programmers

The chart is ordered by the motivators - the blue vertical lines; these are the causes of satisfaction and thus what motivate. The red vertical lines, on the other hand, indicate the extent to which the same categories are de-motivators - causes of dissatisfaction. The height of each line, blue or red, is its statistical importance.

Notice that motivators and de-motivators are not aligned!

Stepping through the Motivators (the blue) is easy, given the chart is ordered by them. Here are the motivators in the order that we believe drive programmers to do great work:
    ▪    Making a Difference in the World
    ▪    Learning and Growing
    ▪    Toys and Technology
    ▪    Recognition and Praise
    ▪    Having fun
    ▪    Upside
    ▪    Interpersonal Relationships
    ▪    Compensation

We devised the first of these, "Making a Difference in the World," to both take in Herzberg’s category of Achievement and take it a step further to capture the reason many if not most of us started programming – the opportunity to make a difference. I remember showing off my first programs to my wife and feeling the pride of achievement. But even with those first small efforts I could see the power of the craft to positively affect the world – what a motivator that was!

It's worth noting that the 6th category, "Upside" - that would be stock, stock options, bonuses, that sort of thing - is the first mention of money, and it's 6th down the order! And "Compensation" doesn't come in until 8th! (That's about where Herzberg puts "Compensation" for the general population, too, by the way.) Once you're paying people enough - adding more comp is just not that significant a motivator.

Stepping through the De-motivators (the red) (lack of these are the reasons programmers quit) is a bit harder. Starting with the highest red line, here are the factors in the order we believe they de-motivate programmers:
    ▪    Lack of respect for supervisor
    ▪    Lack of having fun
    ▪    Lack of learning and growing
    ▪    Working conditions
    ▪    Company policies and administration
    ▪    Lack of ethical management
    ▪    Compensation

Notice the first of those is out-and-out us - poor performance by their manager! Programmers may not hire into a company out of respect for their new manager, but they'll surely consider leaving if they don't respect their manager. In fact, there's a rule of thumb in H.R. that "People don't leave their companies. They leave their managers."

"Having Fun" and "Learning and Growing" appear on both lists - motivators and de-motivators. Contributing to making your environment a fun place to work and ensuring that staff have opportunities to learn and grow are enormously important both to retention (keeping your people from leaving) and to keeping your people engaged and productive. And then come "Working Conditions" and "Company Policies and Administration". Fighting for your people and their needs - and shielding them from corporate politics and inanities - are critical contributions managers must make. Again, people don't hire in for the working conditions or the company policies, but they'll surely leave because of bad ones. And then there's "Ethical Management" - whether from a team's direct manager or all the way to the execs, lack of ethics will squeeze the life out of a programming team.

Much as for the motivators, it's only then, this time 7th on the list of de-motivators, that we come to "Compensation"! (Also near where Herzberg placed it for the general population.)

In fact, my coauthor and I issue warnings - in our discussions of motivating programming teams - about attempting to motivate with money, and the studies and experience that lead us to issue those warnings.

There's not room in a single blog post to discuss every factor, show you where our chart differs from Herzberg's observations of the general populace (and describe why), flip the chart to order it by the de-motivators, share with you the other two motivation theorists we find incisive and wrote about in Managing the Unmanageable, or delve more deeply into why, in general, we caution against motivating with money.

But I'm hoping it's enough to clearly show the tightrope that programming managers walk to motivate their teams and the individual programmers that comprise them - and not deflate their motivation.

And I'm hoping it will suggest how project managers, program managers, product managers and other team leaders, by being aware, can collaborate with programming managers in motivating (and not de-motivating) programming teams.

What makes product teams great? by Ron Lichty

What makes product teams great?

What if you were able to name just five things that:
    ▪    if you don't do them, you have just a 2% chance of high level team performance
    ▪    if you do all of them, you have a 67% likelihood of high level team performance

Last year's Study of Product Team Performance found just that. Five factors to a 65%-more-likely high performance team:
    ▪    unwavering executive team support
    ▪    strong team alignment w strategy
    ▪    post-production development focus and accountability
    ▪    effective on-boarding of new team members
    ▪    assigning core team members based on skills needed

One of the reasons the study is so useful is that several of those require getting your senior execs on board and the rest require getting your peers on board. And one of the hard truths - about senior managers, at least - is that they tend to believe studies and consultants more than their own people. (And that's one reason why my co-author and I collected 300 Rules of Thumb for managing software people and teams that we share as a quickly thumb-able center section of our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams (Addison Wesley, Oct. 2012). And why studies like this one are so important.)

But we need your help.

There were disproportionately too few techies in last year's survey mix. We need to get the word out to developers and development managers and UX designers to join the other members of product teams to ensure teams are fully represented.

Participate in the 2013 Study!
    http://assn2.questionpro.com/
The study is a great way to get data we can all use to influence our management and our teams. (And if you want, it's a great way to get notified of the 2013 study's findings.)

As for those five factors, you can download all of last year's study (including some of the comments behind the data).

(For my blog post on another finding of last year's study, see my previous post, "Better cross-functional collaboration, trust and communication")

Better cross-functional collaboration, trust and communication by Ron Lichty

You can participate:
what makes product teams great?

In 2012's Study of Product Team Performance, respondents were asked what they would change about the core product team.

"Better cross-functional collaboration, trust and communication" was their number-one response.

(You can participate in this year's study. See below.)

I was struck by how closely that aspiration aligns with a step that management took 15 years ago when I was at Schwab, a step I was convinced was responsible for my team becoming one of the highest performing I've had opportunity to manage.

The time was February 1997. The team was embarking on building Schwab's first web-based customer applications in Java. At the beginning of 1997, Java was mostly being used for eye candy: animated thermometers, dancing fonts, blinking messages, that sort of thing. Java applets didn't even print. Not that there were a lot of alternatives: HTML was flat and lifeless; Javascript wasn't yet built into a single browser; the only alternative was COM, also very early as a platform. To build business applications in Java at the beginning of 1997 was not just leading edge, it was bleeding edge.

To support the effort, senior management hired a team-building h.r. group - they go by names like "organizational development" - "OD" - but they're all about building collaboration, trust and communication - soft skills - emotional intelligence - that sort of thing.

As keyed up as management was to prove to the execs that we could do something new, something breathtaking, something never done before - they arranged for the entire product team to take entire afternoons offsite multiple times during our short four-month development cycle. Developers, testers, project managers, product managers, business analysts, architects, instructional designers, the whole bunch of us.

That project progressed from research and ideas to working product - an asset allocation toolkit - with functionality that had never before been built even in the client-server world - in a few short months. The team won awards; the product was lauded; Schwab was lionized.

At the time, I remember thinking that I never wanted to work on a product team again that did not undertake significant cross-functional trust and collaboration team-building. I recounted the experience in our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams. Sadly, I've never worked on one since that did.

The 2012 study reminded me of what gave that Schwab team greatness. The study examined the interactions of Product Managers, Project Managers, Program Managers, Business Analysts, Developers and others actively involved in product development projects.  "The goal of our research was to better understand the dynamics of product team performance and to uncover the practices that make these teams successful. What makes this survey unique is that it enjoys the support of various industry associations and market players — groups and individuals that don't generally work together."

This year's Study of Product Team Performance is launching now and running till February 28th or until it has received 3,000 responses, whichever comes first.

Get in on it. Share what's made teams great in your world - at:
   http://assn2.questionpro.com   

Could we make managing programmers a little easier? by Ron Lichty

Programming managers have typically had years of training in programming…

And no training in management. 

I've been taking a fascinating census of programming managers this fall that has shown me just how true that is. As I talk about our book Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams to groups that include software development managers, I start by asking:

"Stand up if you are now or have ever been a manager of programmers and programmer teams."

I continue:

"Stay standing if you have ever received management training

- let's say a day or more of training -

whether from your company or in college or on your own.

Almost to an audience, half of managers sit down! 

Half of programming managers have never had a single day of management training in their careers!

To the people still standing, I ask:

"Stay standing if you got any of that training before you first became a manager."

The result: 

Of all the programming managers I talk to, only 8-15 percent have received any management training before becoming a manager.

Almost all the training for managing programmers that exists.
There are only two others that are missing from this picture!

Equally telling: there are a ton of books on project management. In fact, there are scores of books on each different flavor of project management.

But the number of books on managing programmers won't use up the fingers on both hands. (It's one of the reasons that Mickey and I wrote Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams.)

Isn't it odd?:

* how long we expect you to have studied the art of programming before we hire you to be a programmer?  languages, libraries, frameworks, tools, techniques, ...?

* the growing expectation that not only have you studied to be a project manager, but you've become certified to do so?

Isn't it odd how little we expect you to have studied the art of managing before we tap you to become a manager?

Orchestrating Software Development by Ron Lichty

For years, I've been pointing development and project managers to the full-length documentary, Tom Dowd & the Language of Music. (You can rent via Netflix, purchase via Amazon or Barnes & Noble.)

Every time I watch Tom Dowd, I'm struck by how closely producing music resembles leading software development.

Tom Dowd, an innovative music producer and recording engineer for over half a century, altered the course of contemporary music not only through his many technical achievements but also through his ability to coach some of the greatest musicians of our lifetimes, including Eric Clapton, the Allman Brothers, Lynyrd Skynyrd, Aretha Franklin, Ray Charles, Marshall Tucker, The Drifters and many, many more.

To accomplish with programming teams what Tom Dowd accomplished with rock and roll musicians is a goal I have long strived for.

Just as Dowd's goal was to stir musicians to together create music more thrilling than they thought possible, our goal as development leaders is to stir our programming teams to create software more thrilling and satisfying than any of us thought possible. 

As a bit more background, Tom Dowd was employed at 16 to do math for, unbeknownst to him, the WWII atom bomb project - the Manhattan Project - because they figured they'd get at least two years of engineering and math from his young mind before he risked being drafted. He was such a part of that project by the time he turned 18 that while he was sent off to basic training, he was immediately returned to NYC to continue calculating. But when the war ended, there was no segue to a nuclear physics career - Tom Dowd knew more than the academics he would have had to study from! In frustration, he turned his engineering ability to his other love, music. It was a time when groups went into the studio and laid down an entire album in an afternoon, each song a single monaural track. He was hired into Atlantic Records as a recording engineer. He continued to engineer, mix and produce music until his death in 2002.

His engineering prowess combined with his musical ear to give him a ticket to play. He was one of the first to lay down increasing numbers of separate tracks. (An actor portrays Tom Dowd in the Ray Charles movie: at the sound board, Dowd not only mixed but invented the overlays and the engineering that Ray Charles needed.)

As Dowd's coaching skills developed, he became responsible for some of the great collaborations in music. When Cream was having trouble bringing their individual parts together in "In the Sunshine of Your Love", it was Dowd who pointed them to the beat change that brought the piece to life and fame. And it was Tom Dowd who introduced Duane Allman and Eric Clapton, resulting in their classic Derek and the Dominos collaboration, Layla.

From the first days of programming, the crossover between programmers and musicians has long been noted. Mickey and I discuss it in our book, Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams. (Mickey was a professional musician himself, once, and still makes command performances for family and friends; and I had years of training in piano, trumpet and guitar.) 

But the management crossover - the similarity between orchestrating programmers and orchestrating musicians - hasn't been as obvious, as written about or as noted.

Eric Clapton appears in the documentary to give voice to the contribution made by his producer. "The quality and the success of those recordings could mostly be laid at the doorstep of Tom Dowd." 

Butch Trucks, drummer of the Allman Brothers, says, "Tom had a way of making things work - had a way of pulling us together - being this father figure and this psychologist that would dig into the depths of us and find what we were capable of doing."

From one of the members of Lynyrd Skynyrd, "He could bring the best out of you without you ever knowing it and all of a sudden, wow, there it is."

Again, Eric Clapton: "I've always been riddled with self-doubt about my work, especially as a composer…. His role was making me feel comfortable and inspiring confidence in myself."

One of Dowd's colleagues said, "He's a coach. He's the absolute perfect coach."

For those of us eager to have that kind of impact in software, Tom Dowd and the Language of Music is a movie worth watching.

Managing the Unmanageable is in print! by Ron Lichty

Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams finally arrived. 

There have ever only been a handful of books on managing programmers and teams (while project managers have benefited from having a ton of books on project management of every kind). 

That's what led my coauthor and I to start writing. That and our wish to share the scores - now hundreds - of rules of thumb and nuggets of wisdom we'd been collecting from our peers. We'd been trading them for the decade we'd known each other. They seemed more broadly useful. 

Now all that's in a book. Rules. Nine chapters of our insights from a combined 70 years in software. And the tools we've used to manage. 

Addison Wesley sent us first copies off the press last week. Amazon is shipping!

"PM"s by Ron Lichty

I am repeatedly in conversations with someone who says, "Oh, PMs blah blah blah." 

And I have to figure out which kind of PM they're talking about. 

(It happened again yesterday afternoon. The person speaking was a product manager, which made it likely that the "PMs" to which he referred were other product managers, not project managers. Then this morning, I got an email with the subject line,  "The Evolving PM". Again I had to think. The sender was a project management association, making it likely that the "PMs" to which it referred were other project managers, not product managers.)

I'm not a PM. I'm not a PM of any kind. I'm not a Project Manager, I'm not a Product Manager, I'm not a Program Manager, I'm not a Publication Manager (engagement last year: Stanford subsidiary HighWire, which hosts web sites for 1400 of the world's most prestigious academic and scholarly journals from 150 publishers worldwide, and when I arrived, had 14 "PMs" for me to manage - who were account managers!).

I'm an interim VP Engineering. And a consulting CTO. And an Agile trainer and coach of Agile transformations. And an author of the Addison Wesley book (just released today!):

   Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams 

   www.ManagingTheUnmanageable.net  

The common abbreviation terminology I've been pushing for, in my engagements:

PjM == Project Manager

PdM == Product Manager

PgM == Program Manager

They're all self-clarifying.

I give talks on Transforming Chaos to Clarity.

Project Managers and Product Managers and Program Managers are, to a one, charged with clarifying software development, not adding to the chaos.

"PM" adds to the chaos. 

I wish that were…

'nuff said.

Software Engineering Radio by Ron Lichty

Robert Blumen recorded the most recent one on Aspect Oriented Programming. He did an earlier one on Hadoop and has upcoming ones on Project Voldemort, NoSQL, workflow management, cloud architectures, high-performance web architecture and event-driven systems. (Robert was a senior systems architect at Morgan Stanley a decade ago when I was leading Schwab's Java initiative.)

Popular topic areas on Software Engineering Radio include languages, patterns, agile, process, SOA, and DSLs.

Looks like good stuff!

Book review: Succeeding with Agile by Ron Lichty

The first half of Mike Cohn's book, "Succeeding with Agile: Software Development Using Scrum", isn't so much about how to be successful with agile as how to be successful replacing non-agile with agile, selling agile, and transitioning to agile. Mike Cohn is neither a wild-eyed dogmatist professing the one true way nor an ivory-tower philosopher; he's a pragmatist. He devotes entire chapters to helping scrum evangelists decide whether to start with a pilot or go all-in across the organization, whether to pilot publicly or in stealth, what factors might prevent success, how to swing things your way, how to approach a transition in steps, mechanisms he calls Improvement Communities that along with a more global Enterprise Transition Community can both spread scrum across your enterprise and debug adoption issues, ways to use scrum to manage the transition to scrum, picking the right first project and first team, and how soon to adopt agile technical practices like simple design, pair programming, TDD, continuous builds, refactoring, and automated testing.

He provides helpful anecdotes of two highly visible companies that, when waterfall stopped being effective, tried getting more rigorous about it, which turned out to just make things worse - then finally yielded and tried agile and got visible benefits almost immediately.

Yet he also notes from the first chapter that agile implementations are hard and don't always "take" - and explains why.

He also does us all the favor of reviewing and summarizing scads of studies and surveys that demonstrate why agile is worth the effort, with data on productivity gains, lower costs, improved employee engagement and job satisfaction, faster time to market, higher quality and, importantly, stakeholder satisfaction gains. (Have you seen the study of pair programming that shows it an effective way to counter Brooks' Law - the one about adding programmers to a late project making it later? Or the study of the negative impact of multitasking on productivity? Or the one showing productivity of teams larger than seven plummeting? Or the one that shows R&D teams do need new members to stay vibrant - but only at the rate of one very 3-4 years!)

About midway through the book, as you're reading insights into teamwork, you realize that this book is not just about transitioning to agile but about getting really good at agile - and that you're going to need to read it not just once but come back to it for new insights again and again. The fact is it's stuffed with tips, guides, data, approaches, failures, successes, and helpful hints of every kind. What's the role of a manager of a self-organizing team? One of them, agile gurus seem to agree, is exercising subtle control (as opposed to command-and-control) and Mike provides an entire framework for debugging team problems and influencing their success. This will be one of those chapters I know I'll revisit repeatedly.

It is good to have in print solutions to the many conundrums and traps that agile teams face:

* How to deal with work that seems indivisible into sprints

* How to make user experience, architecture and database design part of sprints, not separate waterfall efforts

* How to make testers part of the team and testing part of the team's work

* How to hold a hard line against mid-sprint change - and what to do when you can't ("abnormal termination", followed by immediate planning for a new sprint - never extend a sprint nor change its flavor!)

* Avoid the anti-pattern of doing design in one sprint, handing off to development in the next, handing off to QA in the third - designers, developers and testers should be working together in a sprint to solve, develop and deliver complete stories within a single sprint timebox

* How to deliver visible functionality every sprint even when you're mostly developing unobservable features (and why you want to)

* Figuring out how to get a slice of working application functionality every sprint (not just components, libraries or layers)

...and so many others.

Part of what makes this book important for me is repeatedly reading observations consistent with my own experience, but drawing on so many more experiences with agile that it results in emergent patterns, as well as in truths that are too often left unsaid:

* Agile is not successfully implemented by fiat, without support from the folks at the front lines, but similarly it's not successful as a groundswell without senior management support. Check. Seen both.

* Agile is not a set of rules to be followed to the "T" but has to be adapted uniquely to each organization where it's implemented. It requires an iterative "provoke and observe" approach. Check. Seen that one.

* Furthermore, attempting to turn software development into a set of rules is counterproductive for the very reason that you need to keep questioning rules to get continuous improvement. Check.

* Yes, agile teams plan (and often more accurately than teams using a sequential process)

* While enormously rewarding, it's also a real challenge for developers to take on and deliver working software by the end of each of Scrum's short, focused, timeboxed sprints. Yup.

* Scrum transitions almost all need not only an identified sponsor, but a champion who will allot it time.

* "Testers often must learn how to test a system without as much reliance on documentation." Ouch.

* When piloting an organization's first Scrum projects, the pace will initially be less predictable than with the organization's prior approach to software development (and you should be clear about that up front with stakeholders).

* Similarly, consider early calculations involving velocity to be suspect (and communicate that, too, to stakeholders).

* Most teams will overestimate how much they will achieve in their first sprint.

* A good ScrumMaster does not end very many days with impediments left unaddressed.

* It's not a slam-dunk to make a team lead or a project manager the scrummaster. Directing the team will fail; facilitation, not command-and-control, is the managerial behavior to reward.

* A team's sprints should all be the same length to let the team benefit from a regular cadence and make sprint and release planning easier.

* One of the most striking changes for programmers and testers on a scrum team is that they can no longer sit in their cubicles and wait to be told exactly what to do. The words "Hand me the perfect spec" or "Hand me the perfect requirements doc" represent an abdication of responsibility. Each programmer and tester needs to be thinking about the product and asking questions about each feature and how it adds to or detracts from the overall product. It's hard for testers to transition to testing a moving target on the fly. But they need to. Everyone on the team is responsible for the ultimate success of the project in the broadest sense.

And then there are the surprises that come from reading an author who is himself remarkably well-read across genres. In a chapter on Distributed Teams, he reprints part of a table that compares country cultures five ways - speaking truth to power, importance of team unity / comfort with individualism, need to achieve, tolerance for uncertainty, and long-term orientation - that gave me aha's just scanning the chart.

In another example, Mike remarkably provides what in essence is a crash course in change, and that chapter alone, while loaded with specifics about making a change to agile, could be used by any technical leader making a technology-and-culture change of any kind. Know why shopping carts were introduced and how shoppers were convinced to use them? Mike does, and effectively uses the example to teach change leadership. How does a leader deal with the skeptics and the saboteurs, the diehards and the followers certain to show up on your teams? How do you sway the uncertain to get on board?

Yet Cohn is sometimes unsatisfying as well - drilling down not quite far enough in sharing what's worked:

* "Hold individuals accountable... employees need to know they will be held accountable for applying the new skills the organization is paying them to acquire." Great, but what mechanisms have been most effective in doing so? And which that seemed like a good idea initially turned out to reward the wrong behavior?

* "Set reasonable targets... Rather than asking a team to 'start doing test-driven development,' the ScrumMaster should ask the team to develop one feature that way in the next sprint... encourage teams to select realistic, actionable targets." Been there. Ever watched a team select everyone else's feature as the one to develop in a new way? I have. The NIMBY response of coding. (NIMSOC? - Not in My Section of Code.) Again, a little more "how" would be useful.

* "Prepare in this Sprint for the next," he exhorts, but while giving a notion of what to accomplish pre-next-sprint, he fails to give a picture of what that looks like - the mechanics of interrupting this sprint for next sprint's business.

* To scale Scrum, he offers multiple approaches, one of which is sharing team members between feature and component teams, an intriguing and seemingly useful approach about which he writes only two short paragraphs. It desperately cries out for an example.

But in its 400-plus pages, it's remarkable how few of these unsatisfying examples there are.

He doesn't limit himself to the Scrum framework but devotes a chapter to the technical practices that arose out of XP: Test-Driven Development, Refactoring, Collective Ownership, Continuous Integration and Pair Programming.

Mike is practical right down to identifying the non-product groups that must nonetheless change or loosen their strictures for you to succeed at agile, from facilities letting you hang cards and charts on the walls, to h.r. valuing and rewarding teamwork over individual contributions, to finance and marketing's expectations regarding project delivery prediction. And to recognizing the likelihood that many agile transitions in the real world will face waterfall-driven expectations at the beginning (e.g., no funding without a project plan and specs) or at the end (the heave-ho to QA). Or even that agile will be coexisting with and collaborating with waterfall teams simultaneously. He charts courses for each. He clearly has had to resolve questions and objections from all quarters: he devotes a chapter to coexisting with other approaches, with regulatory standards and with customer mandates like ISO 9001 and CMMI. (Software house Systematic, "... after approximately two years at Level 5, decided to also adopt Scrum. It reports that [CMMI and Scrum] complemented each other well: 'Scrum now reduces every category of work (defects, rework, total work required, and process overhead) by almost 50% compared to our previous CMMI Level 5 implementation while maintain the same level of process discipline.' ")

Equally pragmatically, Mike doesn't just give the "what," but explains why his recommendations have value, and closes sections by walking through common objections and how he and others have countered them. Importantly, Mike includes examples from companies ranging from SalesForce, IBM and Google to lesser-known companies of how they have tackled their transitions to scrum. He's familiar enough with other process changes your company may have attempted to be able to compare and contrast and differentiate the substance and style of scrum from everything that came before.

Finally, I should point out that though it may not be immediately obvious, there's one more group of readers for whom this book has enormous value: Agile wannabes who know their current process isn't working. They frequently have heard of agile but have no idea what it looks like or how to sell it. If you're one of those, you also need to read this book. Mike Cohn over and over offers advice and examples and data that will give you aha after aha as you build a picture of what a truly effective process can look like - and how yours could become functional like his.

Web 2.0: Leveraging Community by Ron Lichty

Anyone remember when you put a music CD into your computer and what you saw was always, ALWAYS blank: "Track 1, Track 2, Track 3, ..." unless you typed in the names of the tracks yourself?

Today, things are different. When you use iTunes today, there's a brief wait when it tells you it's checking with "Gracenote". And voila, there's the name of the CD, the artist, the tracks.

But there's no more information recorded on today's CDs than there was 15 years ago. There's no information there. None. So where did it come from?

What a couple guys in a garage started doing sometime over a decade ago, about the time the web was emerging, was to keep track of type-ins of the CD names/artists/tracks -- they made it easy to remember other people's type-ins as well. And they saved them in a database along with a "fingerprint" of the CD -- the length of the CD itself, the number of tracks, and the EXACT length of each track. The next time someone anywhere in the world inserted a copy of that CD, its fingerprint was sent to Gracenote, Gracenote sent back the content information to the Gracenote-enabled player, and it would magically display the CD's name/artists/tracks as though they were recorded on the CD's laser-pitted surface.

What started as a couple guys in a garage became Gracenote, the technology behind iTunes and almost every other playing technology not just for CDs but also MP3s as well.

Newsweek just posted a little story online (may be in the print magazine next week, but not sure they're always the same) called "To Catch a Sneak", in which Gracenote Chief Technical Officer Ty Roberts explains how Gracenote helped expose a fraud in the music industry -- the classical pianist Joyce Hatto, who was briefly called "the greatest pianist that no one had never heard of" until Gracenote technology played a role in exposing that CDs issued as hers by her producer husband had in fact been performed by others.

The fraud and the expose story are fascinating. And the article helps explain what Gracenote does in just a little more detail, if you’re interested.

"Web 2.0" in part references the power that letting readers and users of a site contribute to the site's content can bring. Gracenote is an example of one of the earliest of user-content-generated content, and of that power. Much as Wikipedia is a user-generated encyclopedia, Gracenote was and is user-generated music identification technology that shares the contributions of the first with all the rest of us to make our lives easier.

One Laptop Per Child by Ron Lichty

The status of the One Laptop Per Child project revealed to SDForum's Emerging Tech SIG this week by architect Ivan Krstic is even more remarkable than I'd previously heard.

You may remember Nicholas Negroponte and Kofi Annan announced, in 2005, this nonprofit humanitarian effort to change how kids learn, starting with a laptop so inexpensive it can fill the world's children's needs and provide a computer for every child, starting with the billion school-age kids in the developing world. OLPC is targeting an initial unit cost of $150, getting it down to $100 by mid '08.

Ivan said he was recruited with the question, "Can you secure 100 million laptops? Oh, and rewrite the file system? Oh, and by the way, your first users will be six years old." I remember going to Apple when its mission was still just six words, "We're going to change the world." This project has that feeling, in spades. He's one of a core team that until recently numbered only 12 and is still only 14.

Nonetheless, the project has prototype units and intends in the coming year to ship 5-10 million units starting later this year. Ivan says they've been approached by virtually every country in the world.

There's no longer a crank on the computer itself -- but it's designed to connect to a unique pull-cord generator -- like a yo-yo that adapts to the strength of the user; or an outboard crank; or a car battery; or solar cells. Through multiple innovations, it consumes less than a tenth of the power of the typical laptop today. It supports the 802.11s ESS mesh that lets the laptops form into ad hoc networks with or without an internet connection -- but if any one of them has an internet connection, they all do through the mesh. They've measured up to 2 kilometer connection distances via the two bunny-ear antennae, each of which can connect to a different network.

The rubber membrane keyboard is impermeable to dust, sand and dirt in addition to water, and the touchpad is the width of the entire laptop. The dual mode 7.5" 1200x900-pixel display, at 200 dpi, is higher resolution than 95 percent of displays available today and may be more visible in sunlight than any other laptop existing today. It has three USB ports, an SD card slot (added for those countries that insist on adding Microsoft OS and Office applications), stereo speakers, mic and a VGA 30 frames-per-second camera. Inside is an AMD Geode LX-700 0.8 watt, 433 MHz, with 128K L1 cache and 128K L2 cache. They're developing and focused on open source only. The GUI, window manager, security, file system and search were all written in Python.

They're expecting each country to develop OLPC software for the culture and in the languages needed by their children. Right on the keyboard is a "View Source" key -- nothing hidden, no secrets. They're providing foundational stuff like a plug-in architecture to manage software growth and bloat; the file system Ivan was brought in to work on; an object store for user's work, with version control, meta data tracking, pervasive search and ability to sort, filter and display by timeline; presence -- which feature may set a standard that doesn't yet exist for presence; and out-of-box security without requiring passwords or locking down the physical machines.

It's exciting. And it's cute. And as Ivan kept saying, cool as the technology is, it's really about changing how kids learn.

The TED Conference by Ron Lichty

Recommended viewing:

The TED Conference - Technology, Entertainment and Design - has been on my conference wish list for nearly a decade. I heard of it first from a UI designer colleague who worships Richard Saul Wurman, who founded the conference. I've had a few friends who were lucky enough to attend and came back with stories that succeeded in whetting every envy neuron in my body.

Now, the TED Conference has video clip highlights online (click on the Highlights tab).


The highlights "reels" are all short and virtually every one of them worth watching, but I highly recommend checking out:

• designer William McDonough
  
• science writer Laurie Garrett
  
• inventors Bill Gross and Dean Kamen
  
• artist Arthur Ganson
  
• primatologist Jane Goodall
  

There are also a half dozen of this year's talks -- the whole thing -- online:

• Hans Rosling's talk shows amazing information visualization technology capability.
  
• Believe it or not, Al Gore's warmup is funnier than the comedian in the short highlight clips (and Al's message is a follow-up to his recently released movie).
  
• There's a brief look at Tony Robbins and why the life coach is so compelling to so many.
  
• Finally, you DO want to listen to South Bronx community organizer and MacArthur winner Majora Carter. She'll make you think twice about how things work in New York, in New Orleans, in your town.
  

One day, I'll get to TED firsthand, but until then...

The Nonprofit of the Future by Ron Lichty

Intriguing. By 2016, what changes will technology have wrought in nonprofits? “Can you envision the nonprofit of 2016?” How will the role and the work of nonprofits change?

That’s the topic of a discussion started by writer Paul Lamb. He intends, from the discussion, to create a collective article, with attribution to contributors, and any profits from the piece to be contributed to the Nonprofit Technology Enterprise Network (N-TEN).

Pointer to this discussion was in the SF Chronicle’s The Tech Chronicles blog, which comments...

Over the last decade the number of nonprofits has grown by 68 percent and now includes some 837,000 organizations, according to a recent report by the National Council of Nonprofit Associations. Based on IRS data, the council estimated that these groups had combined assets of $1.76 trillion in 2003 -- which would make the "nonprofit economy the sixth largest in the world, larger than the economies of Brazil, Russia, Canada, Mexico, and South Korea."

If your interest, like mine, is in the community and collaboration that technology can enable, take a look at the discussion, and contribute your own vision of the technology-enabled nonprofit's future.

The next photography revolution by Ron Lichty

Think about what it will be like when every digital camera has a GPS, so that your camera not only date- and time-stamps your photos, but also location-stamps them.

Think about searching through Flickr (or just your own personal collection of photos) by first bringing up a map and zeroing in on Bear Valley to find the cross-country ski training you did for the Leukemia and Lymphoma Society's Team in Training, or West Yellowstone and Anchorage to find photos of the ski marathons you succeeded in completing. Or zeroing in on your parents' street to find the photos they sent of Christmas Eve at their house, or to the PBwiki mansion on the Peninsula to find the photos of the last hackathon held there.

It makes it all the more important to get geocoding support built natively into databases like SQL Server. (Microsoft knows this, by the way. At their recent SF announcement event for SQL Server Service Pack 1, SQL Server Always On, and SQL Server Everywhere, it was Microsoft Data and Storage Platform SVP Paul Flessner from whom I first heard mention of GPS-stamped photos, and Flessner's slide of SQL Server futures had the note "Spatial goes mainstream" tagged onto version "vNext".)

new Ruby conference by Ron Lichty

SDForum is looking to bring together yet another community of technologists in the Bay Area -- programmers using and interested in using the Ruby programming language (and its associated framework Rails).

The kickoff will be the weekend Silicon Valley Ruby Conference April 22-23.

review: The Virtual Handshake by Ron Lichty

I wrote this review for SDForum's bimonthly newsletter. It was just published on page 20 of the Feb/March SDForum newsletter.


The Virtual Handshake
by David Teten and Scott Allen
reviewed by Ron Lichty

How do you most effectively leverage the new social software tools that have emerged in the past few years? As David Teten and Scott Allen ably advise in their new book, The Virtual Handshake, these tools can help you meet new people, maintain relationships, build your own flexible, lifetime network, open doors, close deals and build professional success.

The new tools include blogs, social network sites, relationship capital management software and biography analysis software. Some of the older tools include contact management software, shared workspaces, personal Web sites, e-mail lists, instant messaging and Web conferencing.

In our July SDForum / SofTech meeting, "Architecting Community and Collaboration Solutions," our panel looked at tools like blogs, wikis, meeting software, group email, and message boards, seeking to answer these kinds of questions. (There's a summary of the event, along with a lot of other discoveries both before and since, on the official event blog at:
http://ronlichty.blogspot.com/ )

Teten and Allen got the luxury of a couple more years, 250 pages and a web site (and hopefully an advance and some royalties) to explore those questions, so took them to a deeper level. It's not often that you read a book in an area where you have interest and passion and discover authors who both deepen and broaden your thinking. It's equally rare to find a book that, despite being published, as books are, months after they're written and more months after they were researched, that nonetheless introduces technologies and applications and services that seem as fresh as if they were posted to a web site yesterday. The Virtual Handshake was that for me. Here are everything from lists of providers to rules for effective use to the subtleties of using them for best success. My copy is tagged with Post-its and laced with highlighting.

Who are the providers of all these areas of online services? See the table they provide, by the way updated on the authors’ website at:
www.TheVirtualHandshake.com/map

How do you promote your blog? Have you built relationships with the A-listers, syndicated your page, connected with other bloggers, and submitted your site to the specialized blog search engines? Do you know who all these people and services are? Teten and Allen do.

What can you do that you might not have? iCohere in Walnut Creek used its own software to host a four-day online virtual conference on collaborative learning. Are you holding a business meeting? Have you checked out Cvent, LeverageSoftware.com, or PowerMingle to connect attendees beforehand, and prepare them to meet each other?

How do you improve the effectiveness of your email approaches to people you know well? How about to those to whom you’ve only just been introduced? How do you write emails that don’t get tossed before they’re even read? What’s appropriate use of email and what’s not?

How do you truly connect with people online? How do you build and nurture your network?

More general advice? There's some of that here, too. As you set up your web site and your blog, they advise, make it easy for others to republish your content, perhaps using the Creative Commons licenses.

The Virtual Handshake removes some of the mystery of blogs that newcomers often struggle with, such as terminology like “Permalink” and “Trackback”.

To the other extreme, one of the most fascinating aspects of the book are examples of how people are using social software and services in ways that even long-experienced online networkers have likely never thought of.

Take the Value Investors Club, an exclusive investment ideas network where membership requires writing up an "A+" investment idea for the other members -- and maintaining membership requires two to six more such analyses each year. It’s fascinating that the 250 members know each other only by aliases and can communicate only to the entire community –preventing private messages for purposes like cross-firm recruiting.

Or did you know branding expert Rob Frankel "runs a weekly public chat every Monday morning in which he donates one hour of his time to anyone who drops by. He likens it to the academic tradition of 'office hours.'"

Or do you know the story of the Lockergnome service, a score of free tech newsletters with over a million subscribers -- as well as the annual Gnomedex convention, the exclusive Pirillo.com thinktank, regular TV hosting spots and magazine columns, and a couple books – that Chris Pirillo first began in the form of emails of his online discoveries to his college friends? All that activity has made Pirillo a recognized leader these days in email publishing.

From how to manage the e-mail deluge to setting up the virtual you, from increasing the relevance of your network to doubling your “weak ties” (and why you’d want to), The Virtual Handshake can help. Whether you’re a rookie or a veteran at using these services, and whether you’re in marketing or sales, or looking for a job (or preparing for the eventuality that you one day will be), this book will point you in the right direction.

Blogging in Business by Ron Lichty

"...the real excitement here is not how much money business can make from blogging, but how dramatically blogging will reshape the world of business from top to bottom and create new sources of competitive advantage for firms that learn how to use this new medium intelligently," says David Kline in his AlwaysOn post The Voice of the Customer :: AO.

That's a pretty good description of what we were trying to get at, regarding not only blogs but community and collaboration software of all kinds, with our SDForum/SofTech event, Architecting Community and Collaboration Solutions.

Kline notes it's not so different from the Web itself, where "...the real story of corporate America's embrace of the Internet in the last 10 years is not so much a tale of how much money has been made online, although that is considerable, but of how thoroughly almost every facet of global business has been altered by that embrace."

He goes on to make the case for the opportunity businesses have to leverage blogs for product definition -- to invite customers "to tell you directly, in their own often-rambling but always revealing words, exactly what they would like your new product to look like and why? All firms have to do is get their engineers and marketers blogging with their customers."

Kline is co-author of Blog!
How the Newest Media Revolution Is Changing Politics, Business, and Culture.

For more insight into the varied uses of blogging in business, visit David Kline's BlogRevolt.com

Web 2.0 and ALL that... by Ron Lichty

Always On is hosting Marc Canter's amazing tour-de-force through Web 2.0, Breaking the Web Wide Open! :: AO

Want to know about microformats and mashups and the semantic web, and get understandings of SIP (Session Initiation Protocol for internet telephony and conferencing), FOAF (Friend Of A Friend, a schema to describe people and relationships in a way that computers can parse), xHTML, XFN (a microformat standard for representing your social network), and OPML (Outline Processor Markup Language, a hierarchical file format for storing microcontent and structured data developed by Dave Winer of RSS and podcast fame)?

This is the place.

Podcasting by Ron Lichty

Podcasting was the topic of back-to-back presentations at BayCHI's September meeting, and not only were the presentations highly enlightening regarding podcasts and how to do them, but so was the coverage by Liesel Mendoza that came out last last week in BayCHI's newsletter. Liesel's excellent writeup is not online yet, but is provided under the Creative Commons license, so I am reprinting it here in its entirety.

BayCHI is the Bay Area chapter of the Computer Human Interface SIG of ACM, the software professionals organization. The writeup should be posted to the event's page later this month or early in November.

------- keep reading for Liesel Mendoza's coverage of the Podcasting presentations at BayCHI --------

September Meeting Report

by Liesel Mendoza, lmendoza@baychi.org

Insights on Podcasting: Podcast Solutions and Podcast Problems
Dan Klass, The Bitterest Pill
+
Podcasting: Media Evolution or Revolution?
Doug Kaye, IT Conversations

Audio and Slides: http://www.baychi.org/calendar/20050913/

-----

Insights on Podcasting: Podcast Solutions and Podcast Problems
Dan Klass, The Bitterest Pill

Dan Klass asked what the audience wanted to know about podcasting and
said he would be happy to share what he knows.

Dan introduced himself as a stay-at-home dad. He is often fetching
milk, fetching juice, turning the channel, changing diapers, making play
date phone calls, and driving kids to and from pre-school and play
dates. He expected his BayCHI presentation would be the longest he has
spoken without interruption in years.

Before becoming a stay-at-home dad, he was a failing actor in Los
Angeles. He was failing in the sense that the hook has been in his
mouth and fallen out, again and again. He was also a comedian.
Performing became less attractive once he had children. He said he no
longer feels the sick and psychotic need to be loved by an audience now
he has kids. However, the need to express oneself and be creative in
some way does not die.

Dan's venture into podcasting started when his wife gave him an iPod for
his birthday. To him, the iPod is the technological equivalent of a
Faberge egg. Once you take it out of the box, you have to immediately
guard it so as not to be scratched or marred. He surfed the internet to
look for that glow-in-the-dark scuba suit that people put on their
iPods. Instead, he found an iPod icon on a video of a couple of folks
talking about Adam Curry and podcasting, Adam Curry being a long-haired
MTV VJ he admired from the '80s. This triggered his interest in doing
his own podcast. He dragged his children to Radio Shack, got a
microphone, hooked it up to his computer, and started recording his
first podcast.

He started podcasting about stay-at-home dads on the fringes of the
entertainment industry.

Dan tackled questions from the audience:

Q: What is a podcast?

A: To generalize, a podcast is a sound file (an MP3) on the internet.
You can subscribe to a series of podcasts. Software is available to
subscribe and automatically download podcasts through RSS (Really Simple
Syndication).

There are probably over 10,000 podcasts today. If you have a specific
topic in mind and don't find a podcast on it, you may want to do a
podcast on that topic. Someone just like you is sitting somewhere
looking for a podcast on the same subject.

Q: How did you come from wanting to have a podcast to actually doing
one? What were the challenges you had to face?

A: The biggest challenge is that Dan is not a technological person. His
background is entertainment, laziness, and fatherhood. To read about
RSS on the internet was not simple. But he did it.

He started recording his show, turned it into an MP3 file, and then
published to his blog. He used RSS, which he figured out by stealing
some code and looking for the part where it has ".mp3," hoping that
replacing it with his file would work.

Dan says, to start with, you must have something to say. Content
creation is the big deal! If you have something to say, you should do
it. His book contains a section talking about the importance of content
creation.

As a comedian, he made fun of other people, society, or anything other
than himself. When he started podcasting, he decided that he will only
talk about himself, about being a stay-at-home dad. He had faith in
narrowcasting. He was not alone. True enough, he received a lot of
emails identifying with him, such as, "I am a dad and I don't like to be
a dad like my dad was. I want to be the dad that you seem to be trying
to be." He started building a community about being a dad. That was
when he knew he was on to something.

Q: How is building the community taking place?

A: Initially, people emailed asking him to check it their own podcasts.
After a while, that faded out, and the emails were from active listeners
of his podcast.

The community building starts with podcasters doing their show. This
becomes the first dialogue with the audience. The users download the
podcast and listen to it at their own time, whether on the train or
while driving. Most podcasters have a blog. There is a blog entry for
each podcast. People who had listened to the podcast start posting
comments on the blog, sending email, leaving a voicemail, or emailing a
sound file. As time goes by and people become more sophisticated, they
may start using Skype. The podcaster then talks back to these people
through more podcasts. The dialogue is phenomenal.

Q: How do you promote your podcast? How do you use the web or your blog
to promote your podcasts? How do people find you?

A: A podcaster needs to get into directories by submitting the podcast's
name and feed URL to iTunes, Podcast Alley, and Podcast Pickle. Dan
even designed his own podcast logo, a funny little head. Never
underestimate the value of a good logo. His logo is worth thousands of
listeners! Never underestimate the power of your own blog. Never
underestimate the value of a good title for your podcast. For example,
he would use a title like "George Clooney and Diapers and Jesus What a
Podcast?" Someone has to stumble into this title somehow.

- Google will crawl your blog.
- Send out press releases.
- Submit to podcast directories.
- Use mailing lists and forums.
- Be active in the community of podcasting.
- Use word of mouth.
- Spread the word through friends.

These are ways to promote your podcast. The podcasting community is an
incredibly supportive and incestuous community. When new people show
up, it's all about getting them started. Money and competition have not
turned podcasting into something evil. It is still a very warm and
nurturing atmosphere as opposed to a commercialized one.

Q: Will this change with more and more large corporations getting into
podcasting?

A: Yes, it will change. Dave Winer, one of the creators of podcasting,
wants podcasting to be kept pure. Not going to happen.

Human nature gravitates towards what is familiar. When someone looks
for a podcast, will he look for some guys working in a small room or for
a well-known network? However, Dan's optimistic side tells him that
there will always be an audience for the underdog. There will always an
audience who rejects the establishment, the corporate-fed and nurtured
product. Will it be harder to get noticed? Yes, maybe. iTunes has
always left a space for the small guy. Can the little guy survive?
Sure hope so. Not everybody likes Britney Spears. Not everybody loves
Raymond. Those who don't will look down on the other part of the tail.
There will always be a need for the narrow part of the tail that is
wagging furiously today.

Q: Have you ever been contacted by any organization to advertise for
them?

A: No. Dan jokingly says had been waiting for Pampers or Juice Box to
contact him. There are sponsored podcasts. Dan would consider doing a
sponsored one. However, some people believe that once you start
accepting money, then it turns into something other than honest content.
If Disney is paying you, you can't make fun of Disney!

Q: Do you have Google ads?

A: No. Dan recommends using Google ads in his book, but hasn't used it
yet.

Q: Is vidcasting emerging?

A: In producing a podcast, you create an RSS feed with enclosures. The
enclosure points to a file, usually an audio file, but it can be a video
file. Yes, they are coming. Video files are much larger, unless
they're shorter. But they are coming. One vidcast example is Tiki Bar
TV (http://tikibartv.blogspot.com/). The guy is funny. The girl is
cute. The show teaches you how to mix a drink in the end. It is just
about four minutes.

Q: What are the copyright issues in podcasting?

A: In radio, you can play any song. When you create a podcast, you
cannot put just any song on your podcast. The difference is that radio
does broadcasting; when you create a podcast, you are distributing
peer-to-peer. You can not include a song in a podcast without explicit
permission from the recording companies. The solution is to find music
that you have permission to use. There's the Podsafe Music Network
(http://music.podshow.com/). Go to garageband.com or magnatune.com.
You can find over a million pieces of music from any genre that you can
download in two seconds.

Q: What do you think of standalone player living on a computer vs.
iPod?

A: It doesn't matter, just listen to the show. What matters is that
you can listen to the show when you have the time using any media.

Q: What do you recommend for people who can't afford an iPod?

A: For people who can not afford an iPod, Dan's podcast is broadcast on
1550 AM KYCY in San Francisco, Wednesdays 8:00-8:45 a.m. Not everybody
can afford an iPod. He is thrilled to be able to do broadcast in his
spare bedroom.

-----

Podcasting: Media Evolution or Revolution?
Doug Kaye, IT Conversations

Doug Kaye is the founder of IT Conversations, recently recognized by
Business Week Online as the best podcast provider. He is also the
author of Loosely Coupled: The Missing Pieces of Web Services (2003, RDS
Associates).

IT Conversations is a podcast channel that disseminates exciting
content, rated by listeners.

Doug was a recording engineer and sound editor in the film business. In
the '70s, he started a computer business and ran it for eighteen years
before selling it. He got involved with four dot-coms. In 2000, he
left his last IT gig. He became a consultant and started writing a
book. The first book, Strategies for Web Hosting and Managed Services
(2001, Wiley) was a success. He had fun writing it.

In the process of writing the second book on web services, he
interviewed folks about what web services are. He recorded his
interviews. It occurred to him that the people he interviewed would
always know much more than he did on web services. As an author, he was
getting in the way of sharing what they know. He was not a good
vehicle. So he went back to some of the folks he interviewed and asked
them if they would mind redoing the interview for the purpose of putting
it out on the internet as audio. They agreed. That was in May, 2003,
which pre-dates the term "podcasting." A year after the first audio
posting, podcasting caught up with IT Conversations.

The prerequisite to podcasting is to have something to say. The most
successful podcasts come from a podcaster's passion for something. The
passion translates and transcends whatever technical challenge they may
encounter to do a podcast.

Podcasting was a curiosity, but it has taken over Doug's life and the
lives of the volunteers working with him. They have produced 720
programs, publishing them at a rate of two per day. He now has a team
of volunteers doing post production work. The team is nearly 50 people
all over the world.

A bit more story about Doug's initial venture into IT Conversations:

- Started with interviews.
- Produced a couple of third-party interviews with bloggers.
- Did a live stream of eTech in San Diego with about 220 simultaneous
listeners at peak and recorded the audio.
- Put the eTech recordings up as MP3 files, which turned out to be
more popular than the original stream, with about 20,000 listeners.
- Repeated the process with more conferences and recordings.
- The top show so far has about 180,000 listeners.

From the beginning, listeners have contacted Doug, wanting to send him
money, but he wasn't set up to take money. People insisted, so he set
up a "tip jar." People started sending him audio files that they
recorded for publishing, but the audio quality and content was awful.
Instead, he gave them advice on how do better. Some did improve, and
now he publishes them.

The biggest gratification for Doug is when people tell him he has
changed their lives. To hear that is addictive. He started thinking
what he can do so he can wake up every morning to more emails from
people saying he had changed their lives.

Today, IT Conversations has 90 shows in the queue for post-production
work. July was the biggest month yet: They delivered eleven terabytes
of MP3 files!

Some UI pointers from Doug:

- People hate audio, because one can't scan audio. People who read
fast would like to consume 30 minutes of audio in five minutes.
- Listening time: Most listening habits are the same as radio: during
drive time, in flight, and while exercising.
- Show length: The most popular shows are 20-30 minutes. The curve
falls off pretty quickly for shows over 40 minutes.
- Shelf life: Almost all programs have long shelf life. Some
programs, like Dan Klass's, are sequential: Some new listeners might
go back to previous editions, but most start with the current show
and subscribe to future shows. By working on conferences, IT
Conversations strives for programs that have long shelf life. IT
Conversations stays away from news.
- Player compatibility: It's important to use a sample rate that's a
multiple of 11.025 KHz (22.05 or 44.1 KHz), because Flash players only
play those rates. IT Conversations has standardized on a bit rates
of 64 kilobits per second, a nice compromise between quality and
file size. In the future, the standard may go up to 128kbps.
- Method of download: Provide manual downloads as well as subscription.

Some questions Doug answered during his talk:

Q: How do you work your way through 720 files?

A: Find content, mark it, put it into your personal queue much like
Netflix. You can either get it manually or set an RSS reader to
download it automatically, so new shows arrive like email.

Q: How do people handle long shows?

A: It's important to break up shows that go over 80 minutes, because
some listeners want to burn it to CD and listen to it while driving.

Q: How do people find audio?

A: A huge percentage of people find podcasts through Google, because
Doug and his team invest a great deal in metadata and descriptive text.
Every program they produce has its own page with a description, a
biography of the speaker, links to associated references, and, in some
instances, a photo of the speaker. They have not done any outbound
advertising, but they achieve high search engine ranking because of the
good quality of their content. Half of Doug's team are engineers and
half are writers.

A new project Doug and his team are working on has to do with
narrowcasting. A lot of spoken word events evaporate every day. Doug's
team is developing a tool called events and venues database (EVD) that
people interested in producing podcasts can use. It's a bottom-up
approach, using grassroots, unmoderated content. They will also
continue to produce more seminars and continue recording what IT
Conversations does, which is top-down, curated content and
presentations.

Q: Do you have recommended hardware configurations for little groups who
are into narrowcasting?

A: They provide information on hardware recommendations for small
groups. They are talking about setting up a Podcast Academy for this
purpose.

Q: What are the problems working with volunteers?

A: Whenever you work with volunteers, one of the challenges is keeping
the quality up, both editorial and the audio. On the editing side, it
is about people not writing very well. You have to work with them.
They created a role called series producer. The series producer will
take all the work and decide which of them are worth posting based on
content and audio quality. Series producers also play the role of copy
editors. They improve the copy. They also created the role of mentors.
The mentors work with the volunteers to improve their work, but
recognize that some just won't make the cut.

All the shows need to be produced with the same loudness.

IT Conversations standardized on MP2 as an intermediate format, because
it's non-lossy, with MP3 being the final format for publication.

Q: For IT Conversations, how are shows being rated?

A: IT Conversations has a rating system. They send listeners an email
ballot asking them what they think about the show. They get about 20-30
ballots per show. They have three criteria: Each show must be
educational, inspirational, and entertaining. The most important
criterion is that it must be inspirational. One of the most popular
stories was about a guy who went to the North Pole!

Popularity can be measured by the number of people who listen to a show,
or by how the show is rated, and there is some correlation. When people
give a show high rating, they often blog about it, which brings in more
listeners. High value content does bring in traffic.

The most popular shows are solo presentations in conferences with
well-prepared and condensed content. The next most popular is
one-on-one interviews where the interviewer is really good. At the
bottom of the ratings is the panel discussion. People seem to have
trouble discriminating voices. Panel discussion is a cheap way out for
event producers, often sponsors of the event. The conversation tends to
be inwards, with the panelists talking among themselves rather than to
the audience.

Q: Do you find that listeners ever excerpt shows? How do you link to
the middle of a show?

A: IT Conversations has a clip feature. You can specify start and end
time, and it will create a URL that will play the selected excerpt. On
an average day, they get less than ten clicks to the clip feature.

Q: How about synchronizing audio and slides?

A: It has been very time consuming to produce. They experimented with
Flash and Shockwave. The technology is not standard enough yet.

Q: Are you planning to have translators translate the shows?

A: If someone volunteers to translate a show, IT Conversations tries to
ensure that they have the legal processes and tools in place to do so.
But they are not planning on investing in translation. They are,
however, planning on bringing in foreign content.

Q: Any plans about talking to universities?

A: IT Conversations is talking to Stanford, Berkeley, MIT, and Harvard.
Many universities are recording programs, but they do not know what to
do with them. They are trying to get them out there. SJSU has whole
podcasting department. One of the problems of universities is about
rights and permissions--ownership of content.

Conference organizers are sometimes concerned that publishing audio will
reduce paid registration. To avoid that, IT Conversations releases
conference sessions one per week. If a conference has 25 sessions, the
listener won't complete the series until about six months after the
event. And IT Conversation vastly increases the conference's reach: The
conference itself may reaches 4,000 attendees, but the podcast will let
the organizers reach 40,000 more people.

However, in universities, departments and instructors consider their
materials proprietary. It continues to be a challenge to get these
materials out to the public for free, but Doug would like to work with
universities on this.

Q: Do you foresee any problems about getting rights to content from all
over the world?

A: Yes. IT Conversations has forms or contracts that curators must fill
out to indicate they have permissions. If anyone complains, they take
the show down.

Another of Doug's missions is to keep content free. This is something
pragmatic. Content that is free has more value. Imagine a blog with a
link to a newspaper story. If the link goes to a toll gate, people
can't read the story, so the blogger won't link to it. Charging a fee
for content decreases its value, because not everyone can get to it.
When content is free, it stays in the conversation. It can be remixed,
reused, and repurposed.

-----------------

This entry of my blog is licensed under the Creative Commons Attribution-NonCommercial-
ShareAlike License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/2.5/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.