

More From Forbes

Edit Story

Measuring What Matters: From Outputs to Outcomes: Part 3


Measuring What Matters: From Outputs to Outcomes: Part 3


We live now in the age of customer capitalism. The era of shareholder capitalism, i.e. pushing products and services at customers, of tweaking the supply chain, of parsing and manufacturing demand, with the single-minded goal of making money of shareholders, is for most practical purposes over.

As a result of epochal shift of power in the marketplace from seller to buyer, the customer is now in charge. Making money and corporate survival now depend not merely on satisfying customers but delighting them. To prosper, firms must offer a continuing supply of new value and deliver it sooner. The new bottom line of business is: is the customer delighted? As Umair Haque has pointed out in The New Capitalist Manifesto, it’s a fundamental shift from outputs to outcomes.

Measuring outcomes: customer delight

But it’s not enough just to talk about outcomes and delighting customers, as managers did in the 20th Century. To manage the new bottom line of business, we have to measure it. Thus traditional management was right to insist on the importance of measurement. The trouble was that they were measuring the wrong things. They were measuring outputs, rather than outcomes. By counting outputs, i.e. things, they were missing what is really driving the business, the outcomes with customers. Measuring the right thing, i.e. people based-results or outcomes, requires five fundamental shifts:

  • Part 1: Measuring customer delight at the organizational level (Part 1)
  • Part 2: Measuring customer delight at the working level through user stories: (Part 2)

I will discuss today:

  • Part 3: Measuring customer delight: sizing and prioritizing user stories.

Coming segments discuss:

  • Part 4: Measuring customer delight: the primacy of time (Part 4)
  • Part 5: Measuring customer delight in real time: social media (Part 5)

3.  Measuring customer delight: sizing & prioritizing

User stories represent hypotheses as to what might delight a customer. The next step is to size and prioritize the stories, i.e. measure how much work will be involved in executing the story, i.e. making the story come true, and decide what priority it should have in the work flow. This is a significant shift from traditional management practice.

The contrast with traditional management

In traditional management, work is typically lumped together in a comprehensive project specification to produce a certain output; a team is then assigned to complete the entirety of the work in the specification.

In a world where work is stable and predictable, and customer requirements are unchanging, this approach can work reasonably well. But in a world of rapid change in the marketplace, of an increasing proportion of knowledge work, of increasing technical complexity of the work, and where customers who do not even know themselves what will delight them, stable predictable work environments are largely a thing of the past.

Proceeding with a traditional approach in today’s environment will result in a specification of outputs that will include variety of things—some things that customer really wants, others that the customer would consider nice to have but not essential, and still other things that the customer doesn’t really care about it at all, but which have been included in the specifications, “just in case”.

When work is organized in such “big chunk” specifications, there are usually serious productivity problems:

  • A great deal of time and effort is often spent on things that are of low priority to the ultimate customer, thereby delaying delivery of the things that the customer most dearly wants.
  • Because the management has often not examined the relationship between the amount of effort that each individual component of the specifications will take, or how much delight it will generate, no sensible decision can be made on tradeoffs.
  • Because many tasks are grouped together into a large single specification, the overall project typically takes a long time to complete, with the likelihood that the customer’s situation will have changed by the time of delivery. As a result, the eventual product of service, even if it corresponds to the project specifications, will not be what the customer wants at the time of delivery.
  • When it is ultimately discovered that the output does not delight the customer, a significant amount of rework is probable.

In the traditional approach of “big chunk” specifications, these problems will remain largely invisible, because measurement and management attention are focused on outputs, not outcomes. Once management undertakes the more strenuous goal of achieving desirable customer outcomes, then a more targeted and agile approach to planning and measurement is required, so as to optimize the prospects of delighting the customer.

Optimizing customer outcomes requires short work cycles

Iterations help establish the cadence of work. Every week, or fortnight, or month, something gets done. After a time, people can make plans based on a track record of delivery. The amount of work that can be accomplished in an iteration becomes apparent. The team can calculate its velocity and so understand whether it is improving. In this way, the productivity of work to be measured and tracked.

By organizing work in short cycles, it becomes possible to focus effort on the user stories that the customer most dearly wants and complete those first, thereby optimizing customer delight.

By sizing user stories, it becomes possible to make sensible tradeoffs between the amount of work to be devoted to executing each story and the amount of delight likely to be created. Thus there is no necessary correlation between the amount of work that a story will take to execute and the amount of customer delight that it will generate.

  • It may transpire that two user stories, each having the same value for the customer, will take radically different amounts of work to complete. By focusing effort on the user story that that takes less work, customer delight can be optimized.
  • A user story that takes a great deal of time to complete may relate to some aspect that the customer doesn’t care about and so yield only a small amount of customer delight.
  • Equally, a user story that the customer very dearly wants may be quite simple and quick to complete: finishing this story early will help optimize customer delight.

How stories are sized through story points

Software developers have created a methodology for measuring the work involved in completing hard-to-measure things like user stories in the form of story points, as described in Mike Cohn’s book, Agile Estimating and Planning .

To calculate the total story points of a set of tasks defined in terms of user stories, one starts by finding the user story involving the least amount of work in the group of tasks and assigning it a value of one story point. Software developers have found that the technique known as planning poker[i] can be helpful here: it is used to compare the amount of work in this user story with the amount of work in each of the other user stories to be worked on and assign story point values to each of those. The sum of the story point values of all the user stories is an estimate of how much work is involved in finishing all the tasks.

User stories are prioritized

Once user stories have been sized, the customer (or customer proxy) can then order the user stories in a list from the most valuable to the least important. In software development, such a list is sometimes called a burndown chart. Once the user stories have been sized, it becomes possible to assess how many of the top-priority user stories can be completed in the coming work cycle.

Measuring the team's velocity

The user story can give a handle on what to measure. Through the sizing and prioritization of stories, and the tracking of the progress of their completion, it becomes possible to measure the velocity of a team. In this way, the team can track whether or not its productivity is improving.

The estimated velocity of a team is the sum of the story point estimates of the work that the team believes can be completed in the cycle. The actual velocity of the team is the sum of the story point estimates of work actually completed in the cycle.

How the completion of the user story is measured

In traditional management, when a piece of work is partially completed, the work team is usually credited with a positive accomplishment. “Work in progress” is usually counted as a partial success in terms of progress towards the final accomplishment of an output. Organizations may have large amounts of work “in process” without realizing that they have accomplished nothing in terms of delighting the customer. Moreover having large amounts of work “in process” increases the risk of significant rework and usually signals a serious productivity problem.

By contrast, once the management focus shifts from outputs to outcomes, a stricter accounting is required to ensure greater discipline in getting work finished quickly. This reflects the reality that partial completion of a user story is generally a bad outcome for the customer, i.e. the customer is not getting what was wanted at the time expected. Until the user story is completely finished, the customer is more likely to be frustrated than delighted. Hence “work in progress” is not counted as a positive accomplishment. The only thing that counts is full completion of the user story.

Thus suppose a work cycle includes four user stories. If at the end of the work cycle, the team has completed half of each of the four user stories, traditional management would view the job as 50% complete. In terms of customer outcomes, the team has accomplished nothing. It would have been better for the team to focus on fully completing two of the user stories, so that the customer at least gets something. In the approach to measurement based on outcomes, half-completed user stories count as zero.

For other parts of this series:
Part 3: Measuring customer delight: sizing and prioritizing user stories (this post)
Coming soon:
Part 4: Measuring customer delight: the primacy of time.
Part 5: Measuring customer delight in real time: social media.

Learn how to revolutionize the world of work: May 12-13, Washington DC

My book, The Leader's Guide to Radical Management (Jossey-Bass, 2010) provides a comprehensive overview of how to measure client delight.

The measurement of outcomes and client delight require more than the adoption of a new set of terminologies and tools. It is a different way of thinking, speaking and acting in the workplace.

If you would like to learn how to revolutionize the world of work by delighting customers and measuring their delight, please join me, Rod Collins (author of Leadership in a Wiki World, Seth Kahan (author of Getting Change Right) and others for two days on May 12-13 in Washington DC. Cool, innovative and serious fun. More details here.

[i] Planning poker is a technique for rapidly estimating, prioritizing, or evaluating a set of items by a group of people. It is simple, quick, and fun. It draws on the talents of the entire group of people present, not just those who speak loudest or most often. It engages people actively in the discussion, flushing out different opinions and clarifying the reasons behind the differences. It slices through hierarchical differences among a group as well and gives everyone an equal voice. It encourages expertise-based results rather than status-based outcomes. It is widely used to estimate effort or relative size of tasks in software development. It can be used to establish priorities or preferences among competing items for a wide variety of tasks.