Think Lean
Lean thinking aims to create a smooth, levelled flow of work by removing waste, minimising variation and avoiding overburden. Waste includes inventory and work-in-progress, defects, delays, and unused employee creativity. Examples of variation are frequent changes to the team and varying release cycles. Overburden occurs when people and resources cannot cope with the workload placed on them. If we want a lean product backlog, then it should contain as little waste and variation, and cause as little overburden as possible.

Eliminate Waste
Waste consumes valuable resources and makes it harder to focus on what’s important. To remove waste in the product backlog, reduce the inventory  the backlog holds, avoid overproduction and minimise defects, handoffs and wasted creativity, as I explain in more detail below. Reduce the inventory: Minimise  the amount of detailed product backlog items and only include items in the backlog that are essential for creating a successful product. Ensure  that just-enough high-priority items are detailed just in time for the next sprint planning meeting. As a consequence, product backlog items are  progressively decomposed and refined – from sprint to sprint. Lower-priority items stay coarse grained and sketchy until their priority changes.
Avoid overproduction – providing more functionality than users and customer need. Focus on the minimum functionality necessary to
bring the product to life, and only list truly valuable items in the backlog. Remove all other items from the product backlog. This keeps
the product backlog concise and the Scrum team focused. If the item becomes important for a future version, it will re-emerge.
Minimise defects, handoffs and  unused creativity by involving the team members and the stakeholders in grooming the product backlog. Jointly discovering and describing product backlog items avoids handing off requirements to the team. It ensures clarity of the requirements thereby reducing defects; and it leverages the creativity and knowledge of the team members and stakeholders. Jointly prioritising the product backlog ensures that technical risks and dependencies are accounted for. Problems consequently surface early, which allows us to prevent defects at a later stage of the project.

Limit Undesirable Variation
Not all variation in the product backlog is wrong. The size of the product backlog items should vary according to their priority, for instance. But unnecessary variation, also called unevenness, prevents a smooth flow of work. Try the following to minimise variation in the product backlog: Standardise the techniques for describing  product backlog items. Choose user stories to capture functional requirements and operational qualities such as performance and robustness requirements, for instance. Agree on a common way to describe usability requirements such as sketches, wire-frames or mock-ups.
Use a sprint goal. The sprint goal summarizes the desired outcome of the sprint, and moves the Scrum team a step closer toward the release of the product. A shared sprint goal ensures that everyone is working toward a common goal. It minimises variation by limiting the type of requirements worked on in a given sprint, for instance, by choosing items from the same theme. This facilitates close teamwork and can increase velocity.
Ensure that the high-priority items have roughly the same size and favour small items – items that can be transformed into a part of the product increment within a few days. This reduces variation, improves the progress tracking within the sprint, and prevents defects by allowing the product owner to provide just-in-time feedback on the work results. Note that this approach works best when the team uses agile development practices including storydriven development.
Create a steady cadence by using fixed-length releases. Timebox your projects: Identify the window of opportunity based on the product vision and the product backlog, and freeze the release date. Take Salesforce.com, a leading provider of on-demand customer relationship management services. The company releases a product update every four months. As a consequence, Salesforce.com experienced an amazing increase in the number of features delivered while drastically reducing its lead-time for new functionality. Note that a fast, steady cadence supports other measures including minimising the inventory in the product backlog.

Prevent Overburden
As long as people work crazy hours, and as long as projects and teams are overwhelmed by the amount of work, the removal of waste and variation is ineffective. Waste and variation are likely to creep back in unless we limit the amount of work to the capacity and capabilities of the organization. Let’s assume we try to eliminate defects but the project still suffers from overburden. Chances are that quality problems reappear since the project members still feel pressured and are overworked. In fact, overburden is a major source of waste including work-inprogress, waiting and delays, taskswitching, and defects.
To eliminate overburden, let the product backlog evolve based on customer and user feedback. View changing requirements as a competitive advantage and leverage the feedback together with the project progress to decide which functionality is implemented. Encourage the team to pull only as many items into the sprint as they can transform into a product increment in a sustainable way. Ensure that the high-priority items are ready: They should be clear, testable and feasible. This avoids that the team overlooks tasks and pulls too much work into the sprint. And last but not least, make high-priority items small. This allows the team to optimise its work utilisation. It also avoids the danger of missing tasks – which is a common issue with large stories.

Summary
Keep your backlog simple, concise and lean. Focus on the items that are essential for bringing the product to life. Be courageous and ruthlessly weed out all other items. This will make your product backlog lean and increase the likelihood to develop a successful product.

As a Scrum Coach I sometimes come across projects that simply aren’t working out. Most of those have a problem with the communication between the Product Owner and the Team. Here I present an approach based on the intellect of Dan North.

As I see it, the Product Backlog is not only a list of functionality and issues. It can also be put into service to increase the developer team’s domain knowledge, commitment and collaboration as well. On several occasions, I have been successful in improving the communication between the Product Owner and the Team by using the Product Backlog as the interface through which communication takes place. By shifting some of the Product Owner’s responsibilities to the Team, I have managed to get them involved in the evolution of the requirements.

Epics – Stories – Scenarios

Evolutionary requirements isn’t just a way to minimize the amount of changing requirements and amount of time spent before development starts. It is also a great way to build a bridge from the Product Owner and his problem domain to the Team and the solution domain.
I find it useful to think in terms of Epics-Stories-Scenarios and use the Team to help the Product Owner evolve the requirements along those lines to build the bridge. To get the Team into the habit of working with the Product Owner and the requirements, I introduce the responsibilities of Business Analyst and Quality Assurance to the team, one person per responsibility, and I rotate them each Sprint.

Rotating responsibilities
Introducing the Business Analyst and Quality Assurance and advocating the shared responsibility for the Product Backlog removes uncertainty about what’s expected of everyone involved as well as strengthening the collaborative climate of the project. Rotating the responsibilities provides the team with a natural way of sharing knowledge and increasing the overall competence of the team members. All of this builds trust and confidence – important factors in getting self-organization and commitment up and running.
Some may argue that the introduction of Business Analyst and Quality Assurance goes against the no-role policy of Scrum, but as far as I am concerned it is about delivering working Software and whatever gets you there is good practice. Don’t forget that Scrum is about continuous improvement rather than process roll-out, and the most important thing is to get it right the final time!

The job of a Business Analyst The Business Analyst starts his job in Sprint Zero, i.e. the Sprint preceding the first development Sprint. He collaborates with the Product Owner to:
♣ get the overall business goals of the project in place
♥ establish the level of detail for the requirements and how they should be communicated to the Team (definition of “done” for a story, to put it simply).

It is important not to do too much of it, since the level of detail will shift as the team and Product owner get to know each other and their understanding of the problem and solution domain increases.
The overall business goals are the problems the projects are supposed to solve and are expressed in terms of epics:

As a [stakeholder] I want [feature] so that [benefit].

For each Epic the Stakeholders should be identified to enable the Team to involve actual people when it comes to breaking the Epics down into Stories or understanding the problem at hand. The more involved the team is in this process, the better they will understand the business and the better they will become at making informed decisions.
♣ Keep the Epics on cards in big black letters and stick them on a wall to make them visible to everyone. If you run out of wall it probably means that there are too many Epics.
♦ At the same time this is being done, the Team should start considering the technical solutions available and start creating the overall architecture in the form of simple models – stick those models on the wall as well to keep them visible and usable for sprint planning and daily discussions.
♠ This is also the time for the Team to start asking questions along the lines of ‘what does it actually mean to …’ and ‘how is it done at the moment and why isn’t that good enough’ as well as questions regarding the potential users: ‘who are they’ and ‘how can we make it easy for them’. Asking these kinds of questions helps identify the incidental stakeholders, the owners of the non-functional requirements who come into the picture as a result of the implementation decisions made – anyone affected in some way by the results of the Team’s work. and refined together with the Product Owner, and communicates them to the Team at estimation. When development of a Story is done, Quality Assurance is responsible for verifying that all Scenarios are fulfilled and that the agreement with the Product Owner is honored. If automatic acceptance tests are used, it is Quality Assurance together with a developer that makes sure tests are written for each Scenario and that the tests are in good shape. Having the entire Story in place, you are finally ready to do some estimation. I normally let the team do estimation before the Scenarios are written, simply because I don’t want too much effort going in to the requirements before being sure they will actually be developed and to give the Product Owner some sense of cost to enable planning. This of course means that the estimates might change when the scenarios come into play. In that case you would have to re-estimate or remove some Scenarios to stay within the estimate. The removed Scenarios should be estimated and kept in the Product Backlog to ensure that nothing is lost.. The next step is to drive out the initial Story List and get some more knowledge into the team. The Product Owner prioritizes the Epics and the Business Analyst starts dealing with the top two or three. This means getting some more detail in the form of User Stories, the As a [stakeholder] I want [feature] so that [benefit] format applies here as well as the cards – at least before there are too many. Eventually it always becomes a problem having them on cards, but it is a simple solution that makes it easy to sort, group and prioritize the stories in a collaborative fashion, and that is what is needed at the beginning of a project.

The job of Quality Assurance

Before a story can get into a Sprint Planning, it needs to be estimated and before it can be estimated, the Team needs to understand it and have a clear understanding of when they are done with it – this is where the Scenarios and Quality Assurance come in.

While the Business Analyst works with the Product Owner and the Team to get Epics and Stories in place, the Quality Assurance is responsible for the next evolutionary step – the Scenario. A Scenario takes the form of given [some context] when [I do something] then [this happens], if it is a more complex scenario you could add when [I do another thing] then [this new thing happens] as many times as appropriate. Scenarios are written on the back of the Story cards. If there’s not enough room, chances are there are too many Scenarios and splitting the Story should be considered. Quality Assurance works with the Business Analyst to find Scenarios, gets them verified and refined together with the Product Owner, and communicates them to the Team at estimation.
When development of a Story is done, Quality Assurance is responsible for verifying that all Scenarios are fulfilled and that the agreement with the Product Owner is honored. If automatic acceptance tests are used, it is Quality Assurance together with a developer that makes sure tests are written for each Scenario and that the tests are in good shape.
Having the entire Story in place, you are finally ready to do some estimation. I normally let the team do estimation before the Scenarios are written, simply because I don’t want too much effort going in to the requirements before being sure they will actually be developed and to give the Product Owner some sense of cost to enable planning. This of course means that the estimates might change when the scenarios come into play. In that case you would have to re-estimate or remove some Scenarios to stay within the estimate. The removed Scenarios should be estimated and kept in the Product Backlog to ensure that nothing is lost.

“The other thing was that in telephony”, Cockburn continues, “services are largely additive. A customer owns a simple service, then adds call waiting. The execution of call waiting is that the simple services runs, and then is interrupted by a condition – the call waiting – that the system detects, at which point it goes off and does that behavior, then returns to the simple service. Here again, you can see that the extensions model of use cases makes good sense – it describes interrupting a flow of behavior to inject a new flow of behavior.”

Ivar Jacobsen left use cases alone in the 1970s, and revisited them in the 1980s when he was writing his Ph.D. dissertation. At that point he introduced them for general application development.

“What I had to test for my methodology was whether his form of writing, which works so well for telephony, works well also in application development. It was not clear in 1991 that it would. Over the years, I have found that it works remarkably well for general process descriptions of all sorts, and particularly black-box requirements for interactive software applications.” User stories got their start when Kent Beck was looking for lighter, faster ways to gather feature requests. (Note, he does not like the word “requirements”).
He was working in areas where there were not many users, so he could literally walk down the hall, ask the user a question, walk back and start typing.

“In 1993, I attended a workshop called WOOD (Workshop on Object-Oriented Design)”, says Cockburn. “It was held at the Snowbird ski resort (not accidentally the same place we wrote the agile manifesto – we several times held workshops there). At that workshop, Ivar talked about use cases for almost two days. As we left, I recall thinking that we were in for a rough time, because it was clear that everyone had formed a different impression of what a use case was – and the people in the room were all notable writers: Adele Goldberg, Kenny Rubin, Jim Rumbaugh, Larry Constantine, Kent Beck, Martin Fowler and others.”

Sure enough, at the next WOOD in 1994, Ivar Jacobsen wasn’t present. Among the 17 people present, 14 different definitions for “use case” were found based on memories of the previous year. Martin Fowler offered the quip, “I think use case
is just Swedish for ‘scenario’.” And Kent Beck said, “Just ask the users what they want, and write that on a card.” The cards should have no particular structure – duplication and overlap were allowed. The point was only to find out what the real users wanted. Real diligence would need to be applied.
Kent Beck got to try his card method out in several places, including Wall Street firms and on the Chrysler payroll system. Those are very different environments, and have very different user types – the only thing they have in common is a small user base.  Kent Beck came to call these things “user stories”.

“In strong distinction to use cases, user stories were supposed to be informal ‘reminders to have a conversation’ as opposed to requirements documents, they were to be written on index cards as opposed to being maintained online, they were to reflect what real users requested as opposed to consolidated decisions made by business analysts, and they were not to have any particular structure, as opposed to the structured form that use cases have.”
“And that,” Cockburn concludes, ”started the long wars between use cases and user stories.”

Requirements Management has for a time been a hot topic within the Agile community, and a religious war has seemed imminent between the followers of user stories on one side and use cases on the other. The rebelyell of the user story-buffs have often been “out with the old and in with the new!” – and many have seemingly been against use cases merely because they have RUP written all over them. On the other side we have seen use case devotees in white coats mocking user stories lecturing that this unscientific bumblebee cannot fly.

Lean Magazine has talked to one of the greatest experts on the matter, Alistair Cockburn, who has not only been a leading profile in Requirements Management for twenty years, but also has a moderate view on the pros and cons of the two concepts.

Alistair Cockburn’s engagement in Requirements Management started in 1991. He was working for IBM Research in Zurich and needed a way to return to the United States. IBM had just started the IBM Consulting Group, and was constructing a methodology for its consultants to use.
“At that time it was mostly based on Information Engineering, i.e. dataflow modeling and databases,” Cockburn recalls. “But they needed a track for the emerging Smalltalk and C++ projects
– that was my assignment.”

Noun- or verb-based?

During those days, the object-oriented modeling, design and programming parts were relatively obvious: There was the Booch/Rumbaugh school of OO modeling, there was the Cunningham/ Beck/Wirfs-Brock school of responsibility-driven design, and there were the programming languages Smalltalk and C++.

“However, those left a large gap in the area of requirements. It was not clear to me that requirements should be object-oriented just to follow the design being object-oriented.” Cockburn discovered Ivar Jacobson’s 1987 report on use cases from the OOPSLA conference, and found they had a perfect fit for the information he was missing in the methodology:
“When I say perfect fit, what I mean is that object-oriented models are all noun based – they name the nouns in the domain. Use cases are all verb based – they name the actions being taken by the people and the nouns in the domain. Having all verbs or all nouns leaves a big gap in the model. Having use cases tells us what to accomplish; having objects tells us what will accomplish that. In some sense, it is like having the warp and weft in weaving – some threads run vertically, some run horizontally. You need both. Paying attention to that nounverb issue has been a big help to me in navigating the choppy methodology waters. I still use it to this day.”

Getting the big picture

Today, Cockburn always starts with use cases in order to get the big picture. These use cases can be written quite briefly, perhaps two paragraphs to a half-a-page or at worst a page, so this is generally useful inquiry and not onerous writing.

“Without the big picture I (and others) have no idea what we’re trying to build, its shape, size or complexity. I use the discipline surrounding use-case writing to shine a light into dark and hidden corners, and expose potential trouble spots. This shape then guides what happens next.”
Once the “coarse-grained use cases” are in place, Cockburn is “fairly indifferent” as to whether use cases or user stories are used after that.
“It’s very much a question of how much the people involved can keep in their heads and how good their conversations are. I do like to use the discipline surrounding use cases to check for extension conditions and extension handling, and would apply some version of this discipline in all cases.”
However, use cases have one major limitation with short iterations: It is impractical to implement an entire use case in a single iteration.
“Personally, this doesn’t bother me, because I’m happy to put the use case on the wall and simply color the sentence in the use case being implemented in this iteration. Coloring them like this allows me to implement individual sentences or even phrases from the use case set in any order I like, and to always see what’s done, what’s stacked to be done, what’s in progress, at any given time. In this sense I don’t need user stories at all.”
In Cockburn’s experience few other people are willing to do the above coloring. Instead, they prefer to see separate work items listed on separate cards or as separate line items. For these people, cutting the use cases into user stories is pretty much a necessity.

Use cases have limitations

According to Cockburn, there are two occasions when use cases really don’t serve a useful purpose. The first is for CAD/CAM tool-type systems. In these systems, there is no plot line for the story embedded in the use case. There are only many different tools the user can bring to bear.
“In such cases, I would quickly shift from high-level, coarse-grained use cases to features lists (and I wouldn’t even call them user stories, just ‘features’).”
”The second occasion when use cases don’t serve a real purpose is in bug lists and feature enhancement lists. Related, once the system is fairly far along in implementation, the ongoing requests from users look more and more like feature enhancement lists.”
”Again, I wouldn’t call these user stories except in deference to the implementation group if they are in the habit of saying ‘user story’ instead of ‘feature’ or ‘product backlog item.’ In the case of long bug or feature lists, I would just work from the list.”
In contrast to this, Craig Larman – chief scientist at Valtech – claims that he likes user stories because they scale up to really, really large requests (like “implement the entire system”). In a way, both use cases and user stories could be regarded as fractal. Larman makes use of this fractal nature of user stories to break really big projects into manageable pieces, each time using the “marker” characteristic of user stories to defer detail.
“I should say that this works for Craig because he can keep a lot in his head and is good with his conversations. From my perspective,
if a team can’t keep track of which sentence in a use case they’re implementing on a given iteration, they won’t be able to keep track of the fractal expansion of very large user stories.”

Three essential factors

In Cockburn’s opinion, three things are really important in any project: vision, size and context:

• Vision
“It is crucial that every team member sees approximately the same end goal. This is more important on agile projects because they write down less and leave more to tacit knowledge. Therefore, every project – agile or not, using Scrum or not – should have written down and posted in plain sight a short description of what the system is supposed to do and how it is supposed to help its users and stakeholders. Anything over a page long is too long.
“Ideally, the vision or mission statement captures some emotional feeling to help the developers tell whether they’re on track or not. For my new web site design, we’re using, ‘Get pleasantly lost in all the articles and content.’ Our first user test produced the response, ‘It’s very nice and clean, but I can’t see how to get pleasantly lost in here.’ Because of the vision statement, our viewer knew immediately what to comment on, and we had a direct idea in which direction to make changes. Other ones might be like, “Slide the buyer effortlessly along the conveyor belt to finishing a purchase”, or ‘Ensure the nurse NEVER gives a patient the wrong medication.”

• Size
“Everyone has an idea of the order of the task at hand, whether this is going to be a 10 work-month project or a 250 work-month project. All too often with agile projects, I see a ceiling line that rises faster than the programmers can work. This makes the sponsors nervous and the programmers depressed.
”I’ve written that I like use cases because they reveal the size and shape of the product. If you’re not using use cases, go for some other technique that will hint at the approximate size of the final system, so the sponsors can make appropriate funding and staffing plans.”

• Context
”The programmers, UI designers and testers know how each product backlog item, sprint item or user story fits
in the work or operation of the user. This context helps them make many small design decisions along the way, or even challenge feature requests.”

Use Case & User Stories – a pros & cons approach

Use cases and user stories serve fundamentally different purposes, and so in some sense it makes no sense to compare them:

• The purpose of a use case is to describe the black-box behavior of a system as it interacts with the outside world – not just with the primary user, but also with other systems. A use case is a record of decisions made about the behavior of the system under discussion. Use cases primarily serve the user and business community. These people have long suffered from not knowing what they are going to get. Use cases help with that.Programmers sometimes don’t like use cases because they are user-centric and call for behavior that crosses programmers’ assignments.

• The purpose of a user story is to mark – for later expansion – requests for system functionality. A user story is a token or marker that gets moved, expanded, annotated as the request gets handled. It is not a requirements document, it is not a record of decisions made, and it is not a description of system behavior. It is always just a marker. User stories primarily serve the developer community. These people suffer from having to split up requirements documents into tiny pieces to spread among themselves. User stories can be split small enough to assign to single developers or development groups.

With these two purposes in mind, the pros and cons become quite clear, and are naturally opposing as shown to the right.

USER CASE

- Presents a clear description of system behavior.
- Records outcomes of discussions – Provides discipline for “completeness” and look-ahead, and can be used to get a good handle on the size of the system to be developed and reduce the number of surprises along the way.
- Has a “shape” showing all behavior around a system request– this serves the user and business community well, so that people can more easily see the implications of what they are asking for.

USER STORY

- Short (just a sentence or a phrase).
- Can be used to mark needs outside pure functionality – data enrichment, usability improvement, even internal work items for the team.
- Easy to write on an index card; sets of them can be manipulated in 2D on the wall to indicate priority and spans of behavior across actors quite visibly.
- Can be split into smaller and smaller pieces; a complete user story can fit into any size of iteration.

We all know that the main responsibility of a Product Owner in Scrum is to create, prioritise and groom the Product Backlog based on Business Value. What are the most important part of this responsibility?

Once or twice a month I receive an unsolicited email from someone saying, “Scrum had been working pretty well for us; not great yet, but good. Then we started to pay more attention to our product backlog and, wow, then things really improved.” The importance of having a good product backlog cannot be understated. That’s what drove me to write User Stories Applied for Agile Software Development. And it’s why I insisted to my publisher that the book not be put in the XP series, as was initially planned. User stories are a very broadly applicable technique. Years ago I started to notice that the Scrum teams I worked with that invested time in their product backlogs had an easier time with many other aspects of the project. When I came across the Extreme Programming idea of user stories I instantly wanted to apply that to the Scrum product backlog. So, making sure the team has a good product backlog – even if it’s only roughly prioritized initially – is probably the most important responsibility of the product owner.

When finding a suitable Product Owner, which traditional roles are best suited?

The product owner needs to be someone with a deep understanding of why we’re making the product. In a commercial product setting that meanssomeone with a deep understanding of the market. That means the product owner will usually be someone from the marketing or product management groups. In an in-house environment the product owner usually needs a deep understanding of how the internal users work with the system. In those cases the product owner usually comes from the ranks of the users or perhaps their managers.

What are the most important tools (not SW tools, but rather methods, disciplines, etc.) at hand for the Product Owner?

Product owners need good decisionmaking skills and they need to use them. When a team is moving as quickly as a good Scrum team can move, they need product owners who can be decisive. Product owners also need good leadership skills. Teams look to their product owners for vision and so they need to be able to pass that along in a compelling manner. If we go back to the original article on Scrum, “The New New Product Development Game,” back in 1986 in the Harvard Business Review, the authors wrote that management’s role is limited to providing “guidance, moral support, and money.” Product owners are responsible for much of that – particularly the guidance. Personally, I like product owners to have solid financial analysis skills. I consider that necessary to make good decisions about priorities on the product backlog. In Agile Estimating and Planning I’ve written about non financial techniques that product owners can use to prioritize a product backlog, but I’d still want a product owner to have the basic financial analysis tools at their disposal. These aren’t hard to learn – I’ve taught the skills to many product owners in an afternoon.

The Product Owner is the owner of the Product Backlog and sole responsible of putting priority to the product backlog items. This is usually easier said than done, especially when other stakeholders get involved and have opinions. What good ways can you recommend for Product Owner to deal with different stakeholders?

I like any technique that breaks the problem up into smaller pieces. If you go to a group of stakeholders who have different goals (often motivated by competing financial incentives put in place by the company) you will not get satisfactory agreement if you start with “What’s the most important feature? Then what’s the second-most important
feature?” Instead if you break the problem down into smaller discussions that can work. Start by identifying the handful of most important factors in planning the release – things like contribution to Q2 revenue, features of interest to enterprise customers, and so on. Then for each theme, or collection of user stories you’re considering, ask relative questions like “Which of these two possible themes contributes the most to Q2 revenue?” That’s the type of question where you can typically get people to agree. Then it’s a matter of having a way of looking at the collection of answers you get. There are various ways to do this such as theme screening, theme scoring and relative weighting that I covered in Agile Estimating and Planning. Another approach is analytic hierarchy process but that’s more involved and doesn’t scale particularly well.

How can the Product Owner best involve the Scrum development team in grooming the backlog?

At the start of a significant new effort (not every sprint) hold a story-writing workshop. This is a session where the entire team is invited to join the product owners and ideally some us-ers or customers in writing the product backlog. This starts with the product owner describing her vision for the next few months to the team. In a semi-structured manner they then all write the user stories that will be considered. Starting out like this engages the team right from the start. Following that up by making sure that user stories on the product backlog are estimated so that they can be prioritized keeps the backlog visible to the team. Then occasionally work with assorted team members to split medium and large items into smaller ones as they move up the backlog and then need to be small enough to complete in one sprint.