Demystifying a Good Agile Product Backlog

August 30th, 2021 by inflectra

by Dr. Sriram Rajagopalan, Enterprise Agile Evangelist

Recently, I had an opportunity to provide agile transformation consulting as part of product training for a major organization. A member of this team asked who is responsible for creating user stories. As I used pointed questions to understand the rationale behind the question, two different problems surfaced: a) the team felt they only work on tasks that the Scrum Master assigned them for the user stories the product owner thought valuable; b) the development team had to work additional hours for any technical infrastructure or automation. These findings offered an opening for a major coaching moment.

Nevertheless, I asked why the developers can’t add their technical requirements and specification to the product backlog to promote transparency to the product owner and scrum master. One of the members questioned, “Can we all add access to product backlog? I thought it was product owner’s playground.” Another member reasoned, “We have technical requirements, and the backlog is supposed to be full of user stories!” If the team is unclear about the fundamental empirical pillar of transparency to the product backlog and does not understand how value is a function of the entire team’s contribution, are we truly agile to begin with!? We seem to divide the team with roles and confuse ourselves with terms rather than rally the teams around value to customers and business. This is why I claim that “Agile is a Teenager now, but agility is still a toddler!”

In the spirit of being agile, I would like to focus on one important objective? A good product backlog! In this article, let me unpack some of my thoughts on the need for a product backlog!

Product Backlog is like the Waterhole

Most of you have seen the National Geography or other programs talking about various animals in the jungles and forests. Each animal may have its own food preferences or live in various sections of the forests. Some live on the trees; some roam the grasslands, some hunt to survive, and some feed on the leftover. Regardless of the vast differences, they all quench their thirst for water at the waterhole.


Figure 1: Product Backlog is synonymous with Waterhole

When I look at the product backlog, I think of the waterhole. Every stakeholder may have specific responsibilities. Some may be senior members of the management team evaluating which programs to start based on company strategies. Some may be the middle management focusing on aligning these company strategies with the product roadmap, technology stack to sustain mergers and acquisition, regulatory compliance, and ongoing operations. Some may be the core teams focusing on prioritizing the work based on bandwidth towards alignment and delivering on commitments to produce value. Like the waterhole brings all animals together towards the life-source - water, the product backlog brings every stakeholder together towards alignment. Just like the waterhole is open and noticeable, and transparent.

Product Backlog Promotes Stakeholder Alignment

One of the critical aspects of transparency is to align the stakeholders and team members towards the valuable functionality to be delivered. What produces value is not the product owner’s responsibility alone. The product owner is accountable for prioritizing the value stream mapping, aligning the product strategy in the roadmap to the needs and epics in the releases and requirements and stories in the iterations. This value stream mapping should not only be from the customer’s point of view but also from the business to sustain itself in delivering incremental, recurrent, and consolidated value.

The alignment, therefore, integrates the customers, the business objectives, the information architecture underlying the product, and the processes that continually remove overheads, augments efficiency, and promotes effectiveness. Therefore, alignment in the product backlog is analogous to a creative artist’s work. While the ultimate work-product, painting, is the final end value for the customer, the painter also has to be supplied with the tools enabling the painter to sketch and the processes required to give the time and space for the painter to experiment. If you remove these painter’s toolset and the time and space needed for the painter, can the painter deliver the painting?

 

Figure 2: Product Backlog supports various Stakeholders

But, which paints and canvas are required for the painter? When are they needed? What are the alternatives that can work? The answer to these questions is not always obtained from the manager who manages the initiatives but from the painters themselves? And then the other stakeholders responsible for marketing and promotion, event organizers, and many other stakeholder groups. Their requirements have to be solicited and broken down to needs, expectations and wants to facilitate prioritization.

However, in a rush to deliver value quickly, if the working agreements, necessary tools, and required processes for the team to deliver value are not focused on, a product backlog may have many requests from selected stakeholders making the backlog less valuable. Furthermore, such a backlog cannot facilitate the categorization of functionality for subsequent prioritization of value. This alignment, therefore, requires inspection and adaption, two additional empirical pillars to transparency, from the entire team. As a result, the product owner does not have to write all the requirements, but everyone in the team should contribute to balancing the types of stories.

Characteristics of Good Product Backlog

Now that we have understood who can interface with a product backlog, let us review its characteristics. It is important to reemphasize that a backlog continuously supports transparency, inspection, and adaptation. It is also important to disassociate from incorrect views of the backlog due to the anti-patterns like how a specific company implements agile in their organization by discussing specific roles like the business analyst or technical product owner or having one person function as both product owner as scrum master, or a traveling product owner unavailable for the team, etc.

Accommodate different types of requirements

Once a senior member of an agile team mentioned to me, “Well, we now write user stories. So, I guess that makes us agile!” I am sure agile transformation consultants would disagree. Agile teams frequently get hung up on the terms “user stories” and this is primarily because the work they commit to doing in a sprint leverages user story. However, since different stakeholders may add to a product backlog, their ideas must be categorized into higher-level requirements, disaggregated specific features, and refined as user stories. So, one of the first requirements of a good backlog is that it should allow different requirement types.

Figure 3: Support for Requirement Types in Spira

 

In the example illustrated above in the high-level requirements for implementing a hospital surveillance system, some uses cases may emerge from various stakeholders. Some of these may be broken down by the stakeholder groups as features and the team by specific user stories. Alternatively, some essential non-negotiables may be identified as large epics that may have design compliance needs, architectural design considerations, or quality considerations, as noted. Each one of these requirements can be refined to come up with granular user stories.

Allows Product and Business Categorization

When multiple stakeholders interact, there will not be a shortage of how categorization might be applied. For example, at one extreme, the business may see value in terms of features delivered to customers that support sales and marketing for product penetration in the customer’s organization and market diversification with new industries. At the other extreme, the team working on the product may view the work in terms of the functionality, such as security, portability, reliability, or user interface improvements, delivered to the product.  Any requirement or user story thus developed will have to, therefore, at minimum, support such dual categorization.

Figure 4: Product and Business Categorization

In Figure 4 illustrated, we can see how the various stakeholder requirements on the left of the particular requirement type in the middle are being refined towards the team to potentially work on. Furthermore, the product owner also can balance the requirements around product strategy on what specific areas to focus on. For instance, if the stakeholders agree on Monitoring as the primary value for customers followed by training considerations for security personnel and staff, then such product component categorization allows product functionality prioritization bringing all stakeholders to alignment. As a result, even though policies and procedures related work is important, the stakeholders understand from this categorization that their needs have to be deferred.

Allows Priority Scheme

An important characteristic of the items in the backlog is to enable the product owner to prioritize these items to maximize the value to the customer, simultaneously balancing the team’s expectation of the value-enabling stories (business, technical, and process). Therefore, the backlog must make it very clear what is the most valuable item while also allowing the team members to slice and dice the backlog and filter and query the backlog without altering the priority of the backlog item. The priority scheme itself should be flexible to allow balancing team’s commitment to set them up for success.

Figure 5: Spira’s Support for Importance Schemes

While backlog should give priority schemes, the backlog should also enable the teams to customize the priority schemes based on their domain. For instance, projects related to construction, manufacturing, and digital waste disposal require about 10 levels of prioritization. The easier the backlog promotes these customizations, the more meaningful the backlog becomes for the various stakeholders that benefit from the product features and the teams that deliver these functionalities.

Facilitates the Definition of Ready

One of the critical elements of engaging stakeholders is to constantly lobby them for support. While transparency promotes visibility to work, it does not confirm accountability. As a result, when the team implements a backlog item, it raises questions from stakeholders. Particularly, in regulated industries where the work item has to be evaluated by compliance team, audit personnel, and medical, legal, and regulatory liaison members, facilitating traceable and auditable discussions through the system of record for gathering clarity as part of the “Definition of Ready” before an item is split into appropriate user stories, prioritized and implemented by the development team makes the backlog an inspiring center of excellence.

Figure 6: Confirming Clarity of Stakeholders validating Definition of Ready

As you can see in Figure 6, product owners or technical architects who are accountable for delivering certain features can continue to consult with the appropriate stakeholders seeking validation of requirements. Such traceability of conversation maintains not only validation of definition of ready but also the increased confidence for the team to ensure that appropriate consult is recorded before they begin to implement the work. After all, the last thing a product backlog should promote is a rework because the proper personnel was not consulted before work was performed.

Incorporates activity breakdown

As part of creating the product increment, one of the iteration planning activities is to create tasks that have to be completed so that the related requirement can be completed. When such tasks are created, the traceability to the requirement should not be lost. When tasks are not related to any of the requirements themselves but still have to be completed within the release or iteration, the backlog should still make such enable a seamless way to track these tasks as well.

Figure 7: Breaking Requirements into Tasks for Team’s Granular Progress Tracking

 

Promote multiple estimation approaches

The traditional approaches to project delivery use higher-level estimates (analogous or parametric estimates) for higher deliverables and detailed hourly estimates at the individual activity level. Similarly, many agile practitioners recommend that the backlog items follow the affinity estimation for requirements in the backlog and hourly estimates at the individual task level. When tasks and requirements coexist in the backlog, the backlog itself should allow multiple estimation schemes to also coexist.

Figure 8: Support for Points at Backlog Item and Hours for Task level

In the above Figure 8 illustrated here, you can see the requirement at the backlog level is estimated at a point level. However, when the task to implement that work item is started as part of the accepted workflow, the task supports hours for both estimation and progress tracking.

Summary

As you can see from the above illustrations listing down the critical characteristics of a product backlog, it brings various types of stakeholders together by promoting transparency, inspection, and adaptation. The customers do not care whether the teams work in agile or waterfall approaches to deliver the value for them or not! But the customers do care about how frequently the teams deliver quality work as committed. As a result, all internal and external stakeholders benefit from transparency, inspection, and adaptation. A good backlog, therefore, facilitates not only backlog refinement at the product backlog level but also progress to work commitments at the iteration backlog level by demonstrating test coverage and task progress as demonstrated in Figure 9 (for a different project to show the color-coded progress)

Just like all the animals come together to quench their thirst at the waterhole, a good product backlog allows such visibility on test coverage and task progress on committed work as well as the stakeholders’ commitment to the product strategy at the product backlog item for the requirement validation for proper prioritization and delivery.

In the next part of this blog, I will discuss the agile ceremonies and outline how to use the planning board.

scrum agile product backlog backlog items user stories