An Idea in An Agile World

All new products and new product features start with an idea. This idea becomes the first user story, user story #1, for the product. User story #1 begins a journey that could end with either happy customers and happy business stakeholders or end up in the bucket of disused ideas. So what happens to user story #1 and what do you do with this new idea?

You can find lots of articles on breaking down user stories. You can also find articles on how to write good user stories but you usually don’t see too many articles on how the first user story is used to develop a business case. What follows are some thoughts and insights on how to take user story #1 and developing it for greater success.

User story #1

User story #1 is the idea. It’s an idea to achieve business goals, an idea to solve customer problems, and an idea to create or change a product. We’re all familiar with user stories written with an eye toward solving a customer problem through product updates. User story #1 is different as it’s not so much about solving a customer problem as it is a ‘thought’ experiment to discover what customer problems we can solve. So the question is, how should we go about validating the idea?

Validating

When the new product idea is first presented as a means to achieve a business goal, there will be interest but it’s unlikely the business is going to throw monies your way, at least not at first. The business needs more than just the idea, the business needs data to back-up and justify the idea. This translates to meaning the idea is put through the discipline of Ideation or Envisioning.

The idea is the highest level user story for the product but the picture is incomplete and outcomes uncertain. However, there’s usually enough to the idea to begin analysis and experimentation. The analysis often results in a series of metrics and measures collected during envisioning. These are often measured on a some scale that best serves your business. For example, aligning to company strategy might be scored 0=no alignment, to 5=full alignment. Costs might be 0=<$10,000 to 5=>$450,000. Below are some metrics you might be familiar with.

  • Alignment with the company’s strategy,
  • Moves the product team towards achieving a business goal,
  • Is technically feasible,
  • Product innovation category (new, adjacent, or sustaining),
  • Cost associated with it (implementation is usually the biggest component),
  • Predictable business outcome if implemented (revenue, customers, users, …),
  • The business risk of implementing/not implementing is calculated,
  • The competitive advantage, and
  • The market breadth (new, adjacent, or sustaining).

To do the above takes time and you might look at the list and think, “that doesn’t look too agile to me”. Today’s reality in the digital age is all these items need to be completed in the morning so the development teams can implement and deploy by late afternoon. For most of us this isn’t exactly true, yet. But time is a pressing consideration and being quick is important. What we need to find is a way to do business so we have both enough knowledge to start and we have a clear idea of what further information we’ll need. The key bit of knowledge we need to have up front is that the idea aligns with the business’s strategy and goals.

Big ideas need to align with business strategy & goals

Before the idea is broken down, or before any real effort is made, the idea should be validated against the business strategy and goals. You might be lucky and the business strategy is developed in such a way that the product manager can have their ideas incorporated into the strategy from the start. When the business defines what their goals are, they develop a strategy to achieve those goals.

The three elements of good strategy (source: Richard P. Rumelt, Good Strategy Bad Strategy)

  1. Diagnosis: “A diagnosis that defines or explains the nature of the challenge. A good diagnosis simplifies the often overwhelming complexity of reality by identifying certain aspects of the situation as critical.”
  2. Guiding Policy: “A guiding policy for dealing with the challenge. This is an overall approach chosen to cope with or overcome the obstacles identified in the diagnosis.”
  3. Coherent Actions: “A set of coherent actions that are designed to carry out the guiding policy. These are steps that are coordinated with one another to work together in accomplishing the guiding policy.”

Working backwards on this list, new product ideas describe the actions needed to deal with the challenges (goals) found during the diagnostics. In this way, new product ideas become an integral part of the company’s overall strategy.

If the product manager can align their ideas with the business, then the remaining items of envisioning can happen relatively quickly. It’s very likely that for most ideas, the effort to define feasibility and cost can occur within an agile team’s iteration. This is a tremendous way to cut the time from idea to completing envisioning. It also provides for giving the customers a chance to evaluate the idea in a practical sense: A/B tests, user testing, or customer interviews. This technique of learning is often used in the Startup world as Build-Measure-Learn cycle.

The product manager will likely obtain full approval for any opportunity that delivers overwhelming value relative to its cost. Kenneth Rubin in his book, “Essential Scrum: A Practical Guide to the Most Popular Agile Process“, suggests that if there’s any discussion over tiny differences in cost or value then it clearly doesn’t deliver ‘overwhelming’ value and the idea should be dropped.

The easiest and best way to determine value is by getting the idea in front of your customers and users and measuring their response.

Using the development team for envisioning

When speed is essential to capitalize on an event or competitive opportunity, get something out in front of the users today rather than waiting until you’re sure. The trick will be to do only enough to answer questions and reduce risks without over committing the team or business. Keep in mind that you and the business haven’t validated the idea yet so there is a risk. If you’re doing Agile then this will fairly straight forward: use build-measure-learn techniques to understand the value of your idea with customers and users.

The product manager works with the product owner, team leads, architects, or full development team to break down user story #1 (the idea) into smaller user stories. These are the fragments needed to showcase the new product or feature idea. These particular stories are not necessarily work to build a product but will often be part of a prototype or demo system to gather information and feedback. Part of the envisioning process is to validate the idea, confirming its potential, with customers and users. To do this, a demo or prototype to showcase new features to potential customers and users can be an enormous asset without spending too much money in the process.

If the product manager has the luxury of dedicated team at their disposal that’s great! However, a new product idea may not as yet have a team but only a handful of senior people to help bring a new idea to life. Assuming there is a team, one approach is to get a team to include some research work along with their normal complement of product work. This is handled as spikes, technical or business exploration, added to the sprint if doing scrum. (If the agile teams are Kanban then there’s less of a concern with disruption and immediate priorities can take precedence.) If there’s a time critical component to the idea and waiting isn’t an option, then cancelling a sprint and pivoting to take on higher value work is an option but not always a good idea. Besides, if doing scrum there’s already a mechanism in place to do research, prototypes, and mock-ups: the backlog refinement sessions.

Using the development team during envisioning helps the product manager answer some important business questions including these 2 metrics from above:

  • technically feasibility-team can better answer this after developing a prototype,
  • implementation costs-implementation is usually the main influence on costs and the team can make a more accurate estimate.

By using the feedback and learnings from the demo and prototype, the product manager can better determine the viability of the idea. Using A/B tests, user testing, or customer interviews, the product manager can:

  • better estimate the business outcome if implemented (revenue, customers, users, …),
  • better assess the business risk of implementing/not implementing the idea,
  • better assess the effect on competition, and
  • better understand the market breadth.

Getting the development team involved with the idea early on will benefit the product manager. This early involvement moves the team from being a partner with the product management into being full members of the business team, bringing along their unique skills and insights into the envisioning process.

Two key outcomes having the development team in on envisioning

One big advantage is the development team is better positioned to undertake the new work if the business decides to proceed. The development team also has more ‘skin in the game’ through early involvement.

Another big advantage is the product manager can determine sooner whether to pursue the idea further. Because the development team has created prototypes or demos for customers, the product manager is better able to collect critical information for the business case. If it turns out there is no profitable business case, the product manager has spent relatively little money to learn a lot. The agile principle of maximizing the amount of work not done is an important one and finding out early to stop is better than running out the budget on a hunch that doesn’t pay off.

In a waterfall environment?

If you’re doing a waterfall type methodology then it’s a given that all envisioning and requirements are defined upfront. This is not bad if there’s little to no uncertainty. However, the early involvement by the full development team to pursue the idea using prototypes is still valid. They’ll not only be putting more ‘skin in the game’ but they’ll improve the accuracy of the estimates and potentially uncover more unknowns.

Summary

To increase your chances of success, get the development team involved during envisioning. They can help the business best if they’re a full member of the business team and not just a partner. Use the full development team to validate user story #1. This includes validating customer problems, potential solutions, development costs, and business value.

 

Being Agile When The Business Isn’t – Communications

I recently met a group of people having a hard time ‘doing’ agile. They understand agile but others in the business chose not to participate. These were managers and business leaders. Although the group’s first reaction is to ‘sell’ agile I gave them an alternative of doing things to add value both for the business and for customers without a sales pitch. What if the team communicated a better way of doing work that created more success? Wouldn’t the business embrace better ways of being successful?

Let’s start by establishing that A) No one goes to work hoping to fail and B) Business leaders and managers often don’t see how changing themselves can improve the team’s outcomes.

Although people don’t want to fail, we can’t assume they’ll do their utmost to be successful. Being successful takes effort but this is something that can be leveraged for your advantage. You’re not going to push agile but rather advocate a means to be more successful. Your communications with the team and business leaders are about being successful and about ways to be more successful.

When you communicate with managers and senior stakeholders in the business, they’ll most often listen to you but more importantly, you’ll need to listen to them. These people want something from the teams and your best approach is to understand their real needs and not just their ‘wants’. Although the business usually measures output as a key indicator, it’s been my experience they need reassurance that progress is being made, promised business value is being delivered, and the team is meeting their delivery commitments. Often times however, management are unaware of what they need to do to help the teams be more successful and it is this particular behavior that teams need to communicate.

Simply put, these communications have three parts: 1) teams to understand the needs of the business and what makes the business successful, 2) business leaders need to understand their role in helping teams (and themselves) be successful, and 3) the teams need to find the means to meet the business’s expectation and let the business know the ‘actuals’. Sounds easy, yes?

Inception workshop planning

The best way I’ve seen to get everyone started on the same page and establishing desired communication patterns is using an Inception Workshop. You can use the inception workshop to fully communicate intentions, expectations, and create an initial delivery plan. Although an inception workshop can have many topics and areas of exploration, I would focus particularly hard on the following topics:

  1. Why we are here – the reason for this activity and our outcomes for a successful workshop
  2. Why is this important – the business goals and objectives we want to satisfy by means of solving customer problems
  3. Product Vision – developing a shared product vision that inspires everyone to achieve the business goals using the product
  4. Stakeholders Who / do – the key stakeholders both internal and external and the actions required for a successful project
  5. Empathy Mapping – a deep dive into the users, customers, or stakeholders
  6. Journey Mapping – how users, customers, or stakeholders will use our product or service (
  7. Trade off sliders – the team’s upfront assessment of what’s important and what will or will not be compromised to achieve total success
  8. Epics / Story Cards / Relative order – initial story mapping to discover or confirm the highest priority requirements/user stories
  9. Estimates / Releases – continuing with story mapping to develop a release plan. The team(s) commit to this release plan acknowledging that some information might be incomplete or unknown.

These nine steps are about reaching a common understanding of who benefits from the project and what problems need to be solved to achieve our business goals. However, more important than these two outcomes is the openness and transparency of the communications between the business and team(s). Planning your business’s next project or endeavor using an inception workshop sets the precedent of business teams and creative teams working together as one in an open communicative environment. Now the real trick is to somehow institutionalize this new-found communication pattern into the organization.

Company culture

I’ve worked in organizations where managers were expected to make decisions and be seen as leading their teams. The reality is this can be changed but it does require a lot of openness and introspection (see The Journey to Servant Leadership as an example).  To keep the open and collaborative feelings discovered during the inception workshop going, the senior business managers and middle managers might need to change the way they work. The key to business success is always knowing where you’re at. This is what needs to be continually communicated upwards, from the teams to the business managers and stakeholders. This part is generally easier to achieve than getting business managers and stakeholders to communicate their current situation and results back to the teams. Why should this be difficult?

One of the reasons a company’s culture is not opening itself up to change is the breakdown in communications. This is evidenced in high level managers aggressively stating how things cannot change and giving ‘reasons’ why this is so. Often when this happens, those wishing for change back down and accept the status quo. The following illustrates people taking positions on an issue rather than communicating.

  • Team lead to Business Manager: Can you tell me how the business is doing with our latest delivery?
  • Business Manager to Team lead: Customers seemed to be happy with the new features.
  • Team lead: Can I get their specific comments and the overall NPS?
  • Business Manager: No, sorry. That’s confidential information.

With that little dialog it’s easy to see how the company has limited access to information. The painful reality is the teams must be completely transparent to the business but the business often has policies that limit its transparency back to the teams. This is where the culture shift needs to occur. If the team is to feel fully engaged with the business, the business must fully engage with the team.

One way to overcome these obstacles is to change the dialog. The dialog above has the team lead ‘telling’ the manager what they need to do. The manager responds with a shallow summary but hides behind policy when it comes to any meaningful depth. The above dialog is just that: a dialog. It doesn’t fall into the collaborative conversation category because clearly there is no collaborative effort made by either party. Culture trumps communication.

A different approach would be for the team lead to consider the business value of their request. For example, take a look at this illustration from Michael Sahota of agilitrix.com.

What the illustration shows is a variance of culture between the team solving problems and the business people running the company. It is hard to change either of these cultures so don’t try. As I’ve talked about in previous posts, the best way forward is to work within the business’s culture and make a compelling business case for opening a previous inaccessible communication path. Here’s how that communication might go down:

  • Team lead: We’ve been getting some customer feedback at our reviews but we are having some difficulty mapping our value delivered with the actual progress in achieving two business goals: increasing NPS and increasing customer referrals. If we can get better and more specific insight on what customers are saying and doing, we can make sure we’re working those requirements that can deliver the most business value sooner.
  • Business Manager: How can you use this customer information to achieve our business goals sooner?
  • Team lead: Once we can get their specific comments and the changed NPS, we can analyse what is motivating customers to use our product. Once we have that we can groom our stories to get those stories which enhance features customers are talking about to the top of the backlog.
  • Business Manager: Ok. It’s confidential information, I can’t just give it to you but I tell you what, let’s get all the team leads together and we can review the raw data. Will that help?
  • Team lead: Absolutely. I’ll get the team leads together and we can meet here in your office at say, 2:00 pm?
  • Business Manager: 2:00 pm is good. It should take about an hour and if we need more time we can find it later.
  • Team lead: Great! see you at 2 and thanks a lot.
  • Business Manager: You’re welcome. See you all at 2.

The difference with our second example compared to the first is the second is clearly a collaborative conversation and one that resulted in potentially creating more business value sooner. The team lead has changed the conversation from one that is a request from the manager to one that is offering to serve the manager.

Summary

Communications between the business and the team needs to be collaborative and mutual. Using an inception workshop is a great way to start any project or adventure with an open, collaborative setting.

It’s also important to understand that the cultural bubble around the teams and the organizational culture are probably different. Any ongoing communication needs to account for this. When talking across cultural lines, use language and terms that are meaningful in the ‘other’ culture. In our team lead example above, the team lead used the language of better achieving business goals to break down communication barriers.

When will my agile team self-organize?

Great! Your business has decided to move from a Waterfall paradigm to Agile Scrum and now you’re waiting patiently for the benefits to kick in. You’re hoping your Scrum teams start self-organizing soon and are inventively solving customer problems faster with better efficiently. You’re also waiting for the team to become major contributors to the business decision process based on their intimate knowledge of customers and their pains. And you’re waiting … Still waiting …

You ask yourself what’s holding the team back. The Scrum Master has trained the development team on the agile manifesto including the principle:

“The best architectures, requirements, and designs emerge from self-organizing teams.”

The Scrum Master has also trained the development team on self-organizing as described in the Scrum Guide:

  • defining the Scrum Team: “The Scrum Team consists of a Product Owner, the Development Team, and a Scrum Master. Scrum Teams are self-organizing and cross-functional. Self-organizing teams choose how best to accomplish their work, rather than being directed by others outside the team. … The team model in Scrum is designed to optimize flexibility, creativity, and productivity.”
  • defining the Development Team role: “Development Teams are structured and empowered by the organization to organize and manage their own work. The resulting synergy optimizes the Development Team’s overall efficiency and effectiveness.”
  • defining the Scrum Master role: “The Scrum Master serves the Development Team in several ways, including: Coaching the Development Team in self-organization and cross-functionality.”
  • defining the Sprint Planning Meeting: “The Development Team self-organizes to undertake the work in the Sprint Backlog, both during the Sprint Planning Meeting and as needed throughout the Sprint…By the end of the Sprint Planning meeting, the Development Team should be able to explain to the Product Owner and Scrum Master how it intends to work as a self-organizing team to accomplish the Sprint Goal and create the anticipated Increment.”
  • defining the Daily Scrum: “Every day, the Development Team should be able to explain to the Product Owner and Scrum Master how it intends to work together as a self-organizing team to accomplish the goal and create the anticipated increment in the remainder of the Sprint.”

It’s clear the Scrum Guide expects the Scrum team, and more specifically, the development team, be a self-organizing body. It’s also clear that training people that they should be self-organizing isn’t always enough. The organization must want and the businesses actively support the team becoming self-organizing.

Setting Expectations

Getting a Scrum team to self-organizing is hard primarily due to the fuzzy nature of what ‘self-organizing’ actually means. If you were to ask 5 people to define what a ‘self-organizing’ software team is you’ll probably get 5 different answers. But even in that result some truths are seen, it’s not how I define self-organization, it’s how the team defines it.

In Nitin Mittal’s Scrum Alliance article, “Self-Organizing Teams: What and How“, he describes 5 essentials of self-organizing teams:

  • Competency: Individuals need to be competent for the job at hand. This will result in confidence in their work and will eliminate the need for direction from above.
  • Collaboration: They should work as a team rather than as a group of individuals. Teamwork is encouraged.
  • Motivation: Team motivation is the key to success. Team members should be focused and interested in their work.
  • Trust and respect: Team members trust and respect each other. They believe in collective code ownership and are ready to go the extra mile to help each other resolve issues.
  • Continuity: The team should be together for a reasonable duration; changing its composition every now and then doesn’t help. Continuity is essential for the team.

I was once Scrum Master to a team during which I did training on the advantages of setting priorities, of sharing work, and of getting stuff done. The team was very good at their jobs and helped each other out whenever asked. However, I didn’t realize helping out had a limit until one day at the daily standup when the person working the most important story, the highest priority story, was out. I asked who could pick up the work to move it forward? No one raised their hand. One of the quietest people then said it was massively inefficient to have someone else pick up the work. For me personally this was a sad, sad day. After talking priorities, talking of sharing work, talking of getting stuff done, and talking how the team best works together, the team couldn’t let go of their own personal task responsibilities. The team couldn’t find it in themselves to embrace a team responsibility of getting the most important work done first. A self-organizing team would have kicked into a self-managing mode and re-allocated work so the highest valued work could continue. At the end of the standup, team members felt a stronger bond to their own work in areas they were most comfortable with than to the whole. It wasn’t competency but rather motivation that was holding the team back.

What makes it so hard to motivate a team to feel responsible for the whole? When a business pushes individual roles & responsibilities over team commitments or when survival and blame permeates an environment, it’s very unlikely that the team will move beyond the ‘forming’ stage (team members sticking to what they know best individually and doing what they feel most comfortable with).

In some business environments, the dip in productivity while transitioning from ‘Forming’ to ‘Storming’ is unacceptable. This would be especially true in a strong hierarchical or Command & Control environment where those in the hierarchy are held responsible (accountable) for team performance.  If the business is clinging too tightly to a hierarchical and Command & Control environment, the likelihood of birthing self-organizing teams is diminished. At one company I was at, one of the most senior software engineers was a team lead. The team was always seeking his approval for any decisions and if the manager thought otherwise, the team changed their ideas to conform. This wasn’t the worse thing to happen but it was heavily reliant on the team lead always being right. A consequence of this was lower scores on the company’s engagement evaluations especially around empowerment and feelings of trust.

It’s worth the attempt

Adopting Agile and Scrum will not automatically cause people or businesses to change. For Agile and Scrum to make a positive impact, people will need to see that they need to change their own behavior. It’s possible to see that in an environment where individual knowledge or individual skills are the most sought after commodity, Agile may not be the right tool. In a business where rigid hierarchies and Command & Control are dominate, it may be a long journey to establishing self-organizing teams but certainly worth the effort.

David Ticoll in his Harvard Business Review article, “Get Self-Organized”, states that, “It would be difficult and risky—even foolhardy—to try to wholly transform a hierarchical business model into a self-organizing one. But the potential of self-organizing systems to enhance competitiveness is becoming clear to managers of some conventionally structured businesses.”

David Tricoll further states, “Today, the ability to stream complex, real-time information to the front line gives hierarchical companies greater power than ever to exploit self-organization.”

One approach might be getting the team to participate in business events such as creating business plans, working with customers and users, and respecting them as contributors to business success. Teams with greater insights to business and customer problems are more likely to be engaged with solving these. To be quick and competitive, businesses must relinquish control to the teams in the front lines and the teams must be willing to accept the responsibility. When the business leaders and development teams shared these insights, I’ve seen the development teams rise beyond their own specialties to rally around both the customer and the business.

For a team to become self-organizing, it will take more than a business to want and support it, it requires the team itself to see advantages in self-organization. If you’ve ever seen how an assembly line worked in days gone by, you’ll see an environment where self-organization would have been detrimental. Your job was to put nut ‘A’ onto bolt ‘B’ period. Repeat this for 8 hours a day, 5 days a week and you got paid pretty well for what to some people would be the most boring job on earth. The business wasn’t holding out hope that the workers would self-organize and the workers had no motivation to do so. In the factories, piecework was common. Workers were told what to do, how to do it, and when to do it. Software development is not piecework, if it were, robots would be doing it today. It is knowledge work where thinking both inside and outside the box are assets to the business.

In the situation above when a team member is absent, the moment the remaining team members decided that doing and completing individual tasks is more important, all advantages of self-organization were lost. Instead of adding value for the customer and business, the team had elected to follow a plan. The decision essentially comes down to: I have my own work to do and don’t have time to do yours.

To some, this kind of dedication is how you advance in the company. However, if the company were to emphasize customer and business value of work being done, it would cause a fundamental shift in team thinking, from doing tasks to getting the highest value work done quickly. When the business puts emphasis on value, the team is more likely to devote their combined efforts to achieving value. It would come to pass that even when one of their numbers is absent, progress won’t stop, although it may slow down. If the team is accepted as a fully participating business team member, not just a partner, the team will genuinely feel the success of the business is also their own success and act accordingly.

To me, one of the principle tenets of Agile is the team. The great power of Agile comes from a group of intelligent people working as one. Collectively they are more knowledgeable than any one individual although the shared experience may be less. To ignite the team requires the business to openly give the team responsibility and accountability for the outcomes of each iteration. For the business to move forward and empowering the team, they would need to establish safe boundaries from which the team can safely operate in.

When business leaders, product owners, and scrum masters attend the daily standup, what’s more important than hearing team members say,

I completed task <x> yesterday”,

is hearing them say,

we added <some> value to the product yesterday”.

The role of managers and customers on the self-organizing team

In an InfoQ interview, Rashina Hoda cites two environmental factors needed to be in place to enable self-organization to emerge. These are:

  1. Senior management at the teams’ organization must be able to provide freedom to the teams so that they can self-organize themselves.
  2. Customers must support the teams by being actively involved in the development process through providing regular requirements, clarifications, and feedback as required.

“Self-organizing Agile teams … require organization structures that are informal in practice, where the boundaries of hierarchy do not prohibit free flow of information and feedback. In an informal organizational structure, the senior management is directly accessible by all employees (maintaining an ‘opendoors’ policy), and accepts feedback—both positive and negative.”

– from “Supporting Self-Organizing Agile Teams What’s Senior Management Got To Do With It?” by Rashina Hoda, James Noble, and Stuart Marshall

Summary

For the business leaders, giving development teams insights to both business and customers plus giving development teams space to operate, will motivate teams to accomplish amazing things. When the business states these are our goals and asks the team to solve it in the best manner possible, you’re likely to see a team self-organize to meet the goals. No one goes to work hoping they fail.

When the business has a strong hierarchical and Command & Control environment, moving a team toward self-organizing is made more difficult but not impossible. Building a bridge of trust and respect is essential. By establishing a strong partnership between the business and development team, followed by full membership, the team can exercise their collective knowledge and capabilities to contribute to business successes.

In the end, it’s a mutual effort on the part of both management and the development team to become self-organized. It will probably be a bumpy ride along the way but with a clear goal as a guide, both should arrive at the destination together.