The Goal Is Not Agile

You hear about and read about agile transformations, like this article about ANZ

“We need to break with some of the traditional 20th century approaches to organising and working to ensure we are more responsive to 21st century customer expectations.

“Moving to implement the agile approach at scale in our business is an important evolution in how we run ANZ which will allow us to respond much more quickly to customer needs, create higher staff engagement and make further improvements in efficiency” – ANZ CEO, Shayne Elliott

But what does agile transformation really mean? From the article, the goal of ANZ’s agile transformation is to specifically target customers which is a hopeful sign. If we look at the agile principles we see customer focus is essential:

  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  • Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
  • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

ANZ seems to be looking to improve employee engagement. This can be achieved by recognizing the agile team as equal business team members. They bring a different talent set to the table but they’ll be closer to the customer problem and potential solution than anyone else.

And lastly, ANZ describes their goal to be more efficient. This doesn’t just mean doubling output but doubling the value of each outcome.

ANZ has announced they will use agile as the tool to achieve the objectives. One can interpret from the article that the goal is not to be agile but to use an agile approach to improve and be better. Easy, right?

Although there are several aspects of an agile transformation, I’ll mainly look at it from the agile team’s perspective and how they can effectively change their working environment to embrace the agile culture.

Agile culture

Agile is a mindset or cultural phenomenon that is hard to quickly adopt when the culture is Hierarchical and Command & Control. The ANZ article says, “The transformation program will … shift to a less hierarchical organisation, relying on small, autonomous teams.” I’ve seen companies who take a different road in adopting agile practices to improve. They adopt agile without having a ‘agile transformation’, instead they test and experiment (see this blog post) with new ways of working and discover agile ways worked better than most. The common activity whether doing large-scale ‘agile transformation’ or using a gradual adoption is experimentation.

In the book, “Blue Ocean Strategy,” W. Chan Kim and Renee Mauborgne cite four hurdles a company will face trying to institute broad change:

  1. Cognitive – people must understand why the change in strategy and culture is needed
  2. Resources – changing an organization will require shifting resources
  3. Motivation – workers have to want to make the change
  4. Institutionalized politics – overcoming the way we do things around here

So where does agile fit into this picture? If your organization is doing waterfall or is rigidly hierarchical in nature or is Command & Control, all four hurdles will be a challenge. Agile as a tool won’t solve any of these challenges, in fact, it’s the opposite, these four hurdles will need to be overcome for agile to work best.

How agile transformation happens

Let’s assume for the moment that the organization is a typical C&C waterfall environment. Businesses can be successful with waterfall but that success can have limitations. These limitations primarily center around taking advantage of opportunities requiring speed and the ability to fine tune the product as you go through continuous customer engagement. When we choose to do agile it isn’t because we want agile, it is because we know the waterfall methodology we’re using won’t always work for us.

Why the need for change

Change can be driven for strategic or tactical reasons. The ANZ initiative seems strategic by drawing the customers and the business closer together. Another way agile finds its way into an organization is tactical, like when an important business opportunity comes along requiring very quick results with specific functionality and quality.

For either reason, agile and scrum may seemed the only way forward. There are rumors and anecdotal stories about how great scrum and agile are and for some businesses, the decision to try agile re-enforces one hard truth, “Necessity is the mother of invention” or simply:

Necessity drives innovation

The first hurdle in cultural change, Cognitive, is met: we understand the need to do something different to be more successful. In other words, can the waterfall business cite an instance where a new product is created and delivered in weeks, not months (or years)? With a win here, the business, will be competitively better positioned. The agile team, product, and the executives are convinced a change in strategies and culture are necessary but this understanding is not enough to cause change. Trialing any new idea or new practices is a good way to build confidence that a change will result in success.

Get the right people

For a trial project the business doesn’t engage all developers but only a select group. This trial project has a clear goal and a definite release date, neither can be changed. What can change is the scope (as long as the goal can still be met).

The Resources chosen for the first scrum team are special in their experience and talents. Although there are whole product teams that might be used for this new business opportunity, the business decides to build a new development team by cherry picking people from existing teams. The team might be hand-picked for a number of reasons which may include:

  • Years of experience – with experience comes confidence they’ll be successful.
  • People who are self-motivated – these are not people who sit around waiting to be told what to do.
  • People who are innovative – these people are known for their new ideas and abilities to think around problems.

Selecting people for the first agile initiative is often driven by technical expertise and their ability to solve problems. These people are asked to do something very different from their waterfall paradigm but they are more goal driven than most and change is less stressful to them. After a minimum amount of agile training, 45-minutes feels about right, they’re off.

The idea of shifting resources to get lean will happen without much thought. The initiative to be agile in this trial means traditional reporting lines are cut and the agile team reports directly to the agile sponsor.

The only real casualty will be middle managers who traditionally told the developers what they needed to do. With agile, we push this responsibility onto the team. But agile needs managers to clear away obstacles that slow down or prevent the teams from being successful. It’s only when managers can’t let go of their control over people that they become the obstacle.

Motivation for change

One ingredient used as a motivator is the challenge to be more successful. Using your business’s own track record might be the best evidence that change is required. It’s probably already clear that current processes and practices are not given to quick wins. A long decision chain slows things down. If the decision maker doesn’t like the choices available there could be lots of back and forth negotiations. All of this results in delays. What might not be clear is agile cannot change these circumstances. Agile is more mindset than physical and therefore it takes people to push for change. Agile becomes a guiding set of principles that you aspire to achieve. Arguably the most important of the agile principles after satisfying customers will be getting the development team and business in sync and working together as equals.

One very successful agile transformation I was involved in started with the business leaders meeting with the development team and restating the business problem not as leader-to-subordinate but peer-to-peer; the business has this problem and we need to work with you to solve it. We’re not intending to create an elite team but a team with strong mission understanding. This team wants to be successful and given the business or customer problem, business leaders are asking the team to solve these as best they can. In my experience, this will super-motivate the team to try new ways and explore new ideas. The best part is when the team completes the mission, they become the seeds in other newly formed agile teams. They’ll bring with them stories of both successful and failed adventures, but more importantly, they are the seeds to start institutional change.

Institutional change

“One of the things that limits our learning is our belief that we already know something.” ― L. David Marquet, Turn the Ship Around!: A True Story of Turning Followers into Leaders

Changing the way ‘things are done around here’ is hard. If being successful is a motivator for change then it’s clear that the business needs to understand what processes, practices, and decision paths are slowing us down. It’s likely that processes created for a specific reason have bloated or the original problem has itself changed or disappeared. One way to make this work is to drop everything, then analyse each process or practice to get the essential value of it. For example, if a previous practice had the chief architect review the detailed design once completed, you might change this to the team does the detailed design together with the chief architect. Or maybe an architect needs to be a member of the team. Either way can eliminate a hand-off and a potential delay. The point is to streamline existing processes to capture the essence and intended value, not necessarily throw them away. Of special note are those practices and requirements which are generally non-negotiable such as security, quality, and regulatory. Adopting agile doesn’t mean putting the company or customers at risk. Be cautious and review & streamline these with the right people.

There may also be processes that can be made part of the team’s definition of done (DoD) checklist. For example, we had a governance requirement for high-level designs to be reviewed by a ‘design steering committee’. We made updating the high-level design part of the DoD. We didn’t have a written process for this but did have a checklist that served the development team. High-level designs were decided upon during grooming with the chief architect. Once these were documented the chief architect worked with the steering committee, allowing the team to get on with the business of solving customer problems.

Changing institutionalized practices will be a challenge. Changing institutionalized management structures and practices can be impossible. Changing management often means changing the way human beings see themselves. If you’re the manager of a product team and the team is now empowered to make some decisions you once made, or owning practices you once controlled, you might feel a loss, even if you know:

  • Decisions are best made by people who have the knowledge to make that decision.
  • Keeping practices current, efficient, and relevant is best done by the people who use them the most.

By empowering the agile teams to make some decisions and giving teams the responsibility to continuously improve their work practices doesn’t mean managers are obsolete. Managers are critical in setting the boundaries from which the teams operate. Management are the enablers for the agile teams. It’s management’s responsibility in an agile world to ensure the agile teams understand the boundaries, what decisions they’re empowered to make, what processes and practices they own, and, most importantly, management ensures the agile team has the knowledge and support they’ll need to be successful.

Success is about being better, not being agile

My first scrum team was successful beyond all expectations. So successful in fact that the CEO asked afterwards, why aren’t we doing this “agile thing” with all our teams? With that comment from the CEO, comes the first truth about agile transformations: it’s not about agile, it’s about being better. I would wager most CEO’s are less concerned with how or what you do to be successful than they are about being successful. The article about ANZ says “implement the agile approach at scale” which can mean many things.

In my situation, what agile meant to the CEO and the rest of the business was a different approach that included:

  1. quick feedback loops to make decisions,
  2. high visibility of progress and setbacks,
  3. highly motivated teams,
  4. the agile team’s focus on business goals,
  5. the agile team’s continuous engagement with customers, and
  6. the teams desire to solve customer problems with the customer.

All these things made the teams better, made the business better, and made the customers happy. These agile things could not have happen at the scale it did without change.

An Idea in An Agile World

All new products and new product features start with an idea. This idea becomes the first user story, user story #1, for the product. User story #1 begins a journey that could end with either happy customers and happy business stakeholders or end up in the bucket of disused ideas. So what happens to user story #1 and what do you do with this new idea?

You can find lots of articles on breaking down user stories. You can also find articles on how to write good user stories but you usually don’t see too many articles on how the first user story is used to develop a business case. What follows are some thoughts and insights on how to take user story #1 and developing it for greater success.

User story #1

User story #1 is the idea. It’s an idea to achieve business goals, an idea to solve customer problems, and an idea to create or change a product. We’re all familiar with user stories written with an eye toward solving a customer problem through product updates. User story #1 is different as it’s not so much about solving a customer problem as it is a ‘thought’ experiment to discover what customer problems we can solve. So the question is, how should we go about validating the idea?

Validating

When the new product idea is first presented as a means to achieve a business goal, there will be interest but it’s unlikely the business is going to throw monies your way, at least not at first. The business needs more than just the idea, the business needs data to back-up and justify the idea. This translates to meaning the idea is put through the discipline of Ideation or Envisioning.

The idea is the highest level user story for the product but the picture is incomplete and outcomes uncertain. However, there’s usually enough to the idea to begin analysis and experimentation. The analysis often results in a series of metrics and measures collected during envisioning. These are often measured on a some scale that best serves your business. For example, aligning to company strategy might be scored 0=no alignment, to 5=full alignment. Costs might be 0=<$10,000 to 5=>$450,000. Below are some metrics you might be familiar with.

  • Alignment with the company’s strategy,
  • Moves the product team towards achieving a business goal,
  • Is technically feasible,
  • Product innovation category (new, adjacent, or sustaining),
  • Cost associated with it (implementation is usually the biggest component),
  • Predictable business outcome if implemented (revenue, customers, users, …),
  • The business risk of implementing/not implementing is calculated,
  • The competitive advantage, and
  • The market breadth (new, adjacent, or sustaining).

To do the above takes time and you might look at the list and think, “that doesn’t look too agile to me”. Today’s reality in the digital age is all these items need to be completed in the morning so the development teams can implement and deploy by late afternoon. For most of us this isn’t exactly true, yet. But time is a pressing consideration and being quick is important. What we need to find is a way to do business so we have both enough knowledge to start and we have a clear idea of what further information we’ll need. The key bit of knowledge we need to have up front is that the idea aligns with the business’s strategy and goals.

Big ideas need to align with business strategy & goals

Before the idea is broken down, or before any real effort is made, the idea should be validated against the business strategy and goals. You might be lucky and the business strategy is developed in such a way that the product manager can have their ideas incorporated into the strategy from the start. When the business defines what their goals are, they develop a strategy to achieve those goals.

The three elements of good strategy (source: Richard P. Rumelt, Good Strategy Bad Strategy)

  1. Diagnosis: “A diagnosis that defines or explains the nature of the challenge. A good diagnosis simplifies the often overwhelming complexity of reality by identifying certain aspects of the situation as critical.”
  2. Guiding Policy: “A guiding policy for dealing with the challenge. This is an overall approach chosen to cope with or overcome the obstacles identified in the diagnosis.”
  3. Coherent Actions: “A set of coherent actions that are designed to carry out the guiding policy. These are steps that are coordinated with one another to work together in accomplishing the guiding policy.”

Working backwards on this list, new product ideas describe the actions needed to deal with the challenges (goals) found during the diagnostics. In this way, new product ideas become an integral part of the company’s overall strategy.

If the product manager can align their ideas with the business, then the remaining items of envisioning can happen relatively quickly. It’s very likely that for most ideas, the effort to define feasibility and cost can occur within an agile team’s iteration. This is a tremendous way to cut the time from idea to completing envisioning. It also provides for giving the customers a chance to evaluate the idea in a practical sense: A/B tests, user testing, or customer interviews. This technique of learning is often used in the Startup world as Build-Measure-Learn cycle.

The product manager will likely obtain full approval for any opportunity that delivers overwhelming value relative to its cost. Kenneth Rubin in his book, “Essential Scrum: A Practical Guide to the Most Popular Agile Process“, suggests that if there’s any discussion over tiny differences in cost or value then it clearly doesn’t deliver ‘overwhelming’ value and the idea should be dropped.

The easiest and best way to determine value is by getting the idea in front of your customers and users and measuring their response.

Using the development team for envisioning

When speed is essential to capitalize on an event or competitive opportunity, get something out in front of the users today rather than waiting until you’re sure. The trick will be to do only enough to answer questions and reduce risks without over committing the team or business. Keep in mind that you and the business haven’t validated the idea yet so there is a risk. If you’re doing Agile then this will fairly straight forward: use build-measure-learn techniques to understand the value of your idea with customers and users.

The product manager works with the product owner, team leads, architects, or full development team to break down user story #1 (the idea) into smaller user stories. These are the fragments needed to showcase the new product or feature idea. These particular stories are not necessarily work to build a product but will often be part of a prototype or demo system to gather information and feedback. Part of the envisioning process is to validate the idea, confirming its potential, with customers and users. To do this, a demo or prototype to showcase new features to potential customers and users can be an enormous asset without spending too much money in the process.

If the product manager has the luxury of dedicated team at their disposal that’s great! However, a new product idea may not as yet have a team but only a handful of senior people to help bring a new idea to life. Assuming there is a team, one approach is to get a team to include some research work along with their normal complement of product work. This is handled as spikes, technical or business exploration, added to the sprint if doing scrum. (If the agile teams are Kanban then there’s less of a concern with disruption and immediate priorities can take precedence.) If there’s a time critical component to the idea and waiting isn’t an option, then cancelling a sprint and pivoting to take on higher value work is an option but not always a good idea. Besides, if doing scrum there’s already a mechanism in place to do research, prototypes, and mock-ups: the backlog refinement sessions.

Using the development team during envisioning helps the product manager answer some important business questions including these 2 metrics from above:

  • technically feasibility-team can better answer this after developing a prototype,
  • implementation costs-implementation is usually the main influence on costs and the team can make a more accurate estimate.

By using the feedback and learnings from the demo and prototype, the product manager can better determine the viability of the idea. Using A/B tests, user testing, or customer interviews, the product manager can:

  • better estimate the business outcome if implemented (revenue, customers, users, …),
  • better assess the business risk of implementing/not implementing the idea,
  • better assess the effect on competition, and
  • better understand the market breadth.

Getting the development team involved with the idea early on will benefit the product manager. This early involvement moves the team from being a partner with the product management into being full members of the business team, bringing along their unique skills and insights into the envisioning process.

Two key outcomes having the development team in on envisioning

One big advantage is the development team is better positioned to undertake the new work if the business decides to proceed. The development team also has more ‘skin in the game’ through early involvement.

Another big advantage is the product manager can determine sooner whether to pursue the idea further. Because the development team has created prototypes or demos for customers, the product manager is better able to collect critical information for the business case. If it turns out there is no profitable business case, the product manager has spent relatively little money to learn a lot. The agile principle of maximizing the amount of work not done is an important one and finding out early to stop is better than running out the budget on a hunch that doesn’t pay off.

In a waterfall environment?

If you’re doing a waterfall type methodology then it’s a given that all envisioning and requirements are defined upfront. This is not bad if there’s little to no uncertainty. However, the early involvement by the full development team to pursue the idea using prototypes is still valid. They’ll not only be putting more ‘skin in the game’ but they’ll improve the accuracy of the estimates and potentially uncover more unknowns.

Summary

To increase your chances of success, get the development team involved during envisioning. They can help the business best if they’re a full member of the business team and not just a partner. Use the full development team to validate user story #1. This includes validating customer problems, potential solutions, development costs, and business value.

 

Being Agile When The Business Isn’t – Communications

I recently met a group of people having a hard time ‘doing’ agile. They understand agile but others in the business chose not to participate. These were managers and business leaders. Although the group’s first reaction is to ‘sell’ agile I gave them an alternative of doing things to add value both for the business and for customers without a sales pitch. What if the team communicated a better way of doing work that created more success? Wouldn’t the business embrace better ways of being successful?

Let’s start by establishing that A) No one goes to work hoping to fail and B) Business leaders and managers often don’t see how changing themselves can improve the team’s outcomes.

Although people don’t want to fail, we can’t assume they’ll do their utmost to be successful. Being successful takes effort but this is something that can be leveraged for your advantage. You’re not going to push agile but rather advocate a means to be more successful. Your communications with the team and business leaders are about being successful and about ways to be more successful.

When you communicate with managers and senior stakeholders in the business, they’ll most often listen to you but more importantly, you’ll need to listen to them. These people want something from the teams and your best approach is to understand their real needs and not just their ‘wants’. Although the business usually measures output as a key indicator, it’s been my experience they need reassurance that progress is being made, promised business value is being delivered, and the team is meeting their delivery commitments. Often times however, management are unaware of what they need to do to help the teams be more successful and it is this particular behavior that teams need to communicate.

Simply put, these communications have three parts: 1) teams to understand the needs of the business and what makes the business successful, 2) business leaders need to understand their role in helping teams (and themselves) be successful, and 3) the teams need to find the means to meet the business’s expectation and let the business know the ‘actuals’. Sounds easy, yes?

Inception workshop planning

The best way I’ve seen to get everyone started on the same page and establishing desired communication patterns is using an Inception Workshop. You can use the inception workshop to fully communicate intentions, expectations, and create an initial delivery plan. Although an inception workshop can have many topics and areas of exploration, I would focus particularly hard on the following topics:

  1. Why we are here – the reason for this activity and our outcomes for a successful workshop
  2. Why is this important – the business goals and objectives we want to satisfy by means of solving customer problems
  3. Product Vision – developing a shared product vision that inspires everyone to achieve the business goals using the product
  4. Stakeholders Who / do – the key stakeholders both internal and external and the actions required for a successful project
  5. Empathy Mapping – a deep dive into the users, customers, or stakeholders
  6. Journey Mapping – how users, customers, or stakeholders will use our product or service (
  7. Trade off sliders – the team’s upfront assessment of what’s important and what will or will not be compromised to achieve total success
  8. Epics / Story Cards / Relative order – initial story mapping to discover or confirm the highest priority requirements/user stories
  9. Estimates / Releases – continuing with story mapping to develop a release plan. The team(s) commit to this release plan acknowledging that some information might be incomplete or unknown.

These nine steps are about reaching a common understanding of who benefits from the project and what problems need to be solved to achieve our business goals. However, more important than these two outcomes is the openness and transparency of the communications between the business and team(s). Planning your business’s next project or endeavor using an inception workshop sets the precedent of business teams and creative teams working together as one in an open communicative environment. Now the real trick is to somehow institutionalize this new-found communication pattern into the organization.

Company culture

I’ve worked in organizations where managers were expected to make decisions and be seen as leading their teams. The reality is this can be changed but it does require a lot of openness and introspection (see The Journey to Servant Leadership as an example).  To keep the open and collaborative feelings discovered during the inception workshop going, the senior business managers and middle managers might need to change the way they work. The key to business success is always knowing where you’re at. This is what needs to be continually communicated upwards, from the teams to the business managers and stakeholders. This part is generally easier to achieve than getting business managers and stakeholders to communicate their current situation and results back to the teams. Why should this be difficult?

One of the reasons a company’s culture is not opening itself up to change is the breakdown in communications. This is evidenced in high level managers aggressively stating how things cannot change and giving ‘reasons’ why this is so. Often when this happens, those wishing for change back down and accept the status quo. The following illustrates people taking positions on an issue rather than communicating.

  • Team lead to Business Manager: Can you tell me how the business is doing with our latest delivery?
  • Business Manager to Team lead: Customers seemed to be happy with the new features.
  • Team lead: Can I get their specific comments and the overall NPS?
  • Business Manager: No, sorry. That’s confidential information.

With that little dialog it’s easy to see how the company has limited access to information. The painful reality is the teams must be completely transparent to the business but the business often has policies that limit its transparency back to the teams. This is where the culture shift needs to occur. If the team is to feel fully engaged with the business, the business must fully engage with the team.

One way to overcome these obstacles is to change the dialog. The dialog above has the team lead ‘telling’ the manager what they need to do. The manager responds with a shallow summary but hides behind policy when it comes to any meaningful depth. The above dialog is just that: a dialog. It doesn’t fall into the collaborative conversation category because clearly there is no collaborative effort made by either party. Culture trumps communication.

A different approach would be for the team lead to consider the business value of their request. For example, take a look at this illustration from Michael Sahota of agilitrix.com.

What the illustration shows is a variance of culture between the team solving problems and the business people running the company. It is hard to change either of these cultures so don’t try. As I’ve talked about in previous posts, the best way forward is to work within the business’s culture and make a compelling business case for opening a previous inaccessible communication path. Here’s how that communication might go down:

  • Team lead: We’ve been getting some customer feedback at our reviews but we are having some difficulty mapping our value delivered with the actual progress in achieving two business goals: increasing NPS and increasing customer referrals. If we can get better and more specific insight on what customers are saying and doing, we can make sure we’re working those requirements that can deliver the most business value sooner.
  • Business Manager: How can you use this customer information to achieve our business goals sooner?
  • Team lead: Once we can get their specific comments and the changed NPS, we can analyse what is motivating customers to use our product. Once we have that we can groom our stories to get those stories which enhance features customers are talking about to the top of the backlog.
  • Business Manager: Ok. It’s confidential information, I can’t just give it to you but I tell you what, let’s get all the team leads together and we can review the raw data. Will that help?
  • Team lead: Absolutely. I’ll get the team leads together and we can meet here in your office at say, 2:00 pm?
  • Business Manager: 2:00 pm is good. It should take about an hour and if we need more time we can find it later.
  • Team lead: Great! see you at 2 and thanks a lot.
  • Business Manager: You’re welcome. See you all at 2.

The difference with our second example compared to the first is the second is clearly a collaborative conversation and one that resulted in potentially creating more business value sooner. The team lead has changed the conversation from one that is a request from the manager to one that is offering to serve the manager.

Summary

Communications between the business and the team needs to be collaborative and mutual. Using an inception workshop is a great way to start any project or adventure with an open, collaborative setting.

It’s also important to understand that the cultural bubble around the teams and the organizational culture are probably different. Any ongoing communication needs to account for this. When talking across cultural lines, use language and terms that are meaningful in the ‘other’ culture. In our team lead example above, the team lead used the language of better achieving business goals to break down communication barriers.