When will my agile team self-organize?

Great! Your business has decided to move from a Waterfall paradigm to Agile Scrum and now you’re waiting patiently for the benefits to kick in. You’re hoping your Scrum teams start self-organizing soon and are inventively solving customer problems faster with better efficiently. You’re also waiting for the team to become major contributors to the business decision process based on their intimate knowledge of customers and their pains. And you’re waiting … Still waiting …

You ask yourself what’s holding the team back. The Scrum Master has trained the development team on the agile manifesto including the principle:

“The best architectures, requirements, and designs emerge from self-organizing teams.”

The Scrum Master has also trained the development team on self-organizing as described in the Scrum Guide:

  • defining the Scrum Team: “The Scrum Team consists of a Product Owner, the Development Team, and a Scrum Master. Scrum Teams are self-organizing and cross-functional. Self-organizing teams choose how best to accomplish their work, rather than being directed by others outside the team. … The team model in Scrum is designed to optimize flexibility, creativity, and productivity.”
  • defining the Development Team role: “Development Teams are structured and empowered by the organization to organize and manage their own work. The resulting synergy optimizes the Development Team’s overall efficiency and effectiveness.”
  • defining the Scrum Master role: “The Scrum Master serves the Development Team in several ways, including: Coaching the Development Team in self-organization and cross-functionality.”
  • defining the Sprint Planning Meeting: “The Development Team self-organizes to undertake the work in the Sprint Backlog, both during the Sprint Planning Meeting and as needed throughout the Sprint…By the end of the Sprint Planning meeting, the Development Team should be able to explain to the Product Owner and Scrum Master how it intends to work as a self-organizing team to accomplish the Sprint Goal and create the anticipated Increment.”
  • defining the Daily Scrum: “Every day, the Development Team should be able to explain to the Product Owner and Scrum Master how it intends to work together as a self-organizing team to accomplish the goal and create the anticipated increment in the remainder of the Sprint.”

It’s clear the Scrum Guide expects the Scrum team, and more specifically, the development team, be a self-organizing body. It’s also clear that training people that they should be self-organizing isn’t always enough. The organization must want and the businesses actively support the team becoming self-organizing.

Setting Expectations

Getting a Scrum team to self-organizing is hard primarily due to the fuzzy nature of what ‘self-organizing’ actually means. If you were to ask 5 people to define what a ‘self-organizing’ software team is you’ll probably get 5 different answers. But even in that result some truths are seen, it’s not how I define self-organization, it’s how the team defines it.

In Nitin Mittal’s Scrum Alliance article, “Self-Organizing Teams: What and How“, he describes 5 essentials of self-organizing teams:

  • Competency: Individuals need to be competent for the job at hand. This will result in confidence in their work and will eliminate the need for direction from above.
  • Collaboration: They should work as a team rather than as a group of individuals. Teamwork is encouraged.
  • Motivation: Team motivation is the key to success. Team members should be focused and interested in their work.
  • Trust and respect: Team members trust and respect each other. They believe in collective code ownership and are ready to go the extra mile to help each other resolve issues.
  • Continuity: The team should be together for a reasonable duration; changing its composition every now and then doesn’t help. Continuity is essential for the team.

I was once Scrum Master to a team during which I did training on the advantages of setting priorities, of sharing work, and of getting stuff done. The team was very good at their jobs and helped each other out whenever asked. However, I didn’t realize helping out had a limit until one day at the daily standup when the person working the most important story, the highest priority story, was out. I asked who could pick up the work to move it forward? No one raised their hand. One of the quietest people then said it was massively inefficient to have someone else pick up the work. For me personally this was a sad, sad day. After talking priorities, talking of sharing work, talking of getting stuff done, and talking how the team best works together, the team couldn’t let go of their own personal task responsibilities. The team couldn’t find it in themselves to embrace a team responsibility of getting the most important work done first. A self-organizing team would have kicked into a self-managing mode and re-allocated work so the highest valued work could continue. At the end of the standup, team members felt a stronger bond to their own work in areas they were most comfortable with than to the whole. It wasn’t competency but rather motivation that was holding the team back.

What makes it so hard to motivate a team to feel responsible for the whole? When a business pushes individual roles & responsibilities over team commitments or when survival and blame permeates an environment, it’s very unlikely that the team will move beyond the ‘forming’ stage (team members sticking to what they know best individually and doing what they feel most comfortable with).

In some business environments, the dip in productivity while transitioning from ‘Forming’ to ‘Storming’ is unacceptable. This would be especially true in a strong hierarchical or Command & Control environment where those in the hierarchy are held responsible (accountable) for team performance.  If the business is clinging too tightly to a hierarchical and Command & Control environment, the likelihood of birthing self-organizing teams is diminished. At one company I was at, one of the most senior software engineers was a team lead. The team was always seeking his approval for any decisions and if the manager thought otherwise, the team changed their ideas to conform. This wasn’t the worse thing to happen but it was heavily reliant on the team lead always being right. A consequence of this was lower scores on the company’s engagement evaluations especially around empowerment and feelings of trust.

It’s worth the attempt

Adopting Agile and Scrum will not automatically cause people or businesses to change. For Agile and Scrum to make a positive impact, people will need to see that they need to change their own behavior. It’s possible to see that in an environment where individual knowledge or individual skills are the most sought after commodity, Agile may not be the right tool. In a business where rigid hierarchies and Command & Control are dominate, it may be a long journey to establishing self-organizing teams but certainly worth the effort.

David Ticoll in his Harvard Business Review article, “Get Self-Organized”, states that, “It would be difficult and risky—even foolhardy—to try to wholly transform a hierarchical business model into a self-organizing one. But the potential of self-organizing systems to enhance competitiveness is becoming clear to managers of some conventionally structured businesses.”

David Tricoll further states, “Today, the ability to stream complex, real-time information to the front line gives hierarchical companies greater power than ever to exploit self-organization.”

One approach might be getting the team to participate in business events such as creating business plans, working with customers and users, and respecting them as contributors to business success. Teams with greater insights to business and customer problems are more likely to be engaged with solving these. To be quick and competitive, businesses must relinquish control to the teams in the front lines and the teams must be willing to accept the responsibility. When the business leaders and development teams shared these insights, I’ve seen the development teams rise beyond their own specialties to rally around both the customer and the business.

For a team to become self-organizing, it will take more than a business to want and support it, it requires the team itself to see advantages in self-organization. If you’ve ever seen how an assembly line worked in days gone by, you’ll see an environment where self-organization would have been detrimental. Your job was to put nut ‘A’ onto bolt ‘B’ period. Repeat this for 8 hours a day, 5 days a week and you got paid pretty well for what to some people would be the most boring job on earth. The business wasn’t holding out hope that the workers would self-organize and the workers had no motivation to do so. In the factories, piecework was common. Workers were told what to do, how to do it, and when to do it. Software development is not piecework, if it were, robots would be doing it today. It is knowledge work where thinking both inside and outside the box are assets to the business.

In the situation above when a team member is absent, the moment the remaining team members decided that doing and completing individual tasks is more important, all advantages of self-organization were lost. Instead of adding value for the customer and business, the team had elected to follow a plan. The decision essentially comes down to: I have my own work to do and don’t have time to do yours.

To some, this kind of dedication is how you advance in the company. However, if the company were to emphasize customer and business value of work being done, it would cause a fundamental shift in team thinking, from doing tasks to getting the highest value work done quickly. When the business puts emphasis on value, the team is more likely to devote their combined efforts to achieving value. It would come to pass that even when one of their numbers is absent, progress won’t stop, although it may slow down. If the team is accepted as a fully participating business team member, not just a partner, the team will genuinely feel the success of the business is also their own success and act accordingly.

To me, one of the principle tenets of Agile is the team. The great power of Agile comes from a group of intelligent people working as one. Collectively they are more knowledgeable than any one individual although the shared experience may be less. To ignite the team requires the business to openly give the team responsibility and accountability for the outcomes of each iteration. For the business to move forward and empowering the team, they would need to establish safe boundaries from which the team can safely operate in.

When business leaders, product owners, and scrum masters attend the daily standup, what’s more important than hearing team members say,

I completed task <x> yesterday”,

is hearing them say,

we added <some> value to the product yesterday”.

The role of managers and customers on the self-organizing team

In an InfoQ interview, Rashina Hoda cites two environmental factors needed to be in place to enable self-organization to emerge. These are:

  1. Senior management at the teams’ organization must be able to provide freedom to the teams so that they can self-organize themselves.
  2. Customers must support the teams by being actively involved in the development process through providing regular requirements, clarifications, and feedback as required.

“Self-organizing Agile teams … require organization structures that are informal in practice, where the boundaries of hierarchy do not prohibit free flow of information and feedback. In an informal organizational structure, the senior management is directly accessible by all employees (maintaining an ‘opendoors’ policy), and accepts feedback—both positive and negative.”

– from “Supporting Self-Organizing Agile Teams What’s Senior Management Got To Do With It?” by Rashina Hoda, James Noble, and Stuart Marshall

Summary

For the business leaders, giving development teams insights to both business and customers plus giving development teams space to operate, will motivate teams to accomplish amazing things. When the business states these are our goals and asks the team to solve it in the best manner possible, you’re likely to see a team self-organize to meet the goals. No one goes to work hoping they fail.

When the business has a strong hierarchical and Command & Control environment, moving a team toward self-organizing is made more difficult but not impossible. Building a bridge of trust and respect is essential. By establishing a strong partnership between the business and development team, followed by full membership, the team can exercise their collective knowledge and capabilities to contribute to business successes.

In the end, it’s a mutual effort on the part of both management and the development team to become self-organized. It will probably be a bumpy ride along the way but with a clear goal as a guide, both should arrive at the destination together.

 

 

 

Make Agile User Stories Personal With Personas

Feature image “Proto-Persona” by visual.ch (CC BY-SA 2.0)

When a product owner or product manager helps create user stories with the development team, does the development team walk away feeling empathy for the customer? Can the development team and business benefit if the team ‘feels’ for the customer? Most businesses are pushing their development teams to produce all the time, from project to project, so developers eventually get used to this pressure and find their own behavioral niche within the organization. So why would a team suddenly begin doing more work in less time than any project before?

At a previous company, we had the advantage of being able to know the names of our customers and potential customers. This didn’t always happen but when the development team knew a customer’s name it was magical. During the daily stand-ups, the development team talked about the next bit of work they’d tackle but they’d also talked about the customer, “Is that what Bill (not the real name) wants us to do?” Management also talked in terms of the customer when speaking to developers. The development team knew their stuff and were top developers but they looked at the work with an eye equally on their customer’s wants and needs. The team spoke the customer’s name so it was clear they knew exactly who they were working for. The team’s productivity shot way up and they even worked weekends to get the right product into “Bill’s” hands sooner. This was a dream team for the business and would be the envy of most companies doing Agile. Or Waterfall.

Now that’s a nice, feel good story about a development team’s awareness of their customer but this is probably the exception. E-commerce businesses flourish these days with thousand upon thousands of customers so it would be difficult if not impossible to get a development team rallying around a single customer. The next best thing is a persona representing a customer, a customer segment, or a user profile. I first learned about personas from Alan Cooper in his book, “The Inmates Are Running the Asylum”. At the time we might have missed the mark sometimes because our personas were very detailed in things that nowadays would be considered unnecessary and less helpful in aiding product decisions. However, personas can help development teams feel a closeness to their customers and be more empathetic towards their customer’s pain.

What I generally notice is companies have personas but they’re often used only in Marketing. The idea is to get these personas out to everyone involved in product development and steering the full product team’s thinking towards personas as real people. I was at a great presentation by Atlassian product manager Sherif Mansour who spoke at Agile Australia a couple of years ago on the topic, “Do Agile Right – Lessons Learned From an Atlassian Product Manager“. This easy to listen to video touches a bit on using personas and journey lines. Atlassian uses personas as the backbone of their product development. If you’d like more information on creating personas, see the end of this article.

User stories by Mike Cohn

Connextra is credited with the most popular user story format, “As a <blank> I want <blank> so that <blank>“. This is a simple yet elegant way to capture “who”, “What”, and “Why” for a software requirement. But it’s Mike Cohn and his book, “User Stories Applied“, that really made user stories come alive in the Agile community. Mike Cohn does suggest that the ‘why’ is optional but I think for teams early in their Agile journey, this will be very beneficial as it helps new Agile teams understand the customers’ problems better.

A good user story can provide an innovative and imaginative development team all the necessary information to begin their task of talking with stakeholders and the product manager in order to figure out how they’ll do the work. The goal of the user story is to provoke conversations to get more details rather than writing a specification.

So does this happen and if it does, how often? My experience shows this in fact rarely happens. Yes we write user stories using the format, “As a <blank> I want <blank> so that <blank>”, but we’re really following a cargo cult software engineering ritual without really understanding the purpose of the user story. This is exactly where personalizing the user story can reap great benefits.

Personas in user stories

A lot of user stories I see look like this:

“As a user I need to see an error count so I can track whether the number of errors exceeds my reporting threshold.”

This story was constructed for the “user”. Unfortunately “user” is such a general term doesn’t tell us who the story is for; is the story for an 8-year old or 80-year old grandmother? Is the story for men or women? It’s going to be hard for someone to read the story and be empathetic toward the “user”. There’s no human connection to this story so no one should expect much in the way of human feeling toward implementing the story. The development team is just as likely create a display item that shows a simple error count because that’s the easiest and fastest way but is it the best way or is it what the customer really wants? What’s missing is the personalizing of the story which can be expressed in the form of a persona. By personalizing the user story, the question the developer asks is, “is this what ‘Grandma’ wants?” instead of, “does this meet the acceptance criteria?”. If the whole product team, (finance, marketing, sales, support, development, and product management), are behind helping Grandma out, it’s just that much more likely to happen.

Personas and visibility

Now someone reading this is saying, “but we said who the user was at the grooming session” and that’s great but now you have a story that only a small handful of people knowing who the story is for. This shows a certain complacency within the agile team and not being transparent to the business. By avoiding tying the story to a specific business concern in the form of persona, other parts of the business are in the dark as to the real value provided to the business by delivering this story. How easy would it have been to put in persona “grandma” instead of “user”? The Product Owner and Development team owe the business this insight and should always be aware of what they’re communicating to the business. If the agile team is looking for autonomy and empowerment, what better way to earn this than by being open and showing they are striking at the heart of business concerns.

Summary

The real advantage of personas will come from the development team as they create solutions in quick and caring way. Even if the development team only increases value delivered by a couple of percentage points, the product manager, the business, and the customer win. This can happen if the development team’s interest, excitement and commitment can be inspired by using personas. Atlassian treats their personas as people and the company talks about their personas as real people. This is not too far from the experience I spoke of earlier in this piece where the team talked about their customer.

There are as many styles of personas as there are customers. If you link a user story to a specific persona and that persona maps to a business or customer segment, then it will be easy for the business leaders to quickly see the potential value the team is planning to deliver. It also gives the development team a chance to relate their work to the goals of customers represented by personas.

Learning more

I am assuming most people reading this are familiar with creating personas but for those less familiar, here’s a site to get you started with personas.  There’s also help from Roman Pichler who, in his article “From Personas to User Stories“, provides a template for developing personas. An example of Roman’s persona template is shown below. Roman suggests that personas and their goals begin their appearance in the larger, epic, user stories and flow all the way to sprint ready stories. This will help focus the team on solving problems for a semi-living customer and it will also make user testing easier because the goals of that customer are defined in the persona.

 

 

Being Agile When The Business Isn’t – Transparency

In the last two posts, ‘Being Agile When The Business Isn’t – Requirements‘, and ‘Being Agile When The Business Isn’t – Team‘, I talked about how to be more Agile in an environment where the business is strictly Waterfall. I’d like to continue this theme with some ideas and practices to make the teams more ‘transparent’.

Inspect, Adapt, & Transparency

Most will recognize Inspect, Adapt, & Transparency as the three pillars of Scrum but they’re ideas that even the most Waterfall of businesses can use and take advantage of. These three pillars are the core to continuous improvement, building what customers need to solve their problems, and helping bring the greatest valued work to the forefront. I’ll focus on transparency as I believe once you have transparency, inspection and adaptation will follow naturally.

Perhaps the big difference between Agile Vs traditional project management is how these three items are acted upon. When I was doing big Waterfall projects our alleged transparency came from the project’s communication plan. This communication plan defined the reports the project team was responsible for generating. At or near the end of the project, we wrote project closure reports which often included lessons learned. While writing the closure report, we were being formally transparent but only after everything was done. The real problem with the closure report was the timeliness of it. Because this is traditionally done at the end of the project, nothing learned here can help the just completed project. We also suffered from fading memories. We remembered the specific events but often needed to speculate and debate over the causes. Interestingly, we also seemed to forget these lessons for the next project or determine the next project is different enough so the lessons weren’t considered applicable. I’m not suggesting nothing was learned but any learnings carried forward tended to be the exception.

The result is some learning is done but the changes haven’t been trialed or proven to be effective in solving whatever the problem was. To improve requires openness and transparency so you can inspect what has happened and adapt to new and better ways throughout the project not just at the end. To maximize the value of improving, the project team needs to be continuously inspecting outcomes and results including any new ways of doing things. Building in transparency from the start will help.

Transparency

Transparency is arguably the first step for improvement. Nothing will help a business improve better than seeing undeniable evidence that some work practices, processes, and communications are less than optimal or just plain bad. In the Waterfall environment, the project plan includes what practices and processes the teams will use and the order they’ll use them. Most project plans also have a communication section where and how the various project statuses and metrics are reported. However, it’s only an illusion that transparency exists by executing a communications plan. The reason is, communication plans dictate specific content and intervals, usually within categorizes like any of the following:

  • Current project status
  • Current risk assessment
  • Resourcing status
  • Requirements management status
  • Stakeholder management status
  • Change management status

This doesn’t provide any insights or linkages to bigger or widespread problems. There’s also the problem that the reporting is done by the project manager. This can lead to information being filtered and potentially skewed due to conflict of interest. Project managers are generally held accountable for the project plan which means they’re accountable for perceived progress or the perceived lack  of progress. And finally, after a while, the project manager is most often simple ticking the box on his or her reporting requirements.

Daily standup & transparency

Scrum Trainer Charles Bradley wrote in his article, “The Tao of Scrum – Transparency“, “the best transparency leads to the best collaboration loops”. To maximize the value of collaborative feedback loops requires transparency and to me, transparency implies anyone can quickly gauge what’s happening in a project with a minimum of effort. Although you might be stuck with an old-fashioned communication plan, there’s an easy ceremony you can implement with your Waterfall team that gives extra clarity to what’s happening on the project: a daily standup. Each day the team talks about how their plans for the previous 24 hours went and what their plans are for the next 24 hours. It also provides an opportunity for others with their own unique perspectives to make observations and comment.

In my Waterfall past, as the project was approaching the final stages of test, we had all the project managers, line managers, and the program office people sit together daily to talk about specific tasks the teams were doing and when we thought they’d be done. Although this worked quite well for the management team, the nuance of individual planning was lost. Had there been a team standup, the confidence or uncertainty was more likely to come through. As it was, what was reported to management was an interpretation of plans and status.

In a Waterfall environment, the full project team might be large since the team is built from specialist who usually don’t take on tasks outside their specialty. This can mean the daily standup is large if everyone goes so careful monitoring is essential. The daily standup must remain valuable and true planning must occur otherwise there’s no point to it. In Agile, one valuable component of the daily standup is the collaborative nature of getting the most important work done first. In Waterfall, the project manager and product manager might need to be a little flexible in their detailed project plans to allow the collaborative nature of the implementation team to emerge. Better still is for the project plan be aligned with getting functionality done in order of importance rather than based on components and skills of the team.

Plans & transparency

Another aspect of transparency will be the visibility of the plans. If the project is using MS Project or Jira for Agile teams, it is a huge mistake to think this by itself constitutes transparency. Both these tools track progress when properly maintained but in both cases, people often don’t take the extra steps necessary to open a file or log on to Jira or even learn how to use the tool. The real problem isn’t progress wasn’t tracked, it is, but the real problem is the difficulty of getting to this tracking data and properly interpreting it. These tools have their advantages but ease of use is not always one of them. It’s not that these managers and program office people are lazy but rather their time’s at a premium. Visualizing the project and sprint status on a wall is easy. Bits of sticky paper with notes, in columns is easy. I have used a big wall as a Gantt chart or User Story board to great success. It really doesn’t matter if you’re doing Waterfall or Agile, the advantages of this high visibility is the same: huge.

If your team and stakeholders are global then it’s a wise idea to set up a camera on the planning board. I had used a camera while we did standups to allow global stakeholders to see & hear what was going on. I had a second camera on our ‘story board’ 24/7. This was intended to add more transparency and to give our global stakeholders a bigger slice of the picture, allowing for more feedback and collaboration.

Transparency is a conscious effort

Agile isn’t immune to poor transparency and communications. As Scrum Master, I’ve been cornered by the product owner/manager and asked what the ‘real’ state of progress was in the sprint. Two things needed to happen: the product owner needed to attend the daily standup and the team (with some advice from the Scrum Master) needed to better present state sprint progress. In Waterfall, the project manager has their hands full too. While Agile implies the implementation team is involved and responsible for managing expectations and communicating value completed, Waterfall usually relies on the project manager to do these. It’s up to the project manager to move beyond the project plan and establish a collaborative atmosphere where visibility and transparency become natural within the project team.

Transparency can be scary as it will expose people, their work practices, and their outcomes to all. It’s not the natural state for developers who learned their craft in a Waterfall environment. The first time I had transformed a Waterfall team into a Scrum team, one of the team members told me that they were completely stressed out by the high visibility of their work. In his case, a technical decision had proved wrong in a most open sort of way. In the past, management would probably not know that a technical glitch had happened but Agile made it visible. In hindsight, the team member realized the focus was to identify and correct a technical decision quickly and not to allocate blame or ridicule, (he was unconvinced of this in the first days). Transparency became a tool to do better and improve but not without some pain on the journey. Making previously hidden work habits visible will be stressful for some but this can be countered by avoiding the culture of blame and ridicule.

Summary

By using transparency as an aid to see what’s working well and what can be improved, the opportunities for improvement are more likely to be seen. Transparency has two key accomplices: daily standups and the wall project planner. Both these can be used in Waterfall environments without too much effort beyond simply doing it. Having the implementation team being transparent and talking directly to their plans each day is a key element of Inspection & Adaptation. This daily transparency reduces the risk of going off on a tangent as illustrated in the earlier story about an unsuccessful technical decision.

Transparency also extends beyond traditional tracking tools. Although these tools help track and document progress, it requires both effort and training to use these to their full potential. Training all stakeholders and customers with these tools is hard and may be impractical. Using a simple system of sticky notes and a big wall can convey the same information in a visual way that transcends most communication roadblocks. If there’s no wall, then use a big whiteboard. No whiteboard, then use the windows. You can even put a camera in front to broadcast the planning board across the internet if necessary.