The Science Behind Scrum: Learning, and How to Develop a Learning Organization or Team

The Science Behind Scrum: Learning, and How to Develop a Learning Organization or Team
By Christiaan Verwijs

This article is the first in a series of articles that I will use to explore the science behind Scrum and Agile methodologies. Many working in software development might be familiar with Scrum, but not with the rich and vast array of scientific research it is based on. Most of you have probably heard of the research by Takeuchi & Nonaka (1986) that created the foundations of Scrum. But in these articles, I want to delve deeper into the decades of scientific research by organizational psychologists, sociologists, educational experts and cognitive psychologists that Scrum is based on. For the more practically minded; don’t worry. I will make sure to add a lot of useful (practical) pointers.

In this article, I want to explore how and why learning is relevant to Scrum (and other Agile methodologies) and how learning can be facilitated. As Scrum is a framework for developing and sustaining complex products, it emphasizes that learning is important but it does not tell you how to achieve this. Keep on reading if you are – like me – interested in answering that question. I will begin by taking a look at some of the scientific background for Scrum’s ideas about learning. Based on this scientific foundation, I will investigate how learning can take place within organizations and teams and how this can be facilitated. I will try to keep the scientific stuff as short as possible without sacrificing the goal, so if you are interested in reading more about that, try some of the many source I’ve listed. If you’re not digging the science, just skip to the practical part at the bottom of the article… Also note that the term ‘Organization’ does not necessarily mean a large institution with many people. Even four people working together are an organization (usually, at least).

Embrace change: The Theory

‘Embrace change’ is a well-known saying in Agile methodologies, such as Scrum, Extreme Programming and Kanban. The saying is based on the experience that in our complex world, change is bound to occur continuously. The only way for organizations to survive in these fast-moving, heavily globalized environments is to learn how to continuously adapt to small and large changes. This applies on many levels, from very large organizations consisting of tens of thousands of employees to small development teams. Organizations that do not adapt, will not be able to weather the storm and will be out-competed by more flexible organizations. In the field of organizational psychology, this has led to the development of theories about learning organizations. Originally, the idea can be traced back to Argyris & Schön (1978), but Senge (1991) identified a number of characteristics of these kinds of organizations (Rollinson & Broadfield, 2002):

• Mutually shared vision: All members of the organization share the same vision (for the future);

• Open communication: People communicate openly and transparently without fear of criticism or rejection to facilitate team learning;

• Mental models: The mental models (values) that are shared by members are analyzed and opened up to criticism and change;

• Personal mastery: individuals are committed to disregard old ways of thinking and embrace new routines, ways of thinking and problem-solving;

• Systems thinking: members are able to conceive of the organization’s processes, units, activities and iterations with their environment;

The process of becoming a learning organization is very difficult, and requires many barriers to be bested. Koffman and Senge (1993) identified three barriers that must be overcome to facilitate learning:

• Fragmentation: Specialization makes people lose sight of the bigger picture and will make people focus only on their ‘own’ set of problems, rather than the larger problems. This obstructs learning;

• Competition rather than collaboration: In most conventional organizations, competition is rewarded (explicitly or implicitly) which stifles collaboration and open communication and impedes learning;

• Reactiveness: Stress, high work loads and high demands often take time and focus from learning. It also promotes risk-avoidance strategies. This prevents learning;

Overcoming these barriers is difficult. According to researchers like Argyris, it requires changes on many levels:

• Leadership: Leaders have to shift from an autocratic (micro-management) style to a more visionary (or so-called transformational) style of leadership. This means that leaders should focus not on telling people what to do, but motivating them to get there together through learning and collaboration. In other words, leaders have to inspire rather than direct;

• Culture: An organization’s culture must change to adopt values that are important for learning to take place. This means that mistakes should be viewed as learning experiences, rather than opportunities to place blame and punish. The culture must also reflect values that facilitate open communication, transparancy, autonomy and democratic decision-making;

• Autonomy: Learning can only take place if individuals have the autonomy and ability to learn. This means that they should be able to experiment, make mistakes and learn from them. The opposite of autonomy is telling people exactly what to do and how to do it, which will foster a culture where people don’t feel responsible for their work and their own learning process;

• Communication: Organizations have to invest heavily in new ways to communicate, more openly and throughout the organization. Learning can only take place if an entire organization is involved, on all levels;

These changes together are transformational changes. They (should) completely transform the way people think and behave in organizations and teams. Obviously, that’s quite hard.

And Where Does Scrum Fit In?

The above might sound quite abstract and vague, but Scrum does a pretty job of making this more concrete within the context of a single Scrum Team. As a matter of fact, Scrum addresses quite a number of these barriers by implementing a number of rules:

• Self-organization: Scrum reminds managers that teams are very capable of solving problems by themselves, and that they should be given the freedom to self-optimize and self-organize to do this in the most efficient way. I wrote earlier that self-organization can be considered to be pretty much the same as learning. One could argue that self-organization is the application of learned knowledge, but that’s more a semantical discussion;

• Sprint Retrospectives: By reflecting on the effectiveness of a sprint at the end of the sprint, there is a formal opportunity to identify bottlenecks and correct them in future sprints. The Retrospective actively promotes learning by putting all those relevant to the product in the same room and making them reflect on the past sprint and adjust their process for the next sprint. This is a learning opportunity at it’s best!

• Product Owner as part of the Scrum Team: By making the Product Owner part of the Scrum Team, and thus also responsible for the value of the increments, hierarchical levels that usually exist are broken down. Development Teams collaborate directly with the ‘customer’, rather than communicating through many noisy organizational tiers like analysts, account managers and project managers. This makes communication more effective and allows for collaborative learning;

• Cross-skilled teams: By requiring teams to be cross-skilled, Scrum addresses the barrier of fragmentation within teams, and how this inhibits effective learning. If a team is comprised of many specialists, this might lead to a diffusion of responsibility. Members might not feel responsible for functionality X that should be implemented by specialist A (‘we couldn’t finish the sprint because or visual designer didn’t finish the design on time’). If a team thinks like this, it will not seek alternative solutions and thus not self-organize;

• Inspect and adapt: The empirical control that is important to Scrum is used to reduce reactiveness by implementing a large number of opportunities for Scrum Teams to inspect their process and adapt wherever and as soon as possible. In fact, this is probably the essence of learning in general (we could easily rephrase this to ‘inspect and learn’);

• Servant leadership: Scrum emphasizes that Scrum Masters should take on the role of servant leaders, rather than autocratic (micro-managing) leaders. They should facilitate the process of a Scrum Team rather than directing it. This mirrors what I just wrote about the change in leadership style that is necessary for learning organizations to develop. In fact, Scrum Masters often state that they should inspire teams and make them exceed their own expectations. This is exactly what a visionary (transformational) leader should be doing;

The Core Values: Where It Usually Goes Wrong

So Scrum does a pretty good job of ironing out of lot of the principles that we discussed. Thanks to organizational psychologists, we now know how to facilitate learning and how important learning is to stay competitive, motivate people and deliver quality. Thanks to Scrum we have a good, applied, framework for doing this. But this is not where the story ends. Most implementations of Scrum will suffer because those doing Scrum have not internalized many of the values that Scrum explicitly or implicitly requires for it to be successful. There is actually a good scientific background for this as well. A few years back, Argyris (2000) reported on a large number of studies that show that learning organizations are very hard to come by. In reality, many organizations try to transform but few actually get there. So what is going wrong? Argyris identified two so-called mental models that are prevalent in organizations. Mental models or theories-in-use as Argyris calls them, drive how we interpret our (work) environment and adjust our behavior. A good example is an organization that tries to become a learning organization, but that rewards individuals for their successes and punishes others for their failures. Explicitly or implicitly, this tells employees that learning is quite risky as mistakes will be punished. Or take a team where the Scrum Master is actually micro-managing most of the work. This reinforces member’s theories that they should just do what they’ve been told and stop when things are unclear, rather than try for themselves.

Model I mental models foster values that inhibit learning and actually reinforce values that actively prevent learning from taking place. Argyris gives a few examples:

• Mechanistic hierarchies: Organizations treat individuals and groups of individuals like machines, or units. They all play their (small) part, but there is no sense of whole or attempts to put focus on the whole;

• Top-down leadership: Management tells individuals what to do, and how to behave and act;

• External commitment: Individuals are committed only to the extend that they like their salary. There is no intrinsic commitment (liking your work, being motivated);

• Emphasis on stability: Organizations and individuals prefer stability and avoid taking risks;

• Loyalty to power: Decisions are often based on appeasing those with power, or gaining more power;

These models, and the values behind them, prevent organizations from becoming true learning organizations. This also applies to individual teams. The organizations that do succeed have individuals that (mostly) work from Model II mental models, which are characterized by:

• Organic structures: Organizations organize their work in a more organic manner. There are no hard separations between departments and units. People cooperate and collaborate and there are very few hierarchical levels;

• Participative leadership: Leaders use a more democratic, involving style of leadership. There is no micro-management;

• Internal commitment: Individuals have high internal commitment. They like their work, are proud of it and are always willing to learn;

• Emphasis on continuous change: Organizations accept that continuous change is necessary;

• Loyalty to valid information and competence: Decisions are based on valid information and competence, rather than going power;

Argyris states that the only way to become a learning organization, is to shift from Model I to Model II mental models. This is very hard (again, sadly), because is requires that organizations analyze and change their core assumptions about quite a few things. This is also why Scrum clearly states that it seems easy, but that it is very hard to implement correctly. Implementing Scrum is not just a matter of playing by the rules, but actually understanding why the rules are important on a deep level. The fact that so many organizations fail to become learning organizations is mirrored on the large number of Scrum implementations that fail to achieve their optimum (which, I admit, is based more on what I’ve read on the net rather than actual solid scientific research – which seems to be quite lacking).

Back to the Workfloor (A Few Tips for the Practically-minded)

OK. This all might sound nice and probably quite theoretical. But how do you facilitate learning, exactly? Sadly, there is no silver bullet. But there are many, many ways to make learning more effective in Scrum. Keep in mind that it’s more about changing your core assumptions (your mental models) than implementing these suggestions. But sometimes it helps to just get on with it and reflect on it as you go. Mental models often change over time, as new experiences are acquired. Thankfully, Scrum allows for this and accepts mistakes (in estimates, planning, etc). Just make sure to understand, at least on a theoretical level, what I have described above as this helps to see the ‘bigger’ picture behind the rules of Scrum.

• Create a safe environment

  Learning will not take place if teams can’t or don’t feel safe to experiment with new ways of organizing and performing their work, solving problems and making decisions. So, make sure to create safe environment. Scrum does a pretty good job of this already by putting down a basic structure for dealing with complex products, a number of timeboxed events and a specific number of roles with defined tasks. But safe environments require more:

  • Safe, but honest: A safe environment does not mean that members can’t critique each other on the way they do their work. Criticism, when given in a constructive manner, is a good way to initiate learning. So don’t nip criticism in the butt and don’t cover everything with the mantle of love. Giving good, constructive criticism is quite a hard thing to learn, so check my other article for more information. It is vitally important for the Scrum Master to make sure that all members respect the rules of constructive criticism, including the Product Owner;

  • Deal with conflicts: Conflicts will occur in any team. Just make sure that the team learns how to deal with conflicts in a constructive manner. This will require mediation skills from those involved, or perhaps from the Scrum Master when he or she is called upon. Again, check my other article;

  • Accept mistakes: A safe environment allows experimentation. This means that mistakes will be made. Rather than punishing those responsible, try to learn from mistakes. It is important to continuously emphasize that mistakes are acceptable, as long as everything is done to prevent them and that mistakes are learning experiences. This does not mean that the same mistake should be made ten times without criticism. Just keep it constructive and search for solutions;

  • Foster autonomy: Very crucial to Scrum, but allow the Development Team and the Scrum Team as a whole to self-organize. Give them the autonomy to make decisions on how to solve a problem;

  • Make people feel valuable members: Even if you’re not using Scrum, this one is quite important. Make sure that all members of a team feel valuable to the team. Give members opportunities to showcase what they can do, and how they did it. Don’t allow members to take step back or be bullied by others. Only if all members feel valued, will they add to the learning process;

• Team building

  Put a lot of effort into shaping and forming a team. If members are proud of their team, they will enjoy their work more, be more motivated and the quality of their work will improve. You could try this:

  • Vision: Take some time to formulate vision with the team. The team can answer questions like ‘What are we proud of?’, ‘What do we want to be proud of?’, ‘Where do we want to be two years from now?’. Make sure that the goals are not to vague. A vision should be a ‘big fat ugly goal’, so it should be ambitious. But it should be concrete enough so that it can be achieved. When teams find this hard, I usually ask them to think of a ‘Development SuperTeam’ and list the characteristics that they think these teams will have in how they do and organize their work. This usually works very well. I like to stick these characteristics on the wall for all to see;

  • Make teams choose a name: Very simple, but incredibly effective. If teams can choose their own name, they will become proud of that name, even if they thought it was weird or stupid at first. Make sure to use the name often, so address the team by their name (I sometimes start the day with ‘Good morning, team X!’), allow the team to put there name somewhere visible and use their name when communicating with the outside world (‘Team X will now present Y’). But don’t forget the individuals in the team :);

  • Give teams something to compete with: The best way to strengthen teams is to give them something to compete with. If you have multiple teams, it might be a good idea to organize some friendly team competition. You could do this with a few PCs and some team-based games, like Unreal Tournament or Modern Warfare. But it’s also nice to organize an actual competition by giving teams a problem to solve, and award the best solution. If you do this, make sure to keep it friendly and that all the teams get something out of it. If you reward only one team, you run the risk of lowering a team’s confidence. Too much competition will also inhibit learning, because teams will not be interested in helping each other out;

  • Do things outside of work: Go somewhere with the team that does not have anything to do with work, at least directly. I once went to a (classical) art museum with my team. I will be the first to admit that I don’t think we’ll do that again. But it was a nice change of scenery and it put the team well outside their comfort zone. It’s always nice to go somewhere inspiring. We went to the Microsoft HQ in the Netherlands once to see how they work. That really inspired our teams;

• Use the retrospective exhaustively

  Don’t settle for a short Sprint Retrospective, even though reflecting is hard and takes time. I like to use this list of topics:

  • Sprint Backlog: Was the selection OK? Can we improve the selection?;

  • Task distribution: Has the team effectively distributed tasks? Have there been any bottlenecks?;

  • Sprint Burndown: Was it useful? Was it updated? Did it motivate?;

  • Impediments: How did the team deal with this? Were they resolved?;

  • Definition of Done: Was it useful? Should it be extended?;

  • Product Owner: How did he/she perform/like the role?;

  • Development Team: How did the team perform/like the role?;

  • Scrum Master: How did the Scrum Master perform the role?;

  • Sprint Planning Meeting: Was it useful? Can it be improved?;

  • Sprint Review Meeting: Was it useful? Can it be improved?;

  • Sprint Retrospective: Was it useful? Can it be improved?;

  • Daily Scrum: Was it useful? Can it be improved?;

  • Working together: Did the team work together effectively? Could this be improved?;

  • Estimates: Are the estimates ok? Should we devise the procedure of estimating?;

  • Process: Was the process efficient? Have there been any bottlenecks?

  • Scrum: Did we like Scrum? Can we improve? Do we understand the core values?;

  • How we learn: How do we learn from mistakes? Can we improve how we learn?;

• Do Team-based code reviews

  I know that not everyone will like this one, but I really like doing code reviews with the entire team. We do this every week during the last hour of the Wednesday. And every week, one of the team members offers some code for the review. While the team member explains the code, the other members give their (constructive) feedback. I always emphasize that feedback should begin with the positive, then move on to improvements. Members usually volunteer code, but sometimes I use the code review to give new (and less experienced) members an opportunity to shine. The code itself might be less interesting from a coding point of view, but simply talking about code quality in itself is very useful. The nice thing about code reviews is that they generate debate and collaborative learning.

• Organize workshops or techtalks

  Regular workshops or techtalks (like Google’s) can give teams an opportunity to learn about new technologies or new ways of working. I try to do this on a monthly basis. The TechTalks are usually free for anyone to join. People can offer suggestions for the talks or organize them themselves. We usually do the talks ourselves, but it is also a good idea to invite outsiders once in a while. Maybe a developer working for another company, or some domain expert. The talks should be practical and can also be used for a more workshop-like approach. It might be a good idea to practice communication skills with each other, for example. Or spend a workshop on Scrum, or team vision or something that is useful.

Conclusion

So there you have it. A firm scientific background for the self-organizing and learning that is so fundamental to any successful Scrum implementation. For those more practically-minded readers, I’ve also added a few pointers that might be helpful. But feel free to add your own tips below this article. There are certainly many more to add, and I will certainly do so in the future.

References:

Argyris, C. (2000), Flawed Advice and the Management Trap. New York: Oxford University Press;
Argyris, C. & D. Schön. (1978), Organisational Learning. Reading, MA. Addison-Wesley;
Koffman, F. & Senge, P. (1993). Communities of commitment: the heart of learning organisation, Organisational Dynamics, Autumn: 5-23;
Rollinson, D. & A. Broadfield. Organisational Behaviour and Analysis, Harlow: Prentice Hall
Senge, P. (1991). The Fifth Discipline: The Art and Practice of the Learning Organisation, New York: Random House;
Takeuchi, H. & I. Nonaka (1986), The New New Product Development Game. Harvard Business Review, 1986 (January-February);

Christiaan Verwijs is a scrum coach, analyst,.NET developer working for @NowOnline. You can read more from Christiaan on his blog.