This experience report explores the journey of two Scrum Masters as they experimented with different approaches to create impactful retrospectives. Key findings include the identification of five distinct stances that Scrum Masters can adopt, and the importance of balancing creativity and picking of topics by the Scrum Master to achieve meaningful improvements.
1. INTRODUCTION
Ever feel like your worth as a Scrum Master depends on how good your retrospectives are? Ever found yourself making your retrospectives more and more elaborate and enticing to show off your worth as a Scrum Master? If so, you might be doing what we call Retrospective Theatre, where you strive for engagement and novelty, often at the expense of unearthing the real problems.
The scrum guide (Schwaber) says, “the purpose of the Sprint Retrospective is to plan ways to increase quality and effectiveness… The Scrum Team discusses what went well during the Sprint, what problems it encountered and how these problems were (or were not) solved… The most impactful improvements are addressed as soon as possible”. But it doesn’t say what techniques to use or how to facilitate it to make the most impactful improvements, or even who’s responsible for preparing and facilitating individual retrospectives. The Scrum Master’s job is to make sure that all the “Scrum events happen and are positive, productive, and kept within the timebox”.
At Zeiss, the experience has been that the scrum master is only rarely not the person organizing the event and facilitating it. This is perhaps not surprising when our feelings of worth is often linked in others mind to how successfully we run the scrum events are including the retrospectives. This experience report is going to look at the journey of two Scrum Masters at Zeiss as they have tried out different ways to create an environment for impactful retrospectives.
We’ll break it down by showing five stances that, from our experience, Scrum Master’s can take when running retrospectives:
The 5 stances are:
- The Project Manager
- Rookie Scrum Master
- Retromat Master
- Benevolent Dictator
- Equilibrium
We’ll show how these five stances have two key facets. How “fancy” the retrospective is, and how open the topics are (see figure 1). We’ll propose that there’s a balance that can be struck between “fancy” and “open” styles, and that only when you have found this balance can you get the most out of your team’s retrospective and find solutions to the toughest challenges they’re facing. By “fancy” we mean the creative effort that is put into a retrospective to make it as varied or even “wild” as possible. By an open retrospective, we mean a retrospective in which the team defines topics together with the Scrum Master. In contrast, in a closed retrospective, the Scrum Master defines topics in advance.
The stances we’ll cover might sound a bit stereotypical, and to some extent, they are, but we’ve both been in these situations, and we hope you’ll see some of yourself in them too.
Figure 1. Five Stances of a Scrum Master
2. Background
I (Debbie) am a Scrum Master at Carl Zeiss Microscopy, I have over five years of experience in this role within technology-driven organizations, particularly working in close collaboration with hardware and firmware development. I love being able to see the physical products I have helped to develop doing amazing things. Prior to my current position, I have worked in firmware development and software testing, but mostly using waterfall development styles.
I was attracted to the Scrum Master role because it allowed me to combine my passion for technology development with my proficiency in soft skills, I love that I am given the freedom to work in teams that are striving to not only develop cutting edge technology but are also always looking for ways to do it better. On a day-to-day basis, I particularly enjoy being able to employ my facilitation techniques, especially during retrospectives and workshops.
I (Jonathan) got lucky and started working with agility in 2004. We used XP to develop software during an internship at university, and we were able to get pretty good results even though the circumstances were tough. After graduating, I tried pretty much every role there is in software development. I was a developer, requirements engineer, tech lead, and finally project manager on these big waterfall projects. Then I figured out that using the usual project management stuff ever stricter was just causing ever more chaos. That’s when I remembered how I got started with agility. Over the last few years, I’ve worked with all kinds of teams and organizations as a Scrum Master, Product Owner, Agile Coach, and sometimes a mix of these roles. I’m really interested in the formal and informal structures in organizations and how they affect teams and their members.
Carl Zeiss Microscopy Ltd is a subsidiary of the Carl Zeiss AG group, headquartered in Germany, and is a global leader in the fields of optics and optoelectronics. Founded in 1846, the company has built a distinguished reputation for innovation and excellence across various sectors, particularly in microscopy. Within the Microscopy division, the Electron Microscopy product centre specializes in the development of electron microscopes used by multiple industries, including biology, materials science, and semiconductor manufacturing. This report will focus specifically on the research and development (R&D) software development department dedicated to Electron Microscopy.
This department employs a scaled Scrum framework, comprising eight software and firmware teams that operate in joint two-week increments, with two releases scheduled each year. Team members are distributed across two primary locations: the UK and Germany, with some working fully remotely.
Retrospectives are held remotely in individual teams, facilitated by the Scrum Master. There is also a joint retrospective which is held monthly and covers topics which have overarching impact on the teams. Topics that have been covered include ‘how to reduce bugs’, ‘why we did not meet a milestone’, and every 6 months a teams’ health check based on the Spotify health check.
Our SCRUM MASTER JOURNEY IN FIVE ACTS (OR Stances)
2.1 The Project Manager
Before I (Jonathan) started working in the agile realm, I was working as a classical project manager. I was doing kind of retrospectives, but I was calling them something else (Touch-Downs, Post-mortems), had a different focus and they were done too rarely and too late. To be honest, they were dull and boring sessions. I didn’t use any specific facilitation formats other than brainstorming and open, unguided discussions. Since I was the project manager and thought I knew better, I decided ahead of time what the topics would be. The team had no choice but to follow the set topics. Now and then, I’d hit the nail on the head and found a topic that the team also thought was important. But often, the team and I had different ideas about what to talk about, and I kind of forced them into the topics, which led to less engagement.
We’d come up with a long list of ideas for improvement in these meetings, but they were rarely put into action. What system was supposed to make this happen? The projects were usually already finished, and the project team broke up. On top of that, as the project manager, I was the only one responsible for making improvements. The rest of the team was responsible for their day-to-day tasks. So, everything depended on me alone. It was a pretty tight spot, wouldn’t you say?
Looking back, I learned three important lessons that I was able to apply at Zeiss.
- Effective retrospectives need to be based on better formats than just brainstorming and open, unguided discussion. Also, they need at least a basic level of facilitation skills. Unfortunately, as a project manager, I was neither trained in them, nor did I have the awareness of the need.
- Picking topics for a retrospective can be tricky. Sometimes, you’ve just got to pick a topic for the team. You might see a problem the team doesn’t see, or there is a topic which gets “injected” into the team by the organisation or a transformation. But there’s always a trade-off. You don’t get to tap into the team’s creativity, and if you pick the “wrong” topic, you’ll have folks who aren’t as invested in the retrospective.
- To make improvements, you need a stable group of people to work on them. In Scrum, this is normally the Scrum team. This means that you can regularly work on topics with a group, make changes to the work itself, and have a shared sense of responsibility for the improvements.
2.2 Rookie Scrum Master
Transitioning from a project manager type role to a Scrum Master role brings with it a new way of working and thinking. A side effect of working within the Scrum framework is that we now knew we had a mandate to regularly hold retrospectives.
In my (Debbie’s) first Scrum team as the Scrum Master (proudly clutching my new Scrum Master certification), I took this mandate and discovered that facilitating a retrospective was the best part of my new job. I was a Scrum Master to a team of firmware developers, and we were all new to the framework. I started by running the same retrospective every month having made my first handmade Miro retrospective template and I asked the same questions every sprint. I always set up the space in the Miro board one sprint ahead, and everyone on the team would add to it as the sprint went on. We were doing 4-week sprints at the time, so what happened right at the beginning of a sprint could be tricky to remember on the day of the retrospective. But since I kept the template the same every sprint, I didn’t need to worry about explaining it to the team. They knew what to expect and how to write down issues, observations, and wins without me guiding them. So, we never had a blank sheet or blank face when we started a retrospective. We would also get discussion points relating to events occurring throughout the sprint rather than focusing on issues and successes of the last few days.
For a Rookie Scrum Master who’s trying to learn how to support the new team and a team trying to figure out what needs to happen now that we’re “agile,” this was a logical way to start introducing the concept of consistent and regular reflection and feedback. But was it exciting for anyone in the meeting? Was it fixing problems that weren’t as obvious? Were we focusing our actions on long-term improvement? The answer to all of that was no. In fact, longer term, my stencil technique of repeating the same style again and again became less and less effective. Like a stencil, the first few times the retrospective was held it was effective, but the more and more the stencil was used the less impactful they became. The team members would begin to put less and less on the boards ahead of time, and questions would start to be asked if we can reduce the time spent in the retrospective meeting. Participants would be increasingly bored and disengaged and consequently, we had less enthusiasm to take on actions that resulted.
So, looking back at this phase of my Scrum Master journey, I’ve got two main takeaways.
- This format of retrospective could be a good way to begin if your team is also new to the idea of retrospectives. It can help keep things steady while your team adjusts to a new way of working during the rest of the sprint.
- Just don’t fall into the trap of staying in this mode long-term. It’s easy to keep doing this with each sprint. No one has to prepare much, and we don’t have to deal with the really tough problems. It’s the scrum master’s comfort zone.
2.3 The Retromat Master
- As rookie scrum masters, we had managed to get retrospectives on the agenda for our teams. But the results weren’t what we’d hoped for.
- Team members barely participated in the retrospectives.
- After the retrospectives, we realized that the improvements were rarely worked on.
Luckily, we found some good advice in the agile space: Don’t just do your boring “Start – Stop – Continue” retrospective over and over, but mix it up more. We found a bunch of websites that helped us out with our new project. The Miroverse (Miro) and the Retromat (Baldauf), which both have a ton of different formats for running retrospectives.
Don’t get the wrong idea. The Retromat is a great tool for Scrum Masters who are both reflective and talented and want to create awesome retrospectives. But we didn’t use it right. We put together retrospectives from random individual activities. As a result, the team’s actual problem kind of faded into the background. We started focusing on making the retrospective as varied as possible, just for the sake of it. We were going for a kind of “l’art pour l’art” approach. We made something for the sake of making something. Through the exchange in our local Scrum Master Communities of Practices, we had the impression that we were on the right track. As a number of guides also repeatedly emphasise the idea of avoiding boring retrospectives, we also had the feeling that we were expected to design retrospectives that were as creative as possible.
When we looked at our team, the Snow Leopards, we could see some improvements. The retrospectives became a lot more lively and engaged. But if you look a little deeper, things weren’t looking so good.
The team focused on surface-level improvements like “We need to work better together in the sprint,” and they weren’t too keen on committing to more concrete actions through moderation. They did implement some simple improvements, like “We plan the testing of user stories in sprint planning,” but it wasn’t very successful.
We talked to a few team members one-on-one, and it was pretty clear that they felt like they had to do the retrospective because of Scrum and us as Scrum Masters. But they still didn’t really get what their role was in this setup. But they were more willing to participate because we made the retrospectives more exciting and fun. Basically, we masked a team dynamic issue by doing our retrospectives the way we did.
But overall, it felt good for us Scrum Masters too. At least, we had the feeling we were showing, that we were doing something for the team with our creative retrospectives. At least, that’s what people who came to our retrospectives told us. We were doing something that was clearly visible to management, and people were enjoying, what we were doing. If only there hadn’t been that uneasy feeling of not achieving any real improvements…
2.4 Benevolent Dictator
After having perhaps, a little too much fun playing the Retromat Master role, I (Debbie) found I was slowly moving towards the Benevolent Dictator role during the retrospective. I’d been working as a scrum master for a while, and my teams had come to expect something fun and exciting for their retrospectives. But at the same time, I was starting to wonder if they really worked. Sure, they were fun and a great way to improve my facilitation skills, but I wasn’t sure if they could actually solve the big problems or if we were avoiding even talking about these issues.
At the same time, I was feeling this urge to dig deeper into overarching issues that weren’t just impacting my teams, and I was given the chance to try out a new kind of retrospective and put my retrospective facilitation skills to the test. I took on the challenge of running the Joint Retrospective. When I joined Zeiss, there was a retrospective where all the developers, testers, POs, and Scrum Masters from all the teams working on one of the software platforms went, and they followed the typical “what went well. What went badly, format.” As is often the case for a meeting with many people (usually over 50 people), only a few people were ever brave enough to speak up. So, the challenge was basically how to make sure everyone’s voice was heard on the big issues that affected more than one team.
There were several changes that I made to improve how the Joint Retrospective ran:
- Less talking: This might seem counterintuitive, but it’s not possible to hear 50 voices in a 1-hour meeting. So, we used Miro Stickies and grouping and lots of voting rather than asking.
- Pick a topic: This is where I stepped in as the benevolent dictator. Instead of asking a big group what’s bothering them, I asked the Scrum Masters from the teams for ideas about what’s come up in their team’s retrospectives. Using this I could then focus on one key issue at a time so have time to dig a little deeper.
- Make it individualized but follow a format: I started with an icebreaker, introduced the topic with a bit of theory around the theme, and then launched into individually tailored retrospective questions. By having a basic framework, attendees would know what to expect but still give me the freedom to shape the Scrum event around the key issue.
This type of retrospective was well received, and it got a lot of ideas out there. We were able to dig deeper into topics and get a broader view of the impact it was having. It let people who don’t usually speak up find a comfortable way to be heard. But there were 2 major downsides to this type of retrospective. Firstly, the audience didn’t directly choose the topic and so if the focus was picked poorly then engagement would be low. Secondly, although I had managed to facilitate in such a way that all the attendees voices could be heard, I struggled with getting individuals to take on actions that came out of the session. The attendee usually assumed the facilitator (mostly me) would go away and make the changes on their behalf. But with monthly joint retrospectives and huge overarching changes being asked for, it became unmanageable very quickly. If the attendees can’t see any quick improvements, it quickly seems like a waste of time. There was also a bystander effect, where when an action was for “all developers” or “all testers,” everyone thought someone else would take it on or that it applied to others only. Since it was impossible to follow up with everyone, there was no accountability or pressure to make the change.
Reflecting on my time shepherding the Joint Retrospective, my main lessons learnt were that when I preassigned topics for a retrospective, I was able to gain good feedback but not good follow through of the actions. The less autonomy there is in deciding what to talk about, the less ownership is felt to fix the issue. By being a Benevolent Dictator, I had become very effective at getting engagement to highlight problems, but I was not getting the true value of a retrospective by getting everyone actively continuously improving.
2.5 Equilibrium
We saw some improvements, but we weren’t really happy with the results of our retrospectives. An important impulse for us to change this situation was the “Conditions for Team Effectiveness” model by Hackman, which describes the basic conditions for teamwork:
“Specifically, the likelihood of effectiveness is increased when a team (1) is a real team rather than a team in name only, (2) has a compelling direction for its work, (3) has an enabling structure that facilitates rather than impedes teamwork, (4) operates within a supportive organizational context, and (5) has available ample expert coaching in teamwork.” (Hackman)
We quickly realized that conditions 1 and 2 weren’t met in our context. The team members saw themselves as lone wolves working together, not as a group that depends on each other. There wasn’t a clear shared goal, either.
The way we’d previously moderated our retrospectives had unconsciously masked these problems: We tried to motivate the team members by using the most creative formats possible, but the team members hadn’t yet understood that it was their responsibility to work together on problems. By being more specific about the topics, we actually made them responsible for solving problems on their own, instead of working together as a team.
The first step was to not go after the lone wolf problem directly, but to start with a clear common goal. We figured this behaviour was culturally ingrained and seen as efficient from a Taylorist point of view. But a simple appeal to finally see ourselves as a real team wouldn’t have done much here. But we did get management to commit in advance to working in real teams, even if it meant short-term inefficiencies. Luckily, the team’s main goal was pretty clear: a big product release was just around the corner. But the team members hadn’t realized that this was their big common goal (and not just checking user story after user story into the Git repository). So, I (Jonathan) started the next retrospective with a simple scale question:
Figure 2. Scale question as an introduction to retro
The result was pretty clear: everyone on the team thought the release would be really hard. I was able to back this up with metrics like release burnup and team velocity. Then, I talked with the team about what we needed to change about the way we work to improve our chances of a successful Release. I also asked where each team member depended on the others. After talking about it, the team got a better idea of the common problem they had and why they needed to work together as a real team to solve it. For example, our “Snow Leopard” team realized that their main problem wasn’t actually coding. They spent a lot of time on testing because the tester on the team was pretty much flying solo. The team therefore decided that from now on, each team member should also invest a certain amount of time in testing during the sprint.
We weren’t quite where we wanted to be yet. During the sprint, it became clear that the software developers were testing occasionally, but essentially continuing to work as before. At this point, we realized another big mistake we made as Scrum Masters: Even though we were working on the product in an iterative way, we weren’t doing the same with our impediments. We thought that these could be solved with a single retro. As a result, we had two major realizations.
- If you need to, work on problems over a few retrospectives. Don’t act like each retro is the first one the team has ever had.
- Keep the retrospectives focused on the content, so the team can get into the nitty-gritty without having to learn a new format every time. That’ll keep everyone on track.
With Ralf Kruse, we then found a format for retrospectives that fulfils precisely these requirements (Kruse). The format basically follows the typical five phases of a retrospective (Derby & Larsen).
Figure 3. Ralf Kruse’s Retro Format
In “Setting the Stage,” we’ll take a moment as a team to discuss what’s been going on since our last check-in. Have we worked on our action items at all? Have they solved anything about our problem? Are we even looking at the right problem? Depending on the result of this phase, we’ll either stop collecting new problems again or work on the last retro again. Does this somehow sound familiar to you? It’s nothing else than applying the concept of WIP-Limits, but on improvement items themselves.
However, if we talk about new topics, we vary the formats in the “Gather Data” step. I have become a very big fan of scaling questions, as mentioned in the example above. But this is exactly where the Retromat can come in handy.
In the “Generate Insight” section, we mix it up a bit from the usual way of doing things. Instead of using dot voting to decide on topics, we let team members suggest topics and then take responsibility for them. It’s actually pretty simple to put a dot (especially online) in the right place. But, let’s be real, standing up in front of the team and taking charge of a topic is a whole other ball game. Usually, we find a bunch of topics and people to work on them, and then we send them to breakouts. In the breakouts, we use a pretty simple and recurring format. The team members are asked to discuss three questions:
- What is the problem?
- What is the cause?
- What are concrete actions to solve the problem?
From what we’ve seen, the teams come up with solid ideas during this step, but sometimes they’re too big (they can’t be done in a sprint) or too vague. So, in “Decide what to do,” we send the teams back into a breakout session for a bit to solidify their action item so they can tackle it in the next sprint.
For the last step, “Close the Retrospective,” there can be a little more variation and the Retromat can be used. I usually just do a short “retro over the retro” myself.
For our “Snow Leopards” team, this was the key to making a breakthrough. For the first time, the team had a very honest discussion about how they could solve their “test problem” together (and not just the poor tester alone). From here on out, one team member was there to help out the tester. Then, we did a bunch more retrospectives to make the solution even better.
3. What We Learned
This isn’t the end of our learning journey. Of course, we wish we could have skipped some of the stances of our journey, but perhaps these are simply necessary steps we had to go through. Right now, we’re trying to figure out how to apply our findings to the Joint Retrospectives to make effective improvements on a large scale.
Here are the main things we learned from this trip:
- If your team doesn’t have the right conditions, retrospectives are only of limited value. Doing them is of course still better, than just skipping them, but you are not going to have the impact, that you would like to have. Since a lot of places still have a Tayloristic mindset, you can’t assume your team really sees itself as a team and is responsible for improving.
- If you use overly creative formats or always pick topics in advance, you’ll just mask the problems we’ve talked about. If the conditions are good, you can get way more done with a simple and stable format for retrospectives than with super creative formats in an environment that’s not a good fit.
- As the Scrum Master, you are responsible for the improvement process, but the entire Scrum team is accountable for the improvements themselves. Include your team in this accountability!
At the end of the day, the most important thing is that you can see where you’re at as a Scrum Master and make changes. Some days, your team might need a bit of the Benevolent Dictator, while other days might call for more of the Rookie Scrum Master (sic!). It’s important to be aware of where you’re at right now and be prepared to adapt!
4. Acknowledgements
Zeiss and BettercallPaul supported the creation of this paper. We’d especially like to thank Christoph Graf vom Hagen, David Hubbard, Ralph Pulwey, Michael Dirksmöller, and Benjamin Weinhold. And we can’t forget Joelle Garden, who had the idea to bring us together for this paper.
I (Jonathan) would like to thank my “Agile Partners in Crime”: Ralf Kruse, Julia Jonas, Henning Möller, Jutta Bechtloff, and Henrike Frey. Your mentoring, sparring, and sometimes not-so-nice impulses have helped me a lot! And a big thanks to Lisa März for the awesome graphics!
I (Debbie) would like to thank Joelle Garden and Andrea Sigl for always pushing me to do things outside my comfort zone. Thank you for helping me to strive for constant improvements in all things I tackle.
Finally, we couldn’t have made it without our Shepherd Derk-Jan De Grood! Your insights really helped improve the paper, and we truly appreciated your support.
REFERENCES
Baldauf, Corinna, https://retromat.org/en
Derby, Esther & Larsen, Diana, “Agile Retrospectives – Making Good Teams Great”, 2006, pp. 4-14
Hackman, J. Richard, “Leading Teams – Setting the stage for great performances”, 2002, pp. 31-33
Kruse, Ralf, Scrum Meistern Podcast, https://enablechange.de/scrum/sprint-retrospektive/effektiv-gestalten/
Miro, https://miro.com/miroverse/
Schwaber, Ken & Sutherland Jeff “Scrum Guide” 2020
