Conducting Program/Release retrospectives

February 7, 2024

Technical programme management and product execution is a practice. We develop our skills by doing the work regularly and repeatedly: one project, one feature, one service at a time. All of us know that to improve, we need to learn from our previous experience and build on from there. Each framework, method, playbook in execution has in its core ways a principle to inspect and adapt. Here’s a framework that I refined over time to conduct these release retrospectives effectively. Original inspiration comes from: Agile Retrospectives by Esther D and Diana L.

I first outline the difference between a release retrospective and a Scrum retrospective. Secondly, I move on to outlining the framework, with detailed steps that I follow for conducting a cross-team release retrospective along with a template that you can use to document the outcomes and actions. Finally, I will share a metadata structure for tagging/storing the retrospective document so that you can easily refer to it later if a similar project/release/feature is going to be developed.

Let’s dive in!

What’s the Difference between a release retro, lessons learnt, sprint retro? #

Release/Project Retrospectives: Focus on a broader scope, encompassing the entire release cycle from planning to execution and post-launch support (aka warranty/hypercare). They involve a larger group of stakeholders, including cross-functional teams, development, operations, and external partners as needed. Often Product teams could represent these “external” colleagues’ views, but I have found that it’s often useful to have them included to learn from them directly.
Lessons Learnt: As the name suggests, these workshops tend to focus on what went wrong and what we can learn from those mistakes. I find these a bit negative. I like the word “retrospective” as it helps broaden our purview and invites acknowledging wins as well as lessons. It’s as important to build on why something successful as we delve in why something didn’t go as planned.
Sprint Retrospectives: Address shorter iterations within a release, typically lasting 1-4 weeks. They concentrate on team dynamics, working practices, and improvement within the specific sprint, involving the immediate development team.

Conducting a release retrospective – a “tried and tested” framework #

The framework below can be – to be honest – applied to any brainstorming exercise. I have used it seek ideas on how we can go remote working, to determine the objectives for a programme, and so on. The framework uses a cycle that allows for learnings from cognitive psychology and how our brains work, especially in a team environment. During a release retrospective when there might be many stakeholders, I found this approach much more useful than a more simplistic, “what went well, what didn’t go well, actions” framework.

Prep: Release retros often involve cross-functional teams, stakeholders, DevOps, Service, and product tech support teams. It’s many people, so the more off-line work you can do, the better. Have face-to-face sessions, or surveys if remote, to get a sense of the key themes that are emerging. Read the room before you get everyone involved. This would enable you to prep the workshop effectively and have some volunteers recruited accordingly.
Temp check: As people join the retro, set the scene, highlight the overall objective of the release, flash back on the key events upto the hypercare, provide factual details as to what transpired. Jog the memory, but steer clear of any judgemental or subjective commentary. Here, we’re enabling folk to warm up to sharing their thoughts without steering them to one direction. Be brief.
Gather data: Ask one rep from each team/discipline to represent their team to post their comments on what went well, what didn’t go well, what could have been different, and so on. Keep it time bound. This is a therapeutic exercise of just emptying the proverbial bag on the table.
Generate themes: Go by each post-it/comment/note into themes. These could be about ways of working, documentation, tooling, project phase: let the data guide you, but your pre-work should enable you to form certain themes beforehand.
Prioritise the themes: Now do a voting exercise to identify which theme you would want to focus on. For eg. if the emergent themes were project phases, and you have more tickets/pressing items against “inception” phase, you could choose that. Or if there’s a theme around “system design/solution architecture” theme, you can focus on that.
Identify solutions/learnings, actions, owners: Now that you’re focusing on the theme, it’s finally the time to discuss what has gone on in that theme. As I mentioned above, balance between the good and the not-so-good. Aim is here to distil the number of tickets into actionable learnings. I often found that the entire theme of double-digit post-its yield a couple of key learnings or takeaways. For each takeaway, craft a statement that is actionable – what’s that you’d do because of that discussion.
Repeat Step 6 for other themes.
Closing the Retrospective: Recap key takeaways, express gratitude, and outline follow-up actions.

Sample Activities for Each Stage: #

Prep & Temp Check:

Icebreaker game to build rapport.
Review release timeline and milestones.
Share anonymous pre-retrospective questionnaires.

Generating Data:

Prompt with some open-ended questions.
Utilize voting techniques (e.g., dot voting) to prioritize themes of issues/ideas/options.
Conduct retrospective surveys with Likert scale questions.

Generating Insights:

Group findings from various data sources into themes and look at one theme at a time to identify system forces in play.
Use root cause analysis tools (e.g., 5 Whys) to drill down to underlying issues.
Facilitate discussions to identify patterns and connections.

Deciding Next Steps:

Brainstorm potential solutions collaboratively.
Utilize SMART criteria to refine action items (Specific, Measurable, Achievable, Relevant, Time-bound).
Establish clear ownership and deadlines for each action.

Preparation Activities:

Share the agenda and template in advance.
Encourage participants to complete pre-retrospective questionnaires.
Gather relevant data and metrics about the release.
Consider inviting an external facilitator for large or complex releases.

Best Practices for Leading and Managing Retrospectives: #

Stay neutral: Avoid expressing personal opinions or judgments.
Actively listen: Encourage participation and ensure everyone feels heard.
Focus on the future: Guide discussions towards actionable solutions and improvements.
Celebrate successes: Acknowledge achievements alongside areas for improvement.
Keep it concise: Aim for a focused session within a pre-defined timeframe.

What about a repository of the learnings/retrospectives? #

How do we organise them for easy and effective retrieval?
Once the retro is completed, use a SharePoint library or a common repository to store the retros using the form below. This will enable you or a new Prg Manager to retrieve the learnings and successes easily.

Metadata:

Project Type: Categorize based on project theme (e.g., data migration, new feature launch, system upgrade).
Technology Stack: List the tools and platforms used in the project.
Team Size & Composition: Capture the number of people involved and their roles.
Release Scope & Timeframe: Specify the scale and duration of the project.
Key Objectives & Goals: Define the primary aims and success metrics.

Additional Parameters:

Success Rating: Allow rating past projects based on achieving objectives (e.g., high, medium, low).
Cost & Resource Usage: Capture project budget and resource allocation for cost comparisons.
Stakeholder Satisfaction: Consider surveys or ratings to gauge external perception.
External Links & Resources: Include relevant documentation, reports, and external resources associated with the project.