My last post covered the first product team anti-pattern of this four-part series: (fend-for-your)self-onboarding. I explained how a “self-onboarding” process that focusses on working independently and evaluates fit based on onboarding task completion can be detrimental to both the new team member as well as the whole team.
This post covers another anti-pattern that can also deceive the product team into thinking that it’s operating effectively. When this anti-pattern manifests, the team sees itself as passionate, committed, and ultimately effective “in the face of adversity”.
The fact that this “adversity” is self-inflicted is the first hint that something is off.
Having spent around half of my professional software engineering career working at startups, I have experienced my fair share of heroism. Both as the hero, as well as the helpless stakeholder-in-distress.
The best way to explain what heroism looks like, and why it’s harmful to the team and the product in the long-term, is by means of a short fable. While the fable as a whole is fictitious, the story’s elements are not. I have experienced, first-hand, all of the behaviours covered below.
The product team hero: a short fable
Gillian is the hero of this story. She’s a dedicated, passionate, team player who is committed to doing whatever it takes to meet and exceed expectations as an ambitious software engineer.
The story starts with a scheduled public release. While the deadline looms, the work continues to pile up. The team falls behind schedule. Gillian pulls a few late nights and manages to ensure that the work is done on time.
Finally, the team agrees that the release is ready for public launch, and the flags are flipped. The team celebrates late into the night, and everyone eventually heads home.
Curious about how the new release is going, Gillian checks out some of the app’s performance metrics on her phone, just before bed. To her dismay, there are over a thousand errors reported in Sentry!
She immediately sits up in bed, cracks open her laptop, and begins troubleshooting. She decides not to alert her team yet. “Maybe I can fix this myself and spare them from having to work during off-hours”, she says to herself.
After several hours of intensely-focused debugging, she finds the issue. It turned out easy to fix and deploy — it was a misconfiguration that she mistakenly introduced during one of her sleep-deprived late nights getting the release back on schedule. She feels embarrassed. “Well, at least I was able to fix it without waking up my teammates”, she reflects.
The next day, Gillian receives a pubic kudos from the Team Lead — he noticed the late-night commits and hot-fix push to production. Her all-nighter did not go unrecognized! The embarrassment she felt last night fades away, and is replaced with a sense of accomplishment and pride. She promises herself that she’ll continue to work hard and to be there for her team, no matter what.
Consequently, the team leans more and more on Gillian; with her passion and dedication, she becomes the go-to expert on the codebase and the domain. Several months and releases pass with their fair share of bumps in the road. Fortunately through it all, Gillian is always there to save the day.
Until one day, she isn’t.
One morning, the Team Lead gathers everyone together for an important announcement. Gillian needs to take a leave of absence due to burnout.
“We take burnout very seriously here, so we’re giving her all the time she needs to recover” the Team Lead continues. “I don’t need to tell you all how important Gillian is to the team. I need all of you to step-up over the next while, especially until Gillian comes back.” No one’s surprised by this announcement, but murmurs of empathy and support echo through the team.
Without their hero, the team struggles to meet release deadlines. There are some areas of the codebase that the team does not want to touch until Gillian comes back. Consequently, larger changes to the app are deferred or avoided altogether. The technical debt builds up, and the designer gets frustrated that, one after another, their ambitious UX redesigns are rejected by the team. The team assures itself that it’ll get better once Gillian gets back from her time off.
Except that she doesn’t. During her time off, Gillian is able to gain some valuable perspective. She realizes that her unofficial “hero” designation is detrimental to her long-term psychological and physical health. Not seeing a path to transition out of that role at her current company, she submits her notice, much to the dismay of her team.
Epilogue
The team kept going. Over time, more people were hired while others left. Yet there was always someone who ended up with the understood yet unstated role of “hero”. Thus, the “hero culture” persisted on the team — this left a wake of poorly-understood code that was increasingly-difficult to maintain and extend. Team members stopped volunteering to be the hero, so the team began to recruit for the role. Due to the accumulated product and technical debt, both the product and team lost its lustre, and recruiting talented practitioners to the team became impossible.
How could this story end? It depends. The product itself may end up getting shuttered, or a new team may get created to rebuild a “version 2.0” from scratch. Either outcome would be a sad end to a product team that should have been capable of growing sustainably and indefinitely over time.
Summary and analysis
Let’s start by calling out some key events and behaviours:
Gillian soloing late into the night to ensure the release stays on schedule
Gillian pulling an all-nighter to debug and fix a production issue introduced by the release
The Team Lead praising Gillian’s late-night efforts and initiative
The team relying on Gillian to be the subject matter expert on many critical areas of the codebase
Each of the above made the team progressively more reliant on Gillian. What could have been done instead, to shift the reliance back onto the team? Let’s go through some possibilities.
1. Avoid “soloing” to move faster
Rather than negotiating a team’s practices to meet a deadline, negotiate scope instead. This may involve the whole team working as an ensemble to determine what to de-scope for the release.
For teams who do not have full authority to define their own scope, the Product Manager is typically the liaison between the team and the stakeholders outside of the team. This is why it’s important to include them in the ensemble scoping sessions.
2. Avoid “soloing” during a critical production incident
This could’ve gone much worse for Gillian and the team — if there was another team member who also decided to work on the same issue without bothering the other team members, it could lead to the team members overriding each others’ work and leading to significant delays in getting the hot-fix out.
Typically, teams define an incident-management process that ensures clear responsibilities and tight collaboration, usually via ensemble programming.
3. Run a blameless post-mortem after every production incident
While praise for Gillian was definitely well-deserved, the Team Lead should not have stopped there. He should have ensured that a post-mortem was scheduled, with actionable insights generated that were followed through on.
Usually, team members gravitate towards their routine behaviour of knocking work off of the backlog. It’s therefore up to the Team Lead to ensure that the underlying causes of the incident are determined, and changes are made to prevent it from happening again. This is done by committing to and following through on a post-mortem, even when it feels like a distraction from regular work.
4. Avoid knowledge silos via regular pair-rotation
It’s expected and beneficial for each team member to bring their own set of distinct skills and experience to the team. This is often misunderstood as an acceptability around certain team members having more knowledge of certain areas of the codebase than others. The latter situation is is harmful to team sustainability, as portrayed in the above story.
Regular (i.e. at least daily) pair-rotation can effectively address this. While there can be resistance to this as it can be seen as hampering momentum, the velocity gains from maintaining the same pairings for weeks-on-end are short-term and come at the expense of team sustainability. Therefore, regular pair-rotation should be encouraged.
Conclusion
To be clear, there is nothing wrong with “heroic” acts by the team — just the reliance of a singular “hero” on the team to save the day whenever the team is in trouble.
To summarize my recommendation — avoid relying on one team member considerably more than you would rely on another, and certainly don’t glorify such an arrangement, as it harms both the individual being relied on, as well as the team as a whole.