Flow in Software Engineering
Should we avoid it or embrace it?
Many Software Engineers love Flow, or being “in the zone”. I confess, I relish it. I can stay focused for hours. While in a Flow state, I can hammer through tough problems and get a ton of work done. It feels awesome. I remember back in my startup days, I’d stay in the zone all day and forget to eat. Despite ending the day hungry, I had a sense of fulfillment that went beyond what I had accomplished (though occasional accomplishments never hurt). There was something intrinsically rewarding about being in a state of Flow.
If you have any experience with computer programming, you can probably relate to this. On the other hand, if you have experience working with other Software Engineers, you may have been warned against it. Harking back to my startup years, I can remember times when I worked late into the night alone on the codebase, only to have it rewritten by other members of the software engineering team once other factors were considered.
The case against Flow
Robert “Uncle Bob” Martin makes the case against precisely the kind of behaviour I described above in his book, “Clean Coder:”
Much has been written about the hyper-productive state known as “flow.” Some programmers call it “the Zone.” Whatever it is called, you are probably familiar with it. It is the highly focused, tunnel-vision state of consciousness that programmers can get into while they write code. In this state they feel productive. In this state they feel infallible. And so they desire to attain that state, and often measure their self-worth by how much time they can spend there.
Here’s a little hint from someone whose been there and back: Avoid the Zone. This state of consciousness is not really hyper-productive and is certainly not infallible. It’s really just a mild meditative state in which certain rational faculties are diminished in favour of a sense of speed.
The problem is that you lose some of the big picture while you are in the Zone, so you will likely make decisions that you will later have to go back and reverse. Code written in the Zone may come out faster, but you’ll be going back to visit it more.
Up until this point, I agree with Uncle Bob. The following paragraph, however, is where our opinions diverge:
One of the big benefits of pair programming is that it is virtually impossible for a pair to enter the Zone. The Zone is an uncommunicative state, while pairing requires intense and constant communication. Indeed, one of the complaints I often hear about pairing is that it blocks entry into the Zone. Good! The Zone is not where you want to be.
Having practiced pair programming for over eight years now, I’ve found that Flow states are indeed possible to achieve while pairing. They feel different from the Flow states experienced when solo programming, but who’s to say that Flow has a particular feeling, anyway?
As it turns out, there is someone who’s had a whole lot to say about Flow. He also happens to be the one who coined the term.
What is flow
The late Mihaly Csikszentmihalyi has studied and spoken on the topic of Flow for several decades, having published his first paper on the topic in 1975. He is, however, best known for his seminal book on the topic: Flow: The Psychology of Optimal Experience (first published in 1990).
At its core, the book makes the case that many human experiences can be transformed into Flow experiences, and one should endeavour to do so, as it brings meaningful and lasting enjoyment to one’s life. Through sampling and analyzing the lives of experts across various disciplines such as sports, music, art, and many others, Csikszentmihalyi determines that these individuals are at their happiest and most fulfilled during periods of intense concentration on tasks involving high degrees of skill to perform.
Through metaphor, his interview subjects described feeling caught up in a flowing current which carried them along as they progressed through their work. Consequently, Csikszentmihalyi arrived at “Flow” as a unifying term to describe these experiences.
Eight elements of Flow
Further, Csikszentmihalyi identifies eight elements that are common across all Flow states, regardless of level of expertise, form of behaviour, or type of discipline.
Four conditions
The first four elements can be thought of as the conditions for a Flow state to entered into and sustained over time. They’re summarized below:
A Challenge Requiring Skills: The task being undertaken is challenging in a way that leverages our skills. The difficulty level of the task must be just right, near the limits of our abilities. If the difficulty is too high, we get anxious; if it’s too low, we get bored. The author describes the non-flow states of "anxiety” and “boredom” as “psychic entropy”.
Conditions for Focus: If we are to perform near the limits of our abilities, we need to concentrate on the task at hand. Consequently, we require the conditions that allow for this — a well-rested self and a distraction-free environment are just some examples of what we’d need to enter into Flow in the first place.
Clear Goals: In order to start taking steps forward, we must have a clear destination in mind. Goals that facilitate a Flow state are ones that can be achieved quickly, yet still pose a meaningful challenge. This is related to the first point: Flow states are invoked by our psyche when we perform near the limit of our abilities.
Continuous Feedback: Effective feedback in Flow doesn’t just tell us whether a goal has been achieved — it also illuminates our progress to achieving that goal. The most effective and consistent Flow states observed by Csikszentmihalyi happened when the feedback was continuous. Here’s one example that he mentions in his book: as surgeons operate, they assess how their surgery is going in real-time by continually observing the amount of blood in the patient’s cavity.
Four characteristics
The next four elements can be thought of as distinct characteristics observed once a flow state is entered into. They serve to reinforce the first four conditions described above. The four characteristics of flow are as follows:
Effortless Involvement: While creating the conditions for concentration can be challenging, something interesting happens once we enter into a Flow state: our attention becomes completely absorbed by the activity. At this point, we find it difficult to switch contexts or even get distracted. On the contrary, we find that we need to pull ourselves away — almost by force — when our attention is needed elsewhere.
Confidence of Control: Expert athletes and artists in Flow perform dangerous feats requiring high degrees of skill with extremely narrow margins for error. At the same time, these same performers describe their experience as a feeling of complete control. Csikszentmihalyi describes this as a “paradox of control” as there are still possibilities for negative outcomes — even if the performer does everything right, an external factor may cause an injury, or they may simply lose against a competing performer. Essentially, this characteristic can be thought of as experiencing an irrationally-high feeling of confidence in one’s own abilities while in Flow.
The Loss of Self-Concern: For many of us, self-conscious thoughts nag at us continually as we go about the mundanities of daily life. If we’re alone, we question and second-guess our motivations, intentions, and behaviours. If we’re with others, we speculate on how we’re perceived and what others think of us. By necessity, these self-conscious thought processes evaporate during Flow. The necessity arises from the mind having to dedicate itself fully to Flow state activities. As these activities push us to the limits of our abilities, there is no room for self-conscious streams of thought to manifest and develop.
Transformed Sense of Time: Flow states can alter an individual’s sense of time. We can get so engrossed in a task that we are shocked when we get a moment to glance at our watch — “Is that the time already?” It’s also possible for the opposite to happen, however. Csikszentmihalyi describes how an elaborate Ballet move requiring a high degree of concentration can feel like taking minutes to perform to the dancer, yet they actually complete the move in under a second. In summary, Flow states can significantly transform and even impair our sense of time.
The autotelic experience
In a sea of competing demands for our attention, time, and energy, what hope do we have for Flow to be practiced consistently? Fortunately, there is an underlying aspect of Flow that compels us to return to it: the autotelic experience. Csikszentmihalyi describes this as follows:
The term “autotelic” derives from two Greek words, auto meaning self, and telos meaning goal. It refers to a self-contained activity, one that is done not with the expectation of some future benefit, but simply because the doing itself is the reward.
The author goes on further to clarify that most behaviour is neither purely autotelic nor exotelic:
Surgeons usually enter into their long period of training because of exotelic expectations: to help people, to make money, to achieve prestige. If they are lucky, after a while they begin to enjoy their work, and then surgery becomes to a large extent also autotelic.
At first, we may be extrinsically motivated to complete certain tasks. As we are driven to complete more of these tasks, we put effort towards building the skills needed to complete them. As our skills grow and Flow states deepen, practicing these skills becomes increasingly autotelic — rewarding in and of itself.
Wrap-up
Mihaly Csikszentmihalyi provides us with a well-structured definition of Flow via its eight elements. He also demonstrates how Flow states can be observed across a vast array of activities and disciplines.
Indeed, there are a myriad of human activities that could be enriched thanks to the phenomenon of Flow. I believe that pair programming in Flow is not just a real possibility, but one that is realized more often than we give credence to.
I will back up this claim in my next post, where I describe how the eight components of Flow can be observed during a pairing session. I’ll also cover approaches that can be used to sustain Flow in pair programming.