Fixing Bad Engineering Rituals: Cargo Cults and Chesterton’s Fence
"Waterfall?”
"Nah, we’re Agile."
"Scrum or Kanban?”
"It’s kind of a blend.”
"How so?”
"We do the worst of each and have a retro about it every 2 weeks.”
Agile engineering has won the software development methodology war. Outside of government contractors, big banks, and whatever the hell Valve does, "Agile” has come to dominate as the accepted best way to run a software project. This is not altogether bad: just as "nobody got fired for buying IBM” in the 70s and 80s, no engineering team will get much of a challenge from directors and executives by announcing that they "do a form of agile.” And for good reason: software development is hard because coordination in unbounded knowledge problems is hard. The Mythical Man Month was rejecting the idea that development systems and technology would solve programming back in the 1960s. Picking a methodology, any methodology, and doing it well is superior to attempting to pick the right methodology. Some system is better than no system.
A poorly run system is worse than no system, however, and it’s this state that team members feel most acutely as chaos. When there is no system, teams will adapt to a functional minimum and find a way to get stuff done. When a system is running poorly, there’s an awkward powerlessness through the team and in it’s rituals. Like a first date where both parties have identified that this ain’t it, team members go through the meeting and fulfill the form of the ritual without worrying about the function, content to let the timer run out and they can get back to what they’d rather be doing (be that coding or right swiping to find the next one).
Fixing these rituals is tough, but worth it. Belonging to a team struggling with its process is torture, but when it works the balance of freedom and structure is empowering and a delight to participate in. To begin to breathe life back into a failed process, teams need to consider 2 competing ideas: first, that their rituals don’t do anything, and second, that abandoning their rituals will only make things worse.
Rituals as Cargo Cults
"Cargo Cults” sprung up in the South Pacific in the 1880s through World War 2 as previously isolated island tribes made contact with modern cultures. These contacts brought unthinkable wonders: food in abundance, modern medicine, mass manufactured clothes, and incredible feats of engineering in their ships and planes. Quite logically a religious element emerged in many cultures: new rituals and beliefs to bring more fortune to the island from these strange visitors.
Some of these religious elements were wholesale inventions; others merged existing tribal beliefs into these new encounters. Some tribes began to worship US service members as minor deities. Others emerged from deliberate manipulation by foreign visitors to obtain compliance and labor from the locals. There are apocryphal stories about tribes constructing stone runways and mock airbases to attract the planes of the strange and wealthy visitors. In any case very real rituals formed and attached itself to very fake beliefs.
Too many teams find themselves members of the Agile Cargo Cult. Failing to grasp the original problems and spirit of agile development, teams perform the agile rituals that have no effect on their work. Yet from within the cargo cult, there’s no understanding of the very mundane reality of what’s actually happening in the world around them. There is no feedback loop to realign the team to their goals and deconstruct their operations. The very mechanism agile proposes to prevent against this, the retro, has been fully converted into the cargo cult. The team goes about its business, day after day, week after week, month after month, burning themselves out on meetings, check in, touch bases, stand ups, retros, reviews, endless waves of email and mountains of documents that do not do a single real thing to get a project built and shipped.
Rituals as Chesterton’s Fence
G.K Chesterton was an English writer, philosopher, and literary critic from the turn of the 20th Century. He wrote across political, social, and religious issues. A small-c conservative, Chesterton thought deeply about man, tradition, institutions, and obligations. One of his more famous insights was about, of all things, a fence. I’ll let ChatGPT explain it:
"Chesterton's Fence" is a principle that comes from British writer G.K. Chesterton. It's essentially a metaphor used to illustrate the idea that one should fully understand a system before making changes to it.
The principle gets its name from an analogy Chesterton wrote in his 1929 book "The Thing". He suggested imagining a fence in the middle of a road. The modern reformer comes by and says, "I don't see the use of this; let's clear it away." To this Chesterton responds that if you don't see the use of it, you are the one person who should leave it alone. However, if you can fully explain why the fence was put up in the first place, and why it's no longer needed, then you may be the right person to remove it.
This idea has been used to argue that reform should not be made without understanding why the current system was established, what its purpose is, and what the consequences might be if it is altered. This concept applies to many different areas, including law, culture, and technology, and it encourages thoughtful and informed decision-making.
Chesterton’s Fence is sometimes considered a paradox: you cannot remove the fence until you understand why you cannot remove the fence.
At its heart, Chesterton’s Fence is about respect for practices and institutions even if you do not immediately understand their benefit. Human institutions and practices exist for a reason, and the unseen consequences of change dwarf the assumed benefits. It is a deeply small-c conservative argument: assume things exist for a reason, and must be judged for what it prevents as well as what it creates.
Teams should consider that the ‘fences’ that frustrate them so much are there very good reason. A backlog grooming ritual that decides to skip pointing, to remove timeboxing, or to jettison cumbersome acceptance criteria writing might in fact have been keeping real procedural demons at bay. Instead of viewing a struggling process as an impediment to success, teams should consider the idea that without those painful meetings their productivity and happiness would be worse.
(As an addendum: Tom Wolfe has a meditation on this same effect about diseases in 1960s called The Great Relearning that also demonstrates this point, although about dirty hippies living in communes instead of Agile development practices.)
Team Cargo Fencing
Cargo Fencing (a term I just made up) is the active choice of a team to examine their process using both viewpoints. A team should catalogue their operations and frankly discuss both scenarios. "What if our process is completely out of sync with reality? What meetings are theatre? What ‘stone runways’ are we building?” This will identify time sucks and calendar black holes that are crushing morale and productivity. Next, though, the team should look at their same processes as a fence they don’t appreciate the benefits of. "Why would this meeting exist in the first place? What could go wrong if we removed it? What aren’t we aware of because our process protects us from?”
A team stretching to understand two perspectives at once will develop a deeper understanding of what they are doing and why it matters. By embracing both the Cargo Cult and Chesterton’s Fence mindsets, teams can identify and eliminate ineffective rituals, while preserving the ones that are truly beneficial. The result is a more efficient and effective process that empowers and inspires team members to do their best work.
In the end, it’s not about tossing every inherited ritual overboard, declaring it a cargo cult, nor is it about clinging onto every established practice like a Chesterton's Fence that's off-limits for change. It’s about empowering teams to not just go through the motions but to truly understand their rituals, to critique their own methods, and to figure out if they’re creating value with their actions, or just going through the motions.