Reasons Most Transformations Fail

Why have so many Agile transformations failed altogether, or feigned success for a time, but failed to deliver measurable improvements in effectiveness? There are several common reasons (that are not mutually exclusive), and sometimes, organizations find unique and innovative failure patterns. Bain recently reported that 88% of Agile and Digital transformations fail. Paraphrasing Tolstoy: “Every failed transformation is alike, but the really spectacular disasters invent ways to fail that are all their own.”

Agile Theater

Often, a CIO will undertake a transformation to appear to be up-to-date with trends and competitors. A board member asks: “I read about this big Agile trend in Forbes. Is our IT department doing it?” If the CIO’s goal for Agile “transformation” is to present the appearance that IT is embracing a new trend, the lack of focus on outcomes guarantees failure. A desire to appear “on trend” will not drive the deeper organizational changes that produce the benefits of Agile.

Evidence of this Agile theater approach might include doing superficial activities like lots of training classes and getting lots of people certified, but not really making the investment required to help employees adopt new ways of working. Another sign is that people are retitled, and development teams start attending a lot of new meetings, but most of the old processes and all the legacy structures remain. If the legacy IT resource management model and organizing work into projects persist, no meaningful improvement in delivery will be achieved.

How do you change the culture of the organization? There are a few levers, but they all pale in importance compared to the impact your leadership style has on culture.

The solution to Agile theater is to ground your Agile initiative in concrete performance objectives, ideally expressed as OKRs, to drive continuous improvement of development outcomes and capabilities. It’s important to get focus and alignment on the “why”, and clarity on what the outcomes are is paramount. There are many attractive benefits that can be realized by development organizations becoming agile: shorter time to market, higher quality, increased responsiveness, higher productivity, happier employees, and more. However, a well-managed transformation entails deliberate and thoughtful change management to include a clear chartering of the desired outcomes and measures for the Agile initiative. It is NOT possible to optimize a change program to deliver every possible Agile benefit equally. Focus on one or two of the most impactful ones for your unique organization, track progress, and report benefits. Achieve accountability for real results that drive the success of the business according to your aligned OKRs.

Ignoring Culture and Leadership Styles

Peter Drucker famously said: “Culture eats strategy for breakfast.” If strategy is breakfast, process is barely a light snack for culture to devour. Culture, though invisible, is tangible and packs an overwhelmingly powerful influence over the choices and decisions your employees make every day. Your employees know your organization’s culture by what it (and leaders who set the culture intentionally and unintentionally) are willing to tolerate. Culture is defined by what is formally and informally rewarded and reinforced in the organization. Your culture is constantly evolving based on what leaders demonstrate that they value through their behaviors and reactions every day.

Do leaders cling to the appearance of omniscience or reflect humility and openness to being wrong? Do risk takers get punished when they try something that fails, or are they rewarded for being creative? Are brash “lone wolf” misanthropes celebrated for saving the day with heroics (that should not have been necessary), or do you celebrate team builders who reinforce group collaboration and cooperation to anticipate and avoid problems?

Many times, I have heard senior leaders in corporations exclaim some variant of: “I am so frustrated trying to make my employees become empowered and accountable!” Usually, he is saying this minutes after exiting a meeting where he excoriated someone for trying something that didn’t work out, took personal control of every decision without sharing his thinking process, and never shared the principles by which, in his view, decisions should be guided.

The Agile ghetto will struggle mightily, a lovely walled garden in the midst of a hideously static, slow-moving swamp.

How do you change the culture of the organization? There are a few levers, but they all pale in importance compared to the impact your leadership style has on culture. How do you show up as a leader? What are you radiating to the organization every day about what you value most? Are you focused on control? Do you relish being the organization’s ultimate expert, making decisions on everything? Agile leadership focuses on developing organizational capability, and first and foremost, that is accomplished through developing people.

The Agile Ghetto

Often, the software development organization will attempt to implement agile thinking and practices in a cell isolated from the rest of the organization. While some tactical improvements in delivery are possible, achieving real adaptivity requires the ability to shift funding and investment with alacrity and for all customer-facing business functions to pivot at the same speed, sense threats and opportunities, and adapt. Business agility is achieved by the entire organization encouraging and enabling sense-and-adapt behaviors. The Agile ghetto will struggle mightily, a lovely walled garden in the midst of a hideously static, slow-moving swamp. While I like to say (at least initially) that an Agile ghetto is better than no agile at all. It’s often how Agile pilots are initially implemented, but over time, the lovely walled garden will succumb to the swamp and regress to the organizational norm. The population of the lovely walled garden will become disillusioned and despair that the organization will never evolve. Having tasted agile but become frustrated by the experience of trying to run in the deep mud of the swamp, they will depart for organizations that have embraced agility broadly.

The solution to the Agile ghetto is for the organization to understand how value is delivered to customers through value stream mapping of critical customer-impacting processes. By using the theory of constraints to focus upon and optimize the bottlenecks to value delivery, organizations can force leadership to confront waste and inefficiency and hopefully take action to address underperforming functions outside the walled garden of Agile. Organizations that embrace a culture of continuous improvement will find that parts of the organization that impede effectiveness and agility can succumb to transparent evidence of organizational waste and impediments to value realization.

Flaccid Agile

Adapted from Martin Fowler’s term “Flaccid Scrum”, this extremely common antipattern occurs when organizations implement the work management practices of an Agile process like Scrum, but don’t advance beyond legacy technical practices. Scrum is explicitly and deliberately an incomplete framework. It is focused on work management but is not prescriptive regarding technical practices, though the founders emphasize practitioners DO NEED to practice solid technical practices. Traditional development organizations’ functional silos encourage a “throw it over the wall” approach to quality assurance.

In traditional waterfall, all of the development was completed before the product was integrated, often producing horrendous delays as the code from various developers was found to be incompatible. Once the unpredictable process of integration was achieved, testers would begin identifying defects. By the time the testers were testing, the developers had been assigned to different projects. Defects were hugely expensive to remedy because the developer had to revisit the context from when he produced the code months before. Developers didn’t worry much about making code extensible or maintainable. By the time the product was collecting requirements for an enhancement release, the developers were off working on something else, and with any luck, they would never have to work in that nasty spaghetti code again! Most of these organizations did testing, but didn’t really have anyone accountable for quality assurance, which has to be designed and built in from the beginning (you can’t test quality into a product).

The solution to flaccid Agile is to commit to a defined and mandatory set of technical practices and standards, and to build governance into the development process to ensure quality doesn’t degrade over time (accrue technical debt). One of the strongest Agile methods focused on technical practices is eXtreme Programming (XP), an integrated and mutually reinforcing set of development practices that ensure that:

• Software is always integrated and close to being deployable
• Quality is inherently built into the software asset to prevent defects and debt
• Design supports the iterative extension and expansion of functionality over time.

XP is wonderfully compatible with Scrum, Kanban (or any other method). Incorporating XP practices into your development process is excellent, but probably not sufficient. It is also essential that organizations adopt modern DevOps techniques to achieve high levels of effective (turgid?) agile delivery.

Agile-But

Originally called Scrum-But, this antipattern is in evidence when your team or organization claims to be doing an Agile method while omitting (or doing the opposite of) one of the essential practices of the process.

One of the things that makes Scrum so effective as an improvement tool is that it is a super simple and easy-to-understand process with very few strict rules (see the Nokia test and the Scrum Guide). One of the most foundational rules is that everything developed in the iteration is functionally accepted, fully tested, compliant with your quality standards, and potentially deployable at the end of the iteration. There is no remaining work left undone for that work item. Teams that say: “We do only manual testing, so at the end of the iteration, the testers are too understaffed and overloaded to keep up with development, so we test and accept work in the subsequent iteration” are doing Scrum-But.

Some examples of Agile-But that are common include:

• Infrequent huge releases
• Doing Scrum-Fall (mini-waterfalls with design in one iteration, coding in another, testing in another)
• Teams not “swarming” collaboratively on work with a shared commitment (individuals each working on their own items creates excess WIP)
• Management injecting new work into a committed iteration or pulling team members away for side projects
• Disempowered teams and key roles with inadequate or unclear decision authority
• Handoffs to specialty teams to finish work or provide sign-offs
• Lack of retrospective insight and continuous improvement

All of these are common symptoms of deliberately tolerating Agile-But that preclude high performance.

There are many more failure patterns, but these are some common ones. Please respond with any common ones that I failed to mention, or any really brilliant and creative ones that your organization invented!