DOING Agile versus BEING agile

The number of organizations implementing Agile processes like Scrum, Scaled Agile Framework (SAFe), extreme programming (XP), kanban, Large Scale Scrum (LeSS), and others took off in the last decade and a half. Software companies adopted Agile early because they recognized that developing software with higher productivity and quality, and in particular, being more agile and responsive, would yield a competitive advantage. It’s extremely rare to encounter a software product company that still uses traditional waterfall development processes. Corporate IT lagged substantially—some industries moved to modern techniques earlier, but other industries still today evidence many companies that are struggling to implement Agile practices effectively.

Within financial services, for example, trading companies were early and effective adopters, while large banks and insurance companies have generally been laggards who, in many cases, still struggle today to deliver software competently, in spite of “Agile transformations” that have been running for years. Just to state the obvious, the measure of success here is not compliance with prescribed practices from the Scrum Guide or some framework. The measures of success are:

• Frequent valuable releases (ideally near-continuous delivery)
• The ability to respond immediately to feedback
• Total focus on value and reducing waste
• Exceptional intrinsic and extrinsic quality (zero defect culture)
• Hyper-responsive to change—even late changes can be absorbed with minimum waste and disruption
• Continuous improvement

Monetizing the Method

Of course, large traditional corporations generally, and their IT Departments in particular, consume a LOT of external information, advice, and process guidance, be it content from Gartner/IDC/Forrester, or through the assistance of IT consulting and outsourcing vendors like Accenture, Cap Gemini, Deloitte, Tata, and the like. With the explosive growth in Agile adoption, the ecosystem of training and certification bodies exploded to serve big corporate IT.

They adopt a proprietary brand wholesale rather than thoughtfully adopting the optimal mix of patterns and processes that would best suit their environments and goals

Every old guy who wrote a book or refined a process produced his own branded proprietary Agile method or framework, complete with expensive training courses and certifications for every role and job title he invented. Many of these old guys successfully created significant passive income streams leveraging (and monetizing) their delivery channel of certified trainers around the world because every time a “certified trainer” delivers a certification course, said old guy gets a cut per attendee. Many nice beach houses have been paid off this way. Of course, the consulting companies and global system integrators jumped on the bandwagon. Sadly, many of their IT-shop clients are buying certifications and implementing processes and frameworks in a thoughtless and rote manner, following the compromised guidance of their vendors, instead of dedicating effort to understanding the wide range of Agile approaches and patterns available. They adopt a proprietary brand wholesale rather than thoughtfully adopting the optimal mix of patterns and processes that would best suit their environments and goals. This is the epitome of DOING an Agile process as opposed to BEING agile and lean.

Agile is an umbrella term for many specific methodologies and frameworks that can help organizations improve, in accordance with the Agile values and principles originally codified in the Agile Manifesto and the 12 Agile Principles in 2001. Agile is the software instantiation of Lean product development and manufacturing principles that originated at Toyota in the forties and fifties, with awareness emerging in the U.S. in the eighties.
Lean principles include:

• Define Value
• Map the Value Stream
• Create Flow
• Eliminate Waste*
• Establish Pull
• Pursue Perfection

* I upgrade waste from a practice to a principle in my version

Processes, Patterns, and Frameworks

If you are “doing Agile” but not understanding and implementing lean principles and metrics around value, flow, pull, and waste, you could be missing the whole point behind all the Agile “stuff” your employees are being asked to do. Each of the many specific Agile processes prescribes practices, ceremonies, roles, and activities. Different Agile processes focus on different development and delivery problems. Scrum focuses on disciplines to consistently deliver value incrementally. eXtreme Programming (XP) focuses on developer practices that ensure quality and efficiency. Some frameworks are prescriptive in a particular focus area in order to address a problem that is a common pain within their target market.

For example, SAFe originally had and still has as its centerpiece (among an ever-growing accretion of layers, entities, and roles), a mid-range time horizon planning ceremony (typically performed quarterly) that is primarily a dependency identification and management exercise. It focuses on prioritizing and sequencing work for the quarter and asks every team to identify and coordinate with all the other teams upon which they have dependencies. Ultimately, if successful, every team has loaded up all their iterations for the quarter and satisfied all their dependencies. Finally, everyone signs off that the plan is realistically achievable.

In this model, the tradeoff is that lowered agility within the quarter and high planning costs and overhead will be accepted in exchange for reducing the risk of delays, waste, and errors resulting from unmanaged dependencies. Dependency-related problems are very painful and very common in corporate IT organizations with bad organizational design, poorly structured teams, and weak development and quality practices, so the problem is real, and the tradeoff may be utterly rational, at least in the short term.

Big Room, Bigger Bill

Why just the short term? Originally called Release Planning (which predates SAFe), this mid-range planning ceremony addresses a big problem (dependencies) that is extremely common and very debilitating in large corporate IT organizations, the primary adopters of SAFe. The ceremony has been retitled by SAFe (in one of their many “upgrade” releases of the framework) as Product Increment (PI) planning, but most unSAFe (grin) people who conduct this mid-range planning exercise now call it Big Room Planning (BRP), because it is ideally conducted in person with all the development teams and groups involved in the web of dependencies present and participating (requiring a big room!) Many advocates of SAFe rarely acknowledge that running a huge, expensive quarterly meeting to identify and manage dependencies, in perpetuity, is really dumb.

Many advocates of SAFe rarely acknowledge that running a huge, expensive quarterly meeting to identify and manage dependencies, in perpetuity, is really dumb.

Competent software development organizations, even extremely large ones, eliminate dependencies by addressing the organizational and technical dysfunctions that produce complex and risky dependency networks. If the organization is improving, the number of teams that need to participate in big room planning should be diminishing every quarter until the need to do quarterly big-batch planning exercises is gone. Blindly following a prescribed set of practices without understanding the underlying principles and not integrating practices from other methodologies and frameworks because they’re not native to your chosen proprietary process, is “DOING Agile”, not “BEING agile”, especially if your organization is not demonstrating the Lean principle of continuous improvement (evidenced by requiring the same number of teams to participate in big room planning for six quarters in a row, instead of remediating the root cause problems that produce dependency risks).

Scrambling to Deliver

Another example of doing Agile instead being agile is exhibited in organizations using Scrum, the most popular team-based Agile method (and a core component of the team level of SAFe). The central planning and delivery cadence of Scrum is the “Sprint”—an iteration ideally (in most settings) two weeks in length. Why mandate that teams MUST deliver valuable, fully tested, and functional increments of potentially deployable software every two weeks? Because for most traditional software development organizations, that is a really difficult level of software development capability for them to achieve. Their struggle to achieve consistent deliveries every two weeks provides the organization with tremendously valuable and effective learning about the root causes of their disfunctions and guides them to great improvements in predictability, quality, and productivity by the time they achieve consistent Scrum delivery.

For really good development teams, Scrum contains a lot of unnecessary overhead (like planning tasks for each user story) that is no longer useful.

Unfortunately, a concept many Scrum practitioners fail to embrace is that once they are consistently delivering two-week iterations, there are more great leaps in effectiveness possible by embracing continuous delivery and single-piece flow (again, back to Lean principles). Scrum was a wonderful tool that helped the team improve tremendously, to achieve a decent level of software development competence, but after a certain point in their improvement journey, some Scrum practices are a waste. For really good development teams, Scrum contains a lot of unnecessary overhead (like planning tasks for each user story) that is no longer useful. It’s like continuing to walk with crutches long after the broken leg has healed—it impedes higher performance.

Framework Wars

Each methodology and framework contains its own integrated set of patterns. Frameworks like SAFe and LeSS are each large aggregations of process and organizational patterns that existed long before those frameworks were trademarked. Each framework starts with a radically different set of assumptions about the capabilities and challenges evident in their target markets for adoption. As I said earlier, SAFe assumes adoptees are large IT shops exhibiting all the dysfunctions typical of their ilk: inefficient organizational designs dominated by functional silos (all the database guys report to Bob, all the UX people report to Clara) that maximize dependencies and the communication and collaboration overhead that must be overcome to deliver anything useful. SAFe imposes a heavyweight prescriptive set of levels, roles, artifacts (features, epics, enablers, stories, NFRs, capabilities, etc!), ceremonies, and more to overcome the dysfunction and enable modest competence in delivery.

On the other hand, LeSS assumes that teams are already cross-functional enough to deliver value independently, that there’s a reasonable software quality assurance capability present, and that teams collaborate effectively. The LeSS framework biases towards the lowest overhead practices necessary to manage increased scale and complexity.

LeSS also assumes that teams and organizational leaders are actively improving their organization through root cause analysis, impediment removal, and continuous improvement. It cracks me up that the LeSS guys actually say: “LeSS won’t work in your organization… not the way your organization is now!” LeSS assumes the development organization will stop being stupid on purpose (as I like to say), as opposed to assuming that the organization’s dysfunctions are unchangeable.

If you assume that an organization is committed to improvement and has leaders who have the courage to implement change, you address root causes. If you assume an organization can’t be fundamentally changed, you implement patterns that are workarounds for impediments and shortcomings as the only viable route to improving performance; symptomatic treatments are better than nothing if root causes are unsolvable.

The important point is that every framework contains its own set of assumptions and patterns, a small subset selected from a massive pattern library available in the industry. Each trademarked framework and process must exclude or ignore many patterns that would be confusing or contradictory because they don’t integrate with their selected patterns, or don’t fit assumptions about their clients. This is why understanding the underlying principles that lead to peak performance is essential. Those “doing” Agile will not study and assimilate patterns outside their chosen purveyor. Those who really embrace agile thinking will adopt effective patterns promiscuously, irrespective of their origin, knowing that no single framework could ever contain all the best answers for any real-world organization.

In my next blog, I’ll illustrate a bunch of common antipatterns that help guarantee your agile transformation will fail.