How Did Agile Leadership Emerge?

Agile leadership emerged from four key influences

Agile Software Development

For the past 20 years, there has been another trend emerging and disrupting many industries - Agile Software Development. 

Post-industrial project management is rooted in civil construction and mechanical engineering industries where tools such as the Gantt Chart (1917), Critical Path Analysis (1957), and the Work Breakdown Structure (1962) emerged. Project controls such as Waterfall (1970) and Stage Gates (1980) quickly followed. It is amazing, and a bit sad personally, that these approaches still dominate much of the project management lexicon today, even as the systems they are applying them to have changed tremendously.

With the emergence of software and its flexibility as a medium, many of these earlier project management approaches were challenged. Early forms of agility emerged in the 1980’s and 1990’s - Adaptive Programming (1980), Spiral Method (1985) and Rapid Application Development (1988). More recognized “pre-agile” approaches emerged in the 1990’s with Scrum, Extreme Programming and Kanban.

The forefathers (yes, all male) of these methods came together over a long weekend in 2001 at Snowbird, UT to ski and connect. Surprisingly, a formative document emerged, The Manifesto for Agile Software Development. Its four value statements and 12 principles have held strong and guided an entire movement toward increased agility.

The need for — and value of — agile leadership can be partially attributed to The Agile Manifesto and the movement which it inspired and fueled. Scrum, Kanban, and the many agile scaling frameworks that have been built upon them are expanding well beyond the software industry. In addition, as every aspect of business is being integrated with software, the Agile Software Development industry is also coming to you, whether you like it or not. If you have not heard about this movement, you better start reading.

Leadership Development Research

At the same time the Agile Software Development movement was disrupting the project management industry, research in leadership development was equally advancing from more traditional command and control military and hierarchy models towards more distributed, shared, and humanistic approaches.

The foundations of modern management are often attributed to Frederick Taylor and his consequential paper on The Principles of Scientific Management (1911). Taylor was incredible for his time in improving system workflow, irrespective of human impact. While he routinely improved manufacturing workflow by many multiples and improved quality and repeatability at the same time, he looked upon the workforce as incompetent, unskilled and not of value to train beyond simple repetitive tasks. Leaders think, workers do.

It wasn’t until the second half of the twentieth century that the leadership development community began to shake up the foundational theories and instilled leadership practices laid down by Taylor. Theory X & Y (1960), Servant Leadership (1970), Situational Leadership (1975), Transforming Leadership (1978) were a few of the forerunners of a radical shift in thinking about leadership. Not too different from Copernicus proposing the earth revolves around the sun, these new researchers were proposing the leader to serve the employee and employee motivation should come from within rather than from the leader via external threats and incentives. 

The Agile Software Development movement was connecting with these more relational and adaptive leadership approaches as better fits with agile values and principles. Many early agile leadership definition could be found in the Scrum Guide, “Scrum Masters are true leaders who serve the Scrum Team and the larger organization.”

One of the ways that the Agile Software Development movement intersected with the Leadership Development Research was through the work of Bill Joiner in his book titled Leadership Agility (2007). While having many similar characteristics to earlier works, the agility name invoked a greater intersection in those two separate worlds.

Toyota and Lean Manufacturing

Toyota struggled in the early twentieth century. Following World War II, Japan manufacturing was struggling with quality control of its products. In a chance meeting and speech given by Edward Deming on Statistical Product Quality, Toyota and Deming began a relationship that likely inspired one of the most significant economic stimuli in our lifetimes and held Toyota on top of the auto industry for the rest of the century.

At the time labled The Toyota Way, and now known best as Lean Manufacturing, Deming and Toyota not only shook up the manufacturing process, they were fundamentally challenging the same foundational leadership principles laid down by Taylor (as described above). Deming put the line worker in charge of their process and quality control. He shifted power to “stop the line” from the manager to the worker. They installed an andon cord that ran the length of the line that any worker could pull to stop the entire process. At which point, the nearby assembly workers would gather to evaluate, discuss and deep dive (five whys) into the root cause of the problem.

While technically this new leadership approach predated the theories described in the section above, it didn’t effectively leave the confines of Toyota. Not that they didn’t try. Indeed, they gave numerous tours to many other automakers on their approaches, only to find that while their practices could be copied, other companies were either unable, unwilling, or simply didn’t understand the new leadership principles to make the new processes work. Many companies tried and failed to copy the Toyota Way. Today, while principles of lean can be found in virtually every manufacturing process, the human condition and respect for the worker is hit or miss, proving that process improvement is easier than human development.

Furthermore, most of the Agile Software Development thought leaders credit Deming, Toyota and Lean Manufacturing as laying the groundwork for their more modern movement. In other words, the Agile Software Development movement would likely never have emerged without the groundbreaking work of Deming and Toyota.

Early Human Development

The concept of leadership dates back to primitive times in human history. Dating back 2.5M years, leader and follower reciprocity likely followed a service-for-prestige theory. Voluntary leader-follower relationships evolved through reciprocal exchange of prestige for valued services. For example, providing someone more or better food in exchange for their value in hunting for and protecting the community.

Leadership adaptiveness in these early times was crucial, as communities were nomadic following food and water sources. Furthermore, leaders emerged based on mutual benefit rather than being instilled. And when in conflict with their leaders, members were mostly free to leave the tribe to form new communities. Agility in sharpness of mind and responsiveness of body were critical to survival during these highly dynamic times.

The human brain’s value of status, certainty, autonomy, relatedness and fairness (SCARF) developed during these formative human periods. Our desire for prestige, connectedness and freedom and the threat we feel when they are challenged or taken away are deeply ingrained in all human DNA. 

Yet most leadership today continues to follow another period of human history dating back only 13,000 years. This is when agriculture emerged, populations settled down and formed larger cities protected by walls and castles. Corporations and organization charts today more closely resemble the hierarchical, structural and class-based systems formed during this agricultural period than their earlier nomadic times.

During this later period with larger and more settled states, leaders were installed based more on their relationships, who they were and who they know, then based on experience and mutual benefit to members as was custom during the nomadic times. The connection between leaders and members along with their power and status pull further apart. And the lower-class members are often dependent upon their state and vulnerable if they were to try and leave the protection of their state.

Thus, we can say there is a mismatch between recognized and patterned corporate leadership and the motivations and desires of a human population. And with the increased transparency and connectedness of a global population, it is stressing all existing systems and creating new opportunities for new ways of leading to emerge, thrive and grow. Enter agile leadership.

The Benefit of the Emergence of Agile Leadership

Agile leadership sharpens organizational focus and accelerates organization action and responsiveness in complex, uncertain and rapidly changing environments.