Monday, October 27, 2008

Is Agile a Fad?

I attended today's Agile Denver meeting - this time in Boulder - to hear Mary Poppendieck's presentation, Is Agile a Fad?

I'll summarize some of her material here, mostly in reverse order, starting with the key points and conclusions of the talk, followed by some of the contextual info she presented leading up to those conclusions.

The key to successful development organizations

The key to a successful development is for engineers (developers) to have a deep understanding of their customers - both internal and external. When new engineers start at Toyota, they spend their first 6 months on the production line assembling cars, so they fully understand their internal customers. Then, they spend 6 months working for a dealership - selling cars - so they know what customers really want.

Another key is building a culture that retains quality people for the 6-10 years it takes to build true expertise, and growing leaders from within the organization.

Fads vs. enduring principles

Why do we have these fads that fail?

  • silver bullet thinking. there is no silver bullet
  • trying to apply 1 solution in different contexts. different contexts require different solutions.
  • essential tensions in software. Don't swing too far toward one side or the other; rather find a solution that solves the valid concerns on both sides of the issue.

The principles behind systems engineering are robust over time. The concepts in project management are fragile over time.

Systems Engineering Project management
low dependency architecture complete requirements
quality by design quality by testing (at the end)
technical disciplines maturity levels
respect for complexity scope control
skilled tech leaders resource mgmt
learning & feedback cycles timeboxes
success = accomplishing system's objective success = accomplishing planning scope, cost, schedule

5 essential tensions in development

  • People. self-managed vs. managers. Answer: the servant leader facilitating self-organization.
  • Process: empirical vs. defined.Solution: relentless improvement, rigorous process for effective improvement. Identify the true root cause of problems, hypothesize solution, determine how to measure if it succeeds.
  • Product: development team vs. customers & biz operations. Solution: whole team philosophy. team talks to customers so they understand the problem deeply.
  • Planning: evolving plans vs. predictability. Solution: pull scheduling, set-based design (build multiple options), clear technical vision
  • Performance: concern only for the next iteration vs. long-term scope, schedule & cost. Solution: a team with pride & passion that delights customers - and has a deep knowledge of it's customers' needs, sustainable profit, breakthrough innovation

A brief history of software methodologies and the seeds of agile

What happened to all those methodology buzzwords? RAD, lean, structured programming, etc.? Sprinkled throughout the history of software, various people discovered and promoted practices that we call agile & lean today. They also promoted various practices that were unsuccessful fads.


NATO conference on the software engineering crisis. Edsger Dijkstra said that programming became a problem in relation to the size and complexity of computer hardware. Douglas Ross of MIT said the most deadly thing is the assumption that you can specify what you're going to do, and then do it. The solution (compared to assembly languages): high level languages! (Cobol, Fortran, etc.). This removed drudgery, but increased the level of complexity possible, which led to the same problem all over again(Dijkstra).


New York Times software project

  • structured programming made software more readable. Dijkstra proposed quality by design, as opposed to reliance on testing.
  • Dave Parnas devised information hiding, the concept of objects.
  • Top down programming was introduced by Terry Baker - basically this was the concept of continuous integration.
  • The "chief programmer team" concept introduced the tech lead, design review, pair programming, common code ownership. Result was 100 times more productivity (measured in LOC) and higher quality than typical at the time.


Barry Boehm proposed that software maintenance was becoming the most expensive part of systems, and that the cost of changes got exponentially greater in later phases of the lifecycle. This famous (infamous?) curve was the key reason that everyone tried to nail down all the requirements at the beginning.


  • Daniel McCracken & Michael Jackson wrote that the lifecycle concept (waterfall) was harmful and perpetuates failure by constraining thinking, and ignoring the reality that needs inevitably change over time.
  • James Martin wrote the 4th gen languages would allow application development without programmers. (A gross oversimplification of the inherent complexity.)


  • Scott Schultz at DuPont introduced timebox development. 30 days for analysis & design, 90 days to develop. He called it rapid iterative production prototyping.


  • Boehm introduces the spiral lifecycle model. More evolutionary model, but it's still a project management model, not a systems engineering model.
  • Watts Humphrey introduced software process maturity model (CMM). Attempt to bring in statistical process control and mandate maturity assessments. Focus on project management practices over system engineering practices.


  • James Martin wrote book on RAD, facilitated by CASE tools. (Where are those CASE tools today? Anyone?) Problem: RAD often produced un-maintainable code. Didn't live up to the hype.


  • Internet booms. J.C.R. Licklider serves as the technical visionary for several key internet organizations. Standards were developed.

1 comment:

Anonymous said...

I have noticed that it seems that the fragility of any application development approach is directly related to the degree of tolerance (in an organisation or team) for mediocraty and compromise(both intended and unintended). Thus application of any approach or comination of approaches can be made into a struggle or failure. We people do seem to be able to get in the way of some really smart and useful ideas sometimes.

Aussie Ian