Tuesday, April 29, 2008

Are you really doing Scrum?

Jeff Sutherland, one of the founders of Scrum, has spoken about the Nokia Test - 8 questions to determine if a team is actually doing Scrum. A more succinct summary can be found here. Can your team pass the test?

Monday, April 28, 2008

Stealth agile and agile contracts

I attended tonight's Agile Denver meeting, which was a presentation by Richard Lawrence from Avenade entitled Stealth Agile - how to implement agile techniques when you don't have full -- or any -- buy-in from management. It was a short preso, and the discussion after the formal part was informative. One audience member asked how to introduce agile practices on a project which is architecture-centric and project leaders tend organize developers' work around components or layers, rather than features that provide end-user value. The suggestion from the audience was to build a small feature as a proof-of-concept designed to expose risk. Architects generally favor proofs of concept, and they also generally like the idea of finding and reducing risk. Note that you can present this idea without even using the word agile or any of it's practices. Sneaky, eh? One caution though - when people see that your POC works, it'll likely end up in production, so built it with production quality, including automated tests.

One of the most interesting questions, I thought, had to do with a decidedly non-stealth agile issue, which is, how do you write a contract to be agile from the beginning? Richard's response was that traditional contracts are typical very scope-centric; they focus on fairly low-level details about what software features will be built. For an agile contract, he advocates one that focuses on the product vision, and just enough scope to specify who will own the intellectual property of different parts that get built - something the Avenade legal team insisted on for their consulting contracts. The agile contract specifies the vision, a team size, and a time frame in which the contractor will endeavor to achieve the product/project vision, allowing customer and supplier to collaborate on refining the specific features which best achieve that vision - through iterative development of working software and the feedback that results. I suppose this requires a fair amount of trust between the parties, but if enables success, I'd say a little trust is a small price to pay.

Monday, April 21, 2008

Lean foundation of Agile Methodologies

Agile methodologies such as Extreme Programming and Scrum emerged in the 1990's as a radical departure from traditional, waterfall software methodologies. But were these agile methodologies really so new and radical? Many thought leaders have recently made the point that agile principles and practices are a software manifestation of the principles behind the lean product development strategies applied so successfully by Toyota starting decades before the agile software movement began in earnest.

Listed here are the seven principles of lean software development as identified by Mary and Tom Poppendieck in their book Lean Software Development: an Agile Toolkit and their agile counterparts from the Agile Manifesto (1), Scrum (2), and Extreme Programming (3).

Eliminate Waste
  • Working software is the primary measure of progress (1)
  • Simplicity - maximizing work not done - is essential (1).
  • Simple design - YAGNI (3)
  • The most efficient method of conveying information is face-to-face conversation (1)
  • business people and developers work together daily (1)
  • XP planning game (3)
  • Test-driven development (3)
  • Continuous integration (3)
Amplify learning (feedback)
  • early and continuous delivery of valuable software (1)
  • business people and developers work together daily (1)
  • Scrum sprint reviews held with all stakeholders (2)
  • XP - small, frequent releases (3)
Decide as late as possible
  • Welcome changing requirements, even late in the process (1)
  • Scrum product backlog - prioritized prior to each sprint (2)
  • Sprint planning / XP planning game (2) (3)
Deliver as fast as possible
  • deliver working software frequently (1)
  • potentially shippable software at the end of each short sprint (2)
Empower the team
  • Build projects around motivated individuals...trust them to get the job done (1)
  • The best...designs emerge from self-organizing teams (1)
  • Scrum self-organizing teams and Scrum master as servant leader (2)
Build integrity in (to delight customers)
  • Our highest priority is to satisfy the customer through early...delivery of valuable software (1)
  • Continuous attention to technical excellence and good design (1)
  • Design improvement / refactoring (3)
See the whole (optimize the whole system, don't sub-optimize)
  • At regular intervals, the team reflects on how to become more effective (1)
  • Scrum - sprint retrospective (2)
  • Design improvement / refactoring (3)
  • sustainable development - should be able to maintain a constant pace indefinitely (1) (3)
  • Design improvement / refactoring (3)

Thursday, April 17, 2008

When agile is not a natural fit

For a while now, I've been leading a team in a situation where textbook agile practices aren't a natural fit. My company, IP Commerce, has built a platform to enable electronic payments. Applications (built by third parties) connect to our platform to gain access to a variety of payment services such as credit card processing, electronic check processing, and e-commerce services such as PayPal. My team is responsible for integrating the various payment services with the IP Commerce platform. Each integration is called an adaptor, which translates messages (transactions) from the IP Commerce format to the service provider format & protocol. Why is it that this type of development doesn't easily fit into the typical agile model?
  • The scope of each adaptor is essentially fixed
  • Each adaptor must be certified before it can be deployed
  • The duration of each adaptor project is 6-12 weeks
Let's examine each issue in more detail.

Each adaptor translates messages, and to accomplish useful business functionality, there is a minimal set of messages that it must support. For example, in credit card processing, a customer must be able to do authorizations, voids, refunds, and settlements. Without all of those features, it doesn't meet any customer's minimal requirements. The only scope which is negotiable is minor features, such as corporate purchase cards or certain industry-specific features, for example those that support the lodging industry.

One of the core presumptions of agile is that scope is negotiable and features can be prioritized. If 90% of the scope is fixed and all the features (in this case, message/transaction types) have the same priority, the backlog isn't very interesting.

Agile methodologies, Scrum in particular, assume that a product can be deployed when the product owner judges it to have enough functionality completed. In our case, an adaptor cannot be deployed until the service provider to whom we're integrating certifies it. Not to mention that, as explained above, we need essentially all of the features working before it is useful to customers. This dependency on an external verification process is unavoidable, and unfortunately, it often takes a long time.

Each adaptor project lasts between 6-12 weeks. Most agile projects I've been involved with before now have been much longer, and have many more iterations that establish the all-important feedback loop and team rhythm.

Adapting agile to the situation
One approach we have tried is to treat each adaptor as a single coarse-grained feature in the backlog. This makes sense because these are the units of functionality that the business prioritizes, but on the other hand it doesn't make sense because agile features (user stories or backlog items) need to be small enough to fit into a single iteration (sprint). That disadvantage is a big one in my mind, so we have decided it's not a good approach.

Instead, we have decided to choose smaller backlog items, which for a single adaptor include a combination of true user-stories (message types and other user-identifiable features) and milestones such as 3rd party certification. In the past, I have been a big proponent of using a consistent sprint/iteration duration. In this situation, however, we've found it to be useful to first identify a concrete objective for each iteration (e.g. complete the first 2 message types, or achieve 3rd party certification) and then choose the sprint length based on the tasks required to achieve the objective - while keeping each iteration to 4 weeks or less.

It still feels awkward at times, but I feel we benefit greatly from relatively short iterations.