Agile Advocate

A new home for my blog

2010-03-13T07:04:00.000-08:00

I've moved my blog! I'm leaving Agile Advocate where it is for now, but all my new posts will be at Propero Solutions.

Jira and GreenHopper for agile project management

2009-12-17T21:18:00.000-08:00

I previously blogged about using Jira with some open source tools for managing agile projects, and concluded that it was not a great solution - without GreenHopper. I've been using the latest version of Jira with the GreenHopper agile plugin extensively now, and I like it. It is lacking some features, which I'll describe later, but here is an overview of the main functionality.

The first thing I like is the Planning Board view, which is aimed at Product Owners who spend lots of time ranking and re-ranking backlog items. The GreenHopper planning board allows you to drag and drop items to either change their rank or to move them from the backlog into a particular iteration or sprint (version, in Jira lingo).

Above: dragging a user story from the backlog to a sprint. (Click image to see it full-size.)

For the development team, GreenHopper's Task Board view is just that - it simulates the physical task board that so many co-located teams use. It's easy to drag the virtual cards to columns to update their status, just like you would do with a physical board. Very nice. It also does something you can't quite do with sticky notes; double click to see more information, or click on the item ID to see all the detail.

Above: The Task Board in summary view mode (Click image to see it full-size.)

Above: See more of the task board without scrolling - the List view (Click image to see it full-size.)

Above: Dragging a card from "To Do" to "In Progress"

Above: Double click to see this expanded view of any card on the board.

An advantage of GreenHopper over sticky notes on the wall is that your burn down charts are calculated automatically: in hours, number of items, and story points. The screen images below show these. The red line is the ideal burn down and the green line is the actual.

Above: Burn down in hours (Click image to see it full-size.)

Are you a bleeding-edge adopter of kanban? GreenHopper is decent for that. The columns on your kanban board are customizable, so you can create any columns you want. You can also set a WIP limit for each column. You can't, however, set up horizontal "swim lanes" for different service level agreements (SLAs), such as a row for items that need to be expedited.

Above: The column turns red when the WIP limit is exceeded

The Jira dashboard also offers a reasonable approximation of the cumulative flow diagram (CFD) that replaces burn downs in a lean / kanban process. It does show items created vs. resolved, but it doesn't graph items in-progress. The chart also counts sub-tasks as items, which you may not want.

Above: Almost a cumulative flow diagram. Close, but no cigar.

GreenHopper is an add-in to Jira, so you still get all of the Jira functionality unchanged. Like Jira itself, GreenHopper is extensively configurable; almost annoyingly so because there are so many options it's hard to know what to do with them. Fortunately the default configuration is reasonable, although there are some setup steps required to enable item ranking, which is so fundamental that it should have been done out-of-the-box.

What is it lacking?

If you're a large organization with a portfolio of inter-related projects, Jira + GH doesn't do much to show you the big picture across multiple projects.
Although you have a hierarchy of versions, such as a release containing multiple sprints (described here), it doesn't do much; the planning board simply indicates that one version is a "master" of the other. GreenHopper doesn't offer a timeline view showing the version hierarchy.
You can create a hierarchy of backlog items, for example to have large-scale epics that contain many user stories, as described here. However, this feature isn't enabled by default and requires an adminstrator to configure it. As an alternative, or in addition to the "epic" feature, you can create associations between items for this purpose, but there is no easy way to visualize the relationships. There is free task hierarchy plug-in that will show the hierarchy, but only for one top-level item at a time.

Overall, especially for smaller teams without portfolio management needs, I give the tool high marks.

Definition of Ready (DoR)

2009-12-16T20:10:00.000-08:00

All good agile teams have a Definition of Done (DoD) - the common set of criteria that every feature must meet before it's considered complete. I wrote about the DoD previously in this post. But what about a Definition of Ready? How do you know when your backlog items are ready for the team to accept them?

Serge Beaumont spoke on this topic among others at the 2009 Scrum Gathering in Munich in his presentation Practical Tools for the Product Owner: Focus, Value, Flow. He describes the Definition of Ready as a negotiation between the Product Owner and the team, where the team ultimately decides if a backlog item (user story) is defined well enough for the team to estimate and implement. His checklist for the DoR for backlog items includes:

Why? Business Value
What? Outcome vision
How? Implementation strategy & cost
Does the backlog have enough items for 1 and half to 2 sprints?
Is the size (granularity) of the backlog items acceptable?

I would personally aim for a smaller number of items in the "ready" state; 2 sprints worth of items increases the risk that time will be spent analyzing items that end up in the trash heap.

Serge also advocates using a kanban board with 4 phases to track the readiness of backlog items:

New
Preparing (going through analysis)
Ready
In sprint

I'll be adding the DoR as a standard part of my Scrum implementations.

SourceMonitor for .NET code quality metrics

2009-11-18T19:54:00.000-08:00

A tip of the hat to Paul Rayner for recommending SourceMonitor for .NET code quality metrics. I gave this freeware tool a spin today on a C# project I'm working on for a large client, and was very happy. It's easy to use and within a few minutes I had it installed and ran the analysis on the whole project, gathering these metrics for the whole solution, and for each C# file within the solution:

number of statements
methods per class
calls per method
statements per method
average and maximum complexity
average and maximum block depth (number of levels of indentation/curly braces)

This is a great tool to quickly find potential problem spots within the solution, so you can identify the components that would likely benefit the most from some refactoring and additional testing.

Agile vs. Waterfall: The NetFlix analogy

2009-10-11T20:40:00.000-07:00

The following is a fable to compare traditional waterfall projects to agile/lean projects...

Announcing my new movie rental service, NetPix! Here's how it works.
You get 60 movies each year, delivered on DVD in the mail.
When you first sign up, choose 60 movies you want. When you finish choosing your movies, we'll confirm that it's really the list you want.
Over the next 12 months, NetPix will tirelessly collect your 60 DVDs, and at the end of the year, we'll deliver all 60 movies to your home!
Price: $60 per year (just $5 per month!)
The fine print:
DVDs will be delivered 12 months after you complete and confirm your list of 60 movies.
After confirming your movie list, any changes you make to the list in the first 6 months will incur a charge of $2 per movie.
Any changes in your movie list during months 7-12 will result in a charge of $3 per movie, and a delivery delay of 1 month per movie changed. (We have to lock down the list at some time, after all!)

Now suppose I order, say "Weekend at Bernie's", both I and II, on the advice of my high school buddy, Steve. A year later, I watch "Weekend at Bernie's" (the first one), and I hate it! Steve usually has good taste, so I trusted him, but he must have been smoking crack when he recommended this one. I don't even wanna watch "Weekend at Bernie's 2", but I already paid for it and it's sitting in a cardboard box next to my DVD player. Damn!

And to top it off, eight months into the year the studio released a remastered, Director's cut, collector's edition of my all-time favorite movie "Blazing Saddles", with never-before-seen outtakes! If I want that it'll cost me an extra $3 (5% more) and I'll have to wait an extra month for all of my DVDs! Double damn!

Would you buy this? "No," you say? Well why not? This is the way software customers have been buying software for decades! Customers have to figure out everything we want before we even start working on it, and they can't change their minds later without paying dearly for it!

Alas, much to the chagrin of NetPix, along comes an upstart disruptor, NetFlix! Now customers can get the features (DVDs) they want every month instead of waiting a year! And they can change their priorities (movie list) at any time! And they can watch "Weekend at Bernie's I" before deciding if they want to invest 90 irrecoverable minutes of their lives watching the sequal. The demise of NetPix is written on the wall!

More tips for better retrospectives

2009-09-29T12:57:00.000-07:00

At last night's Agile Denver meeting, I facilitated an open space discussion and one of the topics was conducting more effective retrospective meetings. The group had some great insights and advice that I'd like to share here. Thanks to everyone who participated!

Rotate responsibility for facilitating the retrospective. Get fresh perspectives and creative ideas. Don't let it get stale.
Hold your retrospective over a happy hour.
Read Agile Retrospectives by Esther Derby and Diana Larsen
Instead of asking "what were the problems", ask "what could we have done better."
Follow through with an action plan. If you have lots of improvements, rank them and work on the top priority issue.
Have a virtual buyer/seller transaction at the end of your iteration to determine if stakeholders are willing to "pay" for what you delivered.
Do shorter iterations - fewer issues to discuss
Set the stage with expectation that it's not about blaming. Participants must describe the effect of the problem, not the person they believe caused the problem.
Take collective team ownership over problems. It's the team's job to help every person on the team.
Start the meeting by having participants write issues on cards, then post them on a wall and group them. This can effectively elicit input from shy people who are hesitant to speak in a group.
Ask creative questions, like "if this iteration were a car, what model would it be?"
Keep a retrospective board where team members can at any time write down issues they want to discuss in the retro.
Is there an elephant in the room that nobody wants to talk about? If so, get it out!
End on a positive note. Everyone must say one positive thing.

Open source agile project management tools

2009-09-22T07:26:00.000-07:00

I authored a comparison of five open source agile project management tools: Agilefant, IceScrum, Agilo, eXPlainPMT, and XPlanner. The article is posted on Open Logic's Wazi site. I welcome your feedback!

Seven tips for distributed agile teams

2009-08-19T08:54:00.000-07:00

Agile methods were originally intended for small co-located teams. Can they work for geographically distributed teams? In fact, due to the challenges of working with distributed teams, I believe the fast feedback and validation afforded by agile methods provide a distinct advantage for distributed teams because problems are found sooner. I have experienced great success with teams split between the US, India and China. In this post I share some tips on making distributed agile teams succeed.

For a more in-depth treatment of this topic, see my presentation on Distributed & Offshore Agile Teams.

1. Prepare your collaboration infrastructure

You may need phone, web, & video conferencing, wiki/intranet, source control, VPN, remote desktop tools, SFTP for large file transfer, testing and integration environments, and project management software. Obtain high quality full-duplex speaker phones, headsets, and webcams.

2. Have an agile coach or agile champion at each location

In addition to providing agile training to every team member, you want ongoing guidance at each location from an experienced agile practitioner who can establish an effective team culture.

3. Be extra disciplined in your technical practices

The technical practices from XP (pair programming, TDD, refactoring, continuous integration, coding standard, simple design) are fundamental for any high-performing agile team, but when people are scattered across the globe, they become even more important. These practices ensure that quality is built in from the start, they enable knowledge sharing, and ensure rapid feedback.

4. Do cross-location travel throughout the project

Get the entire team together in one location at the beginning of the project and again at each release. Establish a rotating travel schedule so each team member visits another location for 1-2 iterations. This face time will reap big returns in establishing trust, sharing knowledge, and maintaining an effective team culture.

5. Maximize opportunities for real-time communication and collaboration

Even with a time zone difference, find at least a few hours every day when all team members are available by IM & phone for real-time communication. Share time zone burdens fairly between locations. Hold daily stand-ups and iteration planning meetings with the entire team using video and web conferencing. Many studies have shown that the majority of information shared in a conversation is non-verbal, so use video chat & conferencing often. Google, Yahoo and Skype offer free 2-way video, and Paltalk has free video for up to 10 parties. Webex and many other commerical services offer multi-party video conferencing.

6. Facilitate cross-cultural communication: speak slowly, avoid jargon and idioms

If your team members don't all share the same primary language and culture, be sure that everyone speaks slowly and avoid using jargon, idioms or expressions - they usually don't translate well. Announce your name each time you speak on a conference call: "This is Sally..." Start each meeting with a virtual seating chart as described by Jean Tabaka in her book Collaboration Explained. Appoint a facilitator to ensure each meeting is effective.

7. Find effective, lightweight tools for distributed collaboration

Intranets, wikis and source control systems allow easy information sharing. Virtual white boards such as scriblink, skrbl, and dabbleboard enable remote collaboration. Many web conferencing systems such as ReadyTalk have drawing tools allowing free-form annotation of a shared screen. Instant messaging supports both one-on-one and group chat. VNC, remote desktop and similar tools even allow cross-continent pair programming. For iteration planning and task tracking, share photos or video of your physical task board, or make the leap to an electronic tool. There are numerous open source agile tools plus commercial tools, most with trial or free editions, but choose a lightweight tool unless you truly need the more advanced features.

Eight tips for better retrospectives

2009-08-17T14:00:00.000-07:00

I am often surprised at how many self-described agile teams don't hold retrospective meetings every iteration. For those team that do hold retrospectives, I often find they have a lot of room for making them more effective. An attitude of continuous improvement is fundamental to successful teams, and the retrospective is the primary mechanism for improving, so it follows that effective retrospectives are one of the key elements of a good agile team.

In this post, I offer five tips on how to conduct more effective retrospectives.

1. Review notes & actions from the previous retrospective

This is step #1 in the retrospective, which of course implies that you must first record notes from your retrospective meetings. (See tip #8) Use a projector to display notes & action items from the past meeting. Keep it short - five minutes at most. If you have remote team members, use a webinar and/or video conferencing so everyone sees the same screen. If you find your team bringing up the same problems as in previous iterations, ask why - five whys, actually (see tip #5).

2. Ask what problems, successes and opportunities you have with the team, the product, and the process

The facilitator will ask the team, "What did we learn about the team, the product, and the process? What were our problems, successes and opportunities to further optimize?" Capture these items in a grid as shown below. Use a white board if everyone is in the same room, or take notes in a wiki or other electronic document - preferably the same one you'll use to archive your retrospective notes (See tip #8).

	Problems	Successes	Opportunities
Team
Product
Process

3. Let each team member speak without discussion

The facilitator gives each team member the opportunity to speak without any group discussion; no one else is allowed to comment on anyone's opinion - yet. Only clarifying questions are allowed at this time. Discussion will come later when the team is discovering the root causes of problems. (See tip #5)

4. Vote on the most important items to take action on

You may not be able to immediately address every concern raised in the meeting, but you certainly need to prioritize them. Every team member is allowed to vote for 3 items that he/she believes are most important to take action on. Tally the votes and add the items to your process improvement backlog based on priority. Note that some items may not require any further action. Even successes may need action however; for example, a good pattern or practice may need to be documented in the Definition of Done or added to the coding standard.

5. Use "five whys" to discover the root cause of problems

For each of the highest-priority problems identified, the facilitator will ask "why did that happen?" Once an answer is given, ask the same question again, repeating the process five times. This should lead you to the true root cause of the problem.

6. Create an action plan for your top priority items

For each top priority item, decide what to do, and who can do it - which may be more than one person. Any actions requiring work by team members should be planned and assigned during iteration planning. (See tip #7) Actions assigned to members outside the team must be coordinated and scheduled. Of course the action plans should address the root causes of problems, not just the symptoms.

7. Explicitly allocate time for improvement actions

Maintain a backlog of improvement actions. At each iteration planning, pull the top improvement actions into the iteration. Improvement items should be prioritized along with product backlog items based on overall value to the organization, but teams should reach an agreement with the product owner on a percentage of their time to be spent on improvements.

8. Record your retrospectives & actions

Maintain two artifacts: (1) a process improvement backlog and (2) retrospective notes. The notes may be captured in real-time during the meeting using a wiki, intranet, or other electronic document. Record just enough information to support a review at the next retrospective (tip #1). Appoint someone other than the facilitator to take notes. For the process improvement backlog you may use the same tool you use for product backlogs; this backlog can be owned by the scrum master.

Metrics for agile projects

2009-07-28T12:55:00.000-07:00

Yesterday's Agile Denver meeting was a panel discussion on metrics for Agile Projects. In the end most of the discussion was about software metrics in general, and the discussion produced some good insights that apply to all software projects, not just agile projects. Some highights:

Kevin Sheen from Perficient said you need to know what's important to your customers: deadline, budget, scope, requirements flexibility, product scalability, etc. Use that to decide what metrics are important.

Paul Rayner explained that diagnostic metrics such as code complexity should be initiated by the team, owned by the team, and not reported upward in the organization. He used the analogy of a hospital patient who has his heart rate, temperature, & blood pressure measured. Those metrics are used in combination by doctors and nurses as diagnostic measures, but not reported up to the hospital's board of directors. The problem is that most managers don't understand those metrics and managers generally shouldn't care about them, either. The metrics reported upward to management should be ones needed to make management decisions - about budgeting, schedule, scope. Kevin Sheen pointed out that generally when higher level management wants to "open the hood" and look at these metrics it's because they don't trust that the development team will meet it's estimates, and additional metrics give them a false sense of control over projects.

Brian Boelsterli said that a project's reporting metrics (distinguished from diagnostic metrics) should be clearly rolled up to corporate KPIs. Each project should be able to demonstrate how it is contributing to those KPIs.

Several panelists and audience members spoke about that potential dysfunction that can result when diagnostic metrics are used as carrots or sticks to measure or evaluate individual team members. To avoid dysfunction diagnostic metrics should be chosen by and valued by developers. Even when measured at the team level, compensating metrics should be used: for example if you collect a productivity metric then you should also collect quality metrics so that people don't sacrifice quality for apparent productivity.

Paul Rayner said that he likes the following diagnostic metrics at the development team level:

Cyclomatic complexity - average per namespace and per method
Total lines of code - monitor the trend over time. Should level off or decrease with refactoring.
Coupling - to find classes that have too many dependencies

Richard Lawrence mentioned that he uses Coderush's "maintainability" metric. Visual Studio has it's own maintainability metric which uses a combination of other metrics to arrive at an overall score.

These technical metrics in my view are very valuable indicators - they help you decide when and where a qualitative analysis is required. For example, if you want to know where to get the most bang for your buck on refactoring, look for methods with high cyclomatic complexity or classes with high coupling.

Survey on distributed agile teams

2009-07-08T13:42:00.000-07:00

The User Experience League has created a survey for geographically distributed agile teams. If you have experience in this area, please respond to the survey here.

Agile games & exercises

2009-06-29T13:52:00.001-07:00

Tom Perry has compiled a list of games and exercises for teams on his Agile Tools blog. He has links to a variety of exercises to teach not only agile & lean principles & practices, but also leadership concepts and team building. You can read his blog post here.

Execution: The Discipline of Getting Things Done

2009-06-15T11:12:00.000-07:00

Here is my synopsis of the book Execution: The Discipline of Getting Things Done by Larry Bossidy & Ram Charan.

"To understand execution, you have to keep three key points in mind:

Execution is a discipline [akin to Six Sigma], and integral to strategy
Execution is the major job of the business leader
Execution must be a core element of an organization's culture"

Every business has three core processes which the business leader must not delegate, but be deeply engaged in and build the system and discipline for execution. They are:

The people process (the most important of the three)
The strategy process
The operations process

The 3 core processes must be tightly linked together, and the leader must ensure that high-level objectives are translated into realistic, concrete steps for action in order to be executed.

There are 3 building blocks of execution:

The leader's seven essential behaviors
Creating the framework for cultural change
Having the right people in the right place (no leader should delegate this job)

Building block 1: The leader's seven essential behaviors

Behavior 1: Know your people and your business.

Meet as many people in the organization as you can. Give your view on the state of the business but spend more time listening to them.

Behavior 2: Insist on realism

Create a culture where openness and honesty are rewarded. Compare your company's goals with other companies as a measure of realism.

Behavior 3: Set clear goals and priorities. Focus on no more than 4 priorities.

Behavior 4: Follow through

At the end of every meeting, establish a schedule for follow-up meetings with milestones to be met at each one.

Behavior 5: Reward the doers

To change people's behaviors, link rewards to performance and do it transparently. Reward people for performance against objectives and for their behaviors; for example, reward collaboration. Be sure to regularly record specific instances of people's behaviors, both positive and negative.

"Differentiation [in rewards] is the mother's milk of building a performance culture."

[My commentary: I am deeply skeptical that a performance evaluation process can be designed to eliminate the dysfunction and unintended incentives they often breed. I've written more on this topic in this blog post. I agree with the authors that any such system must evaluate people's behaviors, not just their performance against objectives, but even so there is great potential for mistrust, fear, and promoting competition over collaboration. I also believe it's critical that people receive constructive feedback on their performance much more often than once per year. I recommend bi-weekly reviews if you truly want people to improve their performance quickly.]

Behavior 6: Expand people's capabilities through coaching

Teach others how to get things done and how to lead. Create a deep bench of leadership succession within the organization.

Behavior 7: Know yourself

A leader must have the emotional fortitude to "accept and deal with your own weaknesses", accept differing points of view, deal with conflict, and "be firm with people who aren't performing".

Building Block 2: Creating the framework for cultural change

A company culture consists of it's "hardware" (org structure, compensation structure, communication systems, etc.) and it's "software": values, beliefs, and norms of behavior.

How to create a culture of execution:

First you tell people clearly what results you're looking for. Then you discuss how to get those results, as a key element of the coaching process. Then you reward people for producing the results. If they come up short, you provide additional coaching, withdraw rewards, give them other jobs, or let them go.

A company culture is based on the collective beliefs of it's employees, and their behaviors are their beliefs turned into action. If people believe that they are in competition with their peers for rewards, they will act accordingly. If they believe people are not held accountable, then they won't expect to be held accountable either. The behaviors exhibited and tolerated by leaders trickle down through the organization.

My commentary: The authors, perhaps unintentionally, have revealed the difficulty with traditional performance evaluations here; they put employees in competition with one another to get the biggest slice of the rewards pie, reducing collaboration. Perhaps by adding collaboration and similar behavioral criteria, rather than solely results-based criteria, to the evaluation process, this effect can be mitigated. But can it be eliminated?

Building Block 3: Having the right people in the right place

This job can't be delegated. Larry Bossidy spent 30-40% of his time in the first 2 years at Allied Signal hiring & developing the leadership team. Developing leaders should be a core competency and the company should fill most leadership positions from within. Provide people with various job opportunities matched to their skill sets and potential.

It is critical to know the 3-4 specific key skills & qualities required for a job and to ensure the person has those skills via interviews and reference checks. Specificity is important. For example, the specific skills for a Chief Marketing Officer might be: (1) ability to select the right mix of promotion, advertising, and merchandising; (2) proven sense of what advertising is effective; (3) ability to execute the marketing program with the right timing and sequence in coordination with new product launches.

Determine how the candidate accomplished particular results. Did she sacrifice long-term success to achieve short-term numbers? Did he succeed due to luck? Did she fail to meet an objective due to outside circumstances but still do better than others in the same situation?

The People Process

The people process is the most important of the three core processes, and it must be tightly integrated with the strategy and operations processes. Determine your talent needs over time - they may be different in the future. Know the number and types of people you need in the short, medium and long terms based on strategic milestones. Develop a leadership pipeline with retention risk analysis and succession depth analysis.

When evaluating a person, meet with five other people to candidly discuss the person's strengths and weaknesses. (My commentary: Note that this requires a culture of trust where mistakes are treated as learning opportunities.) Determine when someone can improve via coaching and/or a different role.

The Strategy Process
The strategy must be linked to the people process and operations process. A strong strategy addresses the following questions:

What is the assessment of the external environment?
How well do you understand the customers and markets?
What is the best way to grow the business profitably, and what are obstacles to growth?
Who is the competition?
Can the business execute the strategy?
Are the short term and long term balanced?
What are the important milestones for executing the plan?
What are the critical issues facing the business?
How will the business make money on a sustainable basis?

The strategy review meeting should be interactive, lively, open and creative. This requires a culture of trust that values and rewards honesty. The business leader must regularly follow through to ensure execution toward the strategic goals.

The Operations Process

The operations process must be tightly linked to the people & strategy processes. It "breaks long term goals in to short term targets", specifying how to execute & achieve results. It should include plans for product, marketing, sales, manufacturing, productivity improvement, and contingencies. It is critical to have an open and robust dialogue and debate on the the assumptions underlying the operations process. The budget should be prepared based on the operations plan, not vice-versa. Use quarterly reviews to update the operations plan.

Performance Evaluations

2009-06-15T09:02:00.001-07:00

What do performance evaluations have to do with agile & lean software? As a core component of an organization's culture, they have a huge impact on the way individuals, teams and the whole company operates. I found a great blog post on this topic entitled Performance Evaluations, Business Strategy, and Agile Methodologies. It discusses both the proponents (e.g. Jack Welch) and opponents (e.g. Deming) of the "rank and yank" evaluation process.

In an accompanying discussion on LinkedIn (Agile Alliance group), the blog author Henrik Martensson writes:

Two things to consider are that for rewards to be effective, the feedback loop must be short, and the rewards must be linked to behavior, not results. This is basic behavioral science, but most corporate reward systems miss it. Lean is an exception. Lean companies explicitly use Management By Means (MBM) instead of Management By Results (MBR). MBR practically guarantees dysfunctional behavior, regardless of the reward system used.

No doubt the controversy will continue, but I hope the discussion will cause leaders to challenge their assumptions about performance appraisals and develop better ways for improving the system used to deliver value to customers.

Team Estimation Game

2009-05-19T14:38:00.000-07:00

Estimation: wouldn't it be nice if we didn't have to do it? Well, maybe we don't. In a mature agile or lean/kanban development system where prioritized value is flowing at an acceptable rate, and in a context where knowing both the scope and date of a delivery is not necessary, estimates don't provide much if any value. So don't blindly assume that you must estimate your features/stories. And if you do evaluate the need for estimates and determine that they are worthwhile, you need to determine how much value they provide so you can be sure the effort spent estimating is commensurate with that value.

One of the best-known ways of estimating user stories is planning poker, first described by Mike Cohn (as far as I know anyway). I've used planning poker very effectively, but I recently came across an alternative approach: the Team Estimation Game, originally created by Steve Bockman and described here by NetObjectives (you may need to register to see the link).

In a nutshell, the team starts with a stack of cards containing stories or features. Over the course of the game, story cards are physically arranged linearly according to size: small stories on the left side of the board/table, and larger stories farther to the right. Stories of the same or similar size are stacked together in a group. Taking turns, each member of the team may either (1) place the top card on the stack in the place where he/she believes it should go, or (2) move a card previously placed by another team member, explaining the reason why it was moved. At the end of the game, you can assign story points to each group of cards.

One of the strengths of this game is the emphasis on relative sizing; it makes it more easy and obvious to compare all the stories you're estimating. I also like the tactile nature of the paper cards and the physical involvement of people moving and interacting versus sitting still like they would in planning poker.

Diarised - tool to pick a meeting time

2009-05-19T08:02:00.000-07:00

Is it a challenge to find a meeting time that works for your whole team? If your team is distributed and not all using the same calendar system (e.g. Microsoft Exchange Server), you may not be able to view everyone's calendar. Enter www.diarised.com - a free online site where you can propose multiple meeting times and all the participants can select the dates & times they prefer. As organizer, you can then see which times receive the most votes.

A few things I don't like about it:

You must enter each person's name and email address individually. There is no way to paste in a list of multiple email addresses.
There is no timezone information - receipts will need to know what timezone the organizer meant.

Can't complain for a free service, though.

Comparing Scrum and Kanban

2009-05-15T13:14:00.000-07:00

Henrik Kniberg has posted a good comparison of Scrum and Kanban here. Thanks, Henrik!

Summary of Lean Kanban 2009 talks

2009-05-10T12:51:00.000-07:00

Mike Cottmeyer was busy capturing notes on his blog during the Lean Kanban 2009 conference. See his posts from May 6-8, 2009 at www.leadingagile.com to get a summary of all the great presentations.

Lean Kanban Conference

2009-05-06T13:31:00.000-07:00

Follow the Lean Kanban conference in Miami in real time on Twitter with the hash tag #lk2009. Some great insights from lean thought leaders.

Great summary of kanban

2009-05-04T14:31:00.000-07:00

Thanks to a tweet from Brian Marick, I found this fantastic summary of the Kanban system for software development posted by Jeff Patton. Jeff not only succinctly describes kanban but also sets it in the context of agile methodology.

Lean Thinking: an executive summary

2009-05-01T08:27:00.000-07:00

When reading a book I like to boil it down to it's essentials and write a summary. Here is my summary of Lean Thinking: Banish Waste and Create Wealth in Your Corporation, by James P. Womack and Daniel T. Jones.

The 5 lean principles:

Define value
Identify the value stream and eliminate waste
Flow
Pull
Perfection

Define value

"The critical starting point for lean thinking is value", which is defined by the customer, in terms of a product and/or service that meets the customer's need.
"Lean thinking therefore must start with a conscious attempt to precisely define value in terms of specific products with specific capabilities offered at specific prices through a dialogue with specific customers." Align the organization toward products with dedicated product teams.
Producers tend to continue making what they already are making, and customers tend to ask for products they are already getting, or minor variations thereon. Challenge this way of thinking to discover what customers truly need and want. Example: car manufacturers make cars, dealers sell cars, and customers ask for cars; however the actual product the customer needs is personal mobility. How can the extended enterprise better deliver personal mobility to customers?
Determine the target cost of the product based on the resources required if all the muda (waste) is removed from the process; work toward achieving the target cost, allowing for reduced prices and/or better product features - leading to higher profit.

Identify the Value Stream and eliminate waste

The value stream is all activities necessary to deliver a product to customers

Problem solving: Concept to design and production
Info management: order taking, scheduling, delivery
Physical transformation of raw materials to delivered product

Identify valuable activities vs. waste (muda)

Type 1 muda: create no value but unavoidable with current technology
Type 2 muda: create no value and are avoidable

Example value stream: a can of soda. The stream is dominated by the production of the aluminum can. It takes 319 days to deliver a can of soda to a customer, and only 3 hours of that time is spent actually adding value to the product. The product-in-progress spends more than 99% of its time waiting for the next processing step and being transported - rather than flowing toward completion.

Flow

After eliminating waste in the value stream, make the value-added steps flow together continuously, with no stoppages or rework.
Just-in-time production
Eliminate specialized departments and batches of work done in those departments
Instead of having large, high-speed machines and putting all machines of the same type in one location, use smaller machines and put all the different machines required to produce a single product into closely located "cells" in the exact sequence required.
focus on the end product itself and the steps required to complete a single product - via dedicated, cross-functional product teams
Ignore existing boundaries between companies, departments, and individual roles to envision a lean enterprise focused on "removing all impediments to the continuous flow of the specific product."
Redesign processes and tools to eliminate rework, scrap and stoppages so production of the product can flow continuously

Pull

Deliver only what the customer wants, when the customer asks for it, rather than pushing products out and hoping customers want them
"No one upstream should produce a good or service until the customer downstream asks for it."
Downstream activities use kanban (simple signals) to indicate to upstream activities when more is needed.
Right-size your tools and process so you don't need to produce massive quantities of intermediate parts; be able to switch tools and machines so they can be quickly adjusted (in minutes) to produce different parts or product.

Example of flow: Bumper Works (a Toyota supplier) reduced time to produce a finished product from 28 days to 2 days, with much higher quality, by reducing batch sizes, using

Consider the counter-example of a printed book. Publishers produce a batch of books based on forecasted demand to fill the sales pipeline. Result: Long delay between orders and delivery, product shortages, or over-production.

Perfection

The improvements process never ends: you must always strive to offer a better product while reducing waste.
Do kaikaku - radical transformation - to eliminate the largest sources of waste, and then do kaizen - continuous incremental improvement - to move toward perfection
"Form a vision, select the two or three most important steps to get you there, and defer the other steps until later." Keep your efforts focused for better results.
Don't settle for merely being better than your current competition; eventually someone will come along and beat you.
Transparency across the entire value stream (suppliers, integrators, employees, distributors, etc.) is required to discover opportunities for improvement

Case studies

The book includes a series of case studies of companies that made the lean transformation, from small family owned operations to huge companies in the US, Germany, and Japan. The case studies focus on manufacturing, but also include product development and order taking aspects of the business.

Revolution in Product Development process at Lantech

How does lean thinking apply to product development? Lantech made a single person clearly accountable for the success of a product over it's entire lifetime (profitability and customer satisfaction). The annual planning process ranked the major projects for the year. Lantech created dedicated (full time), cross-functional development teams for the year's top 2 projects. Teams included marketing, mechanical engineering, electrical engineering, manufacturing engineering, purchasing, and production (including hourly workers from the kaizen team), co-locating all team members.

Wiremold

Manufactures wire routing systems and power protection devices
1400 employees in USA and Canada
Unionized workforce (Int'l Brotherhood of Electrical Workers)
Tried TQM (Total Quality Management) and JIT without success before adopting lean

Results of Wiremold's lean adoption over 5 years (1990 to 1995):

Sales per employee from$90,000 to $190,000
Time to produce avg product from 35 days to 1.5 days
Product development time from 3 years to 5 months
Space required reduced by 50%
Profit increased 500%

Pratt & Whitney, the jet engine manufacturer

29,000 employees and $5.8 billion revenue
1993 loss of $262 million, 1995 profit of $380 million, even with sagging sales
Throughput time fell from 18 months to 6 months
Inventory reduced by 70%
Quality problems reduced by 50%
Unit costs for parts reduced by 20% in real dollars even as production volume decreased 50%

Porsche (Germany)

Combined lean thinking with German technik - the concept of superior technology

Porsche's results from 1991 to 1997:

Product development time reduced from 7 years to 3 years
Throughput time reduced from 6 weeks to 3 days (from welding to finished car)
Inventory reduced from 17 to 3.2 days supply on hand
Effort to assemble Porsche 911 or its successor: from 120 hours to 45 hours
Defects per vehicle reduced by 75%
Profits in DM: 1991: 17 M; 193: LOSS of 239 M; 1995: 2 M

Showa manufacturing (Japan)

Makes boilers and radiators, using large batches with long intervals between tool changes
Invited Taiichi Ohno from Toyota to help in 1983
First step was to "convert coil making and assembly from a batch process to single-piece flow"

Results in one week of Showa's lean transformation for coil making only:

eliminated half the plant space required
reduced in-process inventory by 95%
Reduced human effort by 50%
Reduced coil manufacturing time by 95%
Improved quality

Toyota

Taiichi Ohno began formalizing the Toyota Production System in the 1940's
Established the Shusa (chief engineer) product development system under Kiichiro Toyoda in late 1940s
Began extending TPS to their supply chain in 1969. By demanding continuous price reductions from their 1st tier suppliers, those suppliers in turn pushed lean techniques farther down the supply chain.
Applied lean thinking to parts production starting in early 1980s
With the proliferation of different car models, having one shusa per product became unmanageable. In 1992, Toyota re-organized products into 3 product families to share common components, each group led by program managers
Through Toyota persistent effort at improvement over many decades, it has achieved higher productivity, quality, and on-time delivery rates than it's Japanese, US and European competitors

How to put lean thinking into action

"The trick is to find the right leaders with the right knowledge and to begin with the value stream itself, quickly creating dramatic changes in the ways routine things are done every day. The sphere of change then must be steadily widened to include the entire organization and all of its business procedures. Once this is in hand and the process is irreversible inside your own firm, it's time to start looking up- and downstream far beyond the boundaries of individual firms to optimize the whole."

The authors prescribe finding a change agent with a core of lean knowledge, and say it helps to have a crisis to serve as the catalyst for a change. Start with a map of your value stream(s) and be determined to kaikaku (make dramatic change) quickly to produce results the organization can't ignore. Reorganize the company by product family and value stream.

Comparing agile methodologies

2009-04-23T12:10:00.001-07:00

Scrum and XP are by far the most common flavors of agile methodologies - in fact the combination of the two together is the most popular. But what about the others? I found an article that compares Extreme Programming (XP), Scrum, Feature Driven Development (FDD), Lean Software Development, Agile Unified Process (Agile UP or AUP), Crystal, and Dynamic Systems Development Method (DSDM). Part 1 covers XP, Scrum, FDD and lean, while part 2 covers AUP, Crystal, and DSDM.

Tools for distributed team collaboration

2009-04-10T12:09:00.000-07:00

Although agile development was originally intended for small, co-located teams, many organizations have now successfully extended agility to multiple teams in distributed locations. The only one of the 12 core agile principles that is contrary to distributed teams is that the most effective method of conveying information is face-to-face conversation. One of the biggest challenges with distributed teams is overcoming the barriers to communication and collaboration. Here are some of the tools I've found useful for that purpose.

If you're using a physical task board at each location, post digital photos of each board on your wiki daily. Better yet, keep a web cam pointed at your task board at all times so remote teams can see it whenever they want.
Did I mention a wiki? Make it as easy as possible for teams to capture and share written information with a wiki or intranet. Host your own mediawiki, use Confluence, Sharepoint, etc. Or let someone else host it, SaaS-style - plenty of companies offer this service.
File sharing and collaboration sites. Google Sites and Google Docs is a powerful, free combo for teams to share information. Box.net is another hosted file sharing solution.
If you're sharing large files between locations, you'll probably need an (S)FTP or SCP server.
Full-duplex speaker phones are a must-have so you can hear all of the conversation.
Each team member should have a decent quality headset for clear, hands-free audio on one-on-one calls.
Video conferencing. Many studies show that the majority of information conveyed in a conversation is non-verbal. WebEx allows up to 6 video feeds in a conference. PalTalk is free and it allows a video conference with up to 10 participants. There are many other reasonably-priced video conferencing services, such as iVisit and Megameeting.
Virtual white boards. Nothing beats a white board for a design session or brainstorming. Scriblink, Skrbl, and Dabbleboard all offer free online whiteboards.
Web conferences with freehand drawing/whiteboard tools. Readytalk has a great feature for drawing right on top of the shared screen in a web presentation.
Online Planning Poker for estimating work. Thanks to Mike Cohn!
Scrumworks - the free basic edition is very slick and intuitive for all the basic needs. The Pro version has lots more to offer. I prefer Scrumworks over XPlanner.
Rally - the free community edition has limited functionality and supports up to 10 users, and the Enterprise edition is very rich and sophisticated if you have greater needs. VersionOne is a competitor to Rally with similar features and editions, but I don't have personal experience with it.
Jira? As I wrote in an earlier post, with some free add-ins you force-fit Jira into an agile process, but it's not ideal. Greenhopper is a reasonably-priced agile add-in for Jira that beats the free add-ins.
TangyOrange is an online agile task board for less than $5 per month, but it doesn't have the agile project management features of the other tools.

Presentation: Distributed & Offshore Agile Development

2009-03-30T09:32:00.001-07:00

The slides and a webcast of my presentation at Agile Denver on Distributed & Offshore Agile Development are available on my web site. Many thanks to Amit Malhotra for recording and editing the video!

Definition of Done: UI standards

2009-03-16T12:17:00.000-07:00

If you are developing an application with a UI, I recommend adopting a set of UI standards or guidelines to ensure a consistent, high quality user experience. The standard forms a baseline expectation for the team and product owner eliminating the need to explicitly define mundane details for every screen for aspects such as tabbing through controls, initial cursor position, tool tips, progress bars, etc. Instead, those generally accepted best practices can be included in the team's "definition of done" (DoD) for each story / feature.

Here is a list of UI guidelines from Microsoft that can serve as a checklist for your UI. There are others out there, too. If you know a good one, please let me know.

Visas for travel to the US

2009-03-14T14:50:00.001-07:00

Why a post about Visas on an agile software blog, you ask? Well, if you have a distributed team with members around the globe, and you believe in the agile principle that the most effective method of conveying information is face-to-face communication, then you might need to have some people hop a plane. Here is a summary of the types of US Visas that you might pursue for offshore team members.

B-1: for conferences, meetings, business events

These can typically be used for visits of 2-4 weeks, and they must not involve US employment. In the past many companies got away with much longer visits on B-1, but the government has cracked down on that abuse recently.

H-1B: for temporary employment in the US

Only about 65,000 H-1B visas are granted each year, and most of those are claimed by large companies. Visa holder must be compensated at US market rates.

L-1: for intra-company transfer to a US-based branch.

For L-1 The US & foreign company must share common ownership, and applicant must already be employed by the foreign affiliate. Larger companies can obtain blanket L status.

H-3: for training at a US company that is not available in the applicant's home country*

* Many restrictions apply ;-)

J-1: “exchange” visitors in pre-approved areas of study.*

* Again, many restrictions apply with the J-1 program.

For all types of visas, applicants that don't have strong ties to their home country (such as a spouse, children, and property) may be denied. There is a Visa waiver program for visits of 90 days or less from 27 countries (mostly European, also Japan, Singapore, NZ).

Allow lots of time for visa applications to be approved, too. Plan on several months for applicants from places like India and China.

Are you thinking about pushing your luck on a B-1 Visa? Tread carefully. Attempting to obtain a visa by the willful misrepresentation of a material fact, or fraud, may result in the permanent refusal of a visa or denial of entry into the United States.

On the bright side, it's very easy for US citizens to obtain visas to almost any country - unless your offshore team is in Cuba, that is.

Problem with Windows Vista VPN resolved

2009-03-05T10:16:00.000-08:00

I've been using a PPTP VPN connection from my Windows Vista box for a while, and suddenly it just stopped connecting, and throwing error code 800. After many hours of banging my head against the wall and googling for a solution, I finally found it at this link. It turns out to be related to Microsoft Virtual PC 2007.

"If you have installed Virtual PC 2007 on your machine, it can happen that some windows update (I would really like to know which one) destroy some kind of mapping between VPN connection and WAN Miniport driver."

To fix it you need to uninstall "Virtual Machine Network Services" on your physical network adaptor properties.

Arghhhh.

Jira for agile projects? Get GreenHopper

2009-02-18T13:33:00.000-08:00

I previously blogged about trying to use Atlassian Jira as an agile project management tool. My current client already uses Jira across multiple projects and they are reluctant to change tools or spend for new tools. We use Jira "versions" to define iterations. Jira items without a "Fix for version" are part of the product backlog. We installed the free Jira plug-in Custom Issue Order which lets you set the absolute order (rank) of items in a list. Here's a screen shot of it in action, with the "Rank" column added.

We also installed the free Agile Velocity Tracking plug-in.

So far, I must say I'm not particularly happy with the results. It works, but I find myself yearning for a tool that was truly designed to accomodate agile development. Some specific things I don't like:

I can't drag and drop items to change list order. As a product manager, I'm frequently re-ordering the backlog and it needs to be fast and painless. With the Custome Issue Order plug-in, you have to move an item one position up or down, or type in it's new rank, but either way the page refreshes each time. With a Jira server on the opposite side of the planet, the latency is killing me.
If the backlog is too long, each user must customize the page size to be able to view all items in the list. When it's split between pages, you can't move an item from one page to another.
Suppose the team is starting iteration 3.0.2 (2nd iteration toward release of V3), and a defect is reported on V2.1 that must be fixed in a V2.2 release. You can't include this defect in the 3.0.2 iteration backlog because it's based on Jira's "fix for version" 3.0.2, and this item will be fixed in Jira version 2.2. So you have to manage these items separately.
You can't "collapse" a single story to show/hide it's subtasks in the list. This is a feature of ScrumWorks that I really like.
The agile velocity tracking plug-in isn't very intuitive. It might be more useful after our project has more iterations completed, but for now the data isn't very helpful.
No burndown chart. We tried a free plug-in from Laughing Panda but couldn't get it to work. The closest things Jira has is the progress bar: go to Reports -- Versions -- (a specific version) and you get a graphic like this:

The burndown chart is far more useful since it clearly communicates the estimated work remaining (in hours) and the trend line for completing it by the end of the iteration.

My recommendation for co-located teams is still to use a physical task board. If you have a distributed team, or you've tried the task board and have specific reasons for moving to an electronic tool, then I'd recommend Jira + GreenHopper, ScrumWorks, XPlanner, or the free editions of Rally or VersionOne. When I first wrote this post, I hadn't yet tried GreenHopper, but now I have, and it's a great tool. Read about it here.

Kanban in action

2009-02-12T12:11:00.000-08:00

Kanban is a Japanese term common in the lean literature meaning a signaling device: downstream activities signal to an upstream activity when they have capacity to do more work, enabling value to be pulled through to a customer in continuous flow. In software, it starts with a ranked backlog of stories/features to be done, just like in agile. But instead of fixed iterations or sprints, the team pulls items the highest priority items from the backlog as soon as it has capacity. Software may still be released on a regular cadence, and the release includes whatever backlog items have been completed on the release date.

This blog post is a great primer describing how kanban was applied at Corbis.

I have always thought that the schedule pressure of short iterations can have a positive motivating effect on team member and keep them focused on the goal, but I realize now that it can also have negative consequences, invoking feat and causing developers to feel pressured to make short cuts and bury quality problems under the rug in order to give the appearance of completing tasks on time. David Anderson addresses this issue eloquently in this post on the kanbandev discussion group. In a nutshell, David describes how the objective in a kanban system is to reduce cycle time while maintaining quality. With those measurements in place, the team is motivated to continually improve but the system is tolerant of missteps and underestimation on individual features. Done properly, a kanban system should eliminate the potential negative motivations that can be caused by iterations.

New generation of collaboration tools

2009-02-11T14:48:00.001-08:00

One of my session proposals for Agile 2009 is titled Collaboration Tools 2.0 - Which is Best? I plan to have an interactive workshop comparing the merits of tools like Basecamp, Huddle, Zoho Projects, Sosius, and Google Docs. If you have experience with these tools or others in the same genre, please let me know.

Agile 2009 session proposals

2009-02-11T14:21:00.000-08:00

I've submitted proposals to present 3 different sessions at the Agile 2009 conference in Chicago in August:

Agile Outsourcing in China: A Happy Family
Collaboration Tools 2.0: Which is Best?
Strategies for highly effective distributed teams

I'd love to get some feedback - positive or negative! You can find my proposals here.

Forrester Research Survey on Impact of Agile

2009-01-24T16:11:00.000-08:00

Forrester Research is conducting a survey on the impact of agile processes on technology companies. They are looking for feedback from both technical and business people within organizations. It takes only about 10 minutes to complete the survey, and I encourage people to do so. Take the survy here.

Lean techniques to repair humvees

2009-01-15T11:17:00.000-08:00

As reported on NPR, the US Army turned to lean manufacturing techniques, inspired by Toyota, to deal with the large number of war-damaged vehicles that need to be repaired and rebuilt. The repair facility at Red River Army Depot uses a takt time of 16 minutes - producing a completely rebuilt vehicle every 16 minutes, for a total of 32 per day. If that's the true takt time, or course, that would imply that the Army needs exactly 32 rebuilt humvees per day. The NPR story doesn't state if that's the case, but it does say that the vehicles are parked everywhere around the base - let's hope those are the wrecked ones, and not repaired vehicles sitting unused. We all know by now that an inventory of product that isn't being used is muda (waste).

Balsamiq Mockups

2009-01-12T08:23:00.000-08:00

What do you use to create mockups of your UI? I prefer a very lightweight approach using a good ol' whiteboard or even sketches on paper. You can capture a whiteboard mockup with a camera, or scan a paper mock up. For distributed teams, though, you can't collaborate very well with either one of those approaches. I just discovered Balsamiq Mockups and after trying it out, it looks like a good solution. It also has a sweet integration with Confluence, Jira, and XWiki.

Using Jira for Agile

2009-01-12T08:17:00.000-08:00

I'm working with a team that has been using Jira and wanted to see if they could use Jira for the product backlog and iteration tracking. There are a set of customizations and add-ins for Jira that can make it work, as I found at this informative site. The basic idea is to use Jira versions for each iteration, un-versioned items become the product backlog, and add-ins let you fine-tune the ranking of each item and produce burndown metrics.

I'm still a proponent of using a physical task board when the team is co-located, and I think other tools like XPlanner and Scrumworks are better suited to agile development, but I'll give this setup with Jira a try and report back on how well it works.

How projects really work - a cartoon

2009-01-08T13:44:00.000-08:00

I found this great cartoon satirizing how projects really work - or perhaps it should be titled "Why waterfall projects don't work". Funny. Clear. And right on. The cartoon shows what happens when you have handoffs between different people or teams without fast feedback to ensure you're building the what the customer really needs.

TangyOrange: funny name for online scrum board

2009-01-07T19:30:00.000-08:00

For co-located teams, it's hard to beat a physical board with cards or sticky notes for tracking iteration status. Put it in a prominent place and it's an ever-present reminder to the team about what needs to be done. It's a rallying point for the daily scrum. If you have a distributed team, however, it doesn't work so well, and web-based tools often are necessary. There is a new online scrum board tool called TangyOrange, which has the look and feel of a physical scrum board. You can use a simple version for free, or pay just $5 every 40 days (strange billing period, I'd say) for the full-featured version. You can play around with it here.

Definition of Done

2008-12-17T13:22:00.000-08:00

The Boulder Agile Meetup group met last night and discussed the definition of "done", or DoD, on an agile project. Here is a summary of our discussion.

The DoD should be agreed to by the team, written down and rigorously adhered to
Legacy systems - any code base without automated tests - present many challenges to meeting a useful DoD within short iterations. The group didn't reach any conclusion on how to solve that problem. I'd say your choices are either to (1) put new development on hold long enough to write the tests, (2) gradually introduce tests over several iterations, or (3) live with the pain for a long time.
The DoD is different for tasks, backlog items, iterations, and releases
Operational needs for the software should be considered early on and addressed within normal iterations

DoD for tasks:

unit tests pass
code was pair programmed or code reviewed
coding standards are met
test coverage standard is met
code and tests are included in continuous integration system
task was completed with the simplest possible implementation
code base was refactored to support the new task
sufficient negative unit tests were written (more negative than positive)

DoD for backlog items / user stories:

acceptance criteria are met (presumes that Product Owner sufficiently defined acceptance criteria)
functional tests pass
non-functional tests pass (scalability, reliability, security, etc.)
story documentation is completed
item follows architectural & design guidelines
automated installation & deployment completed (recommended that it includes the full stack - even the operating system - where practical)

DoD for releases:

Customer acceptance (functional & non-functional requirements)
Release documentation completed
Operational needs met
Regulatory & compliance requirements are met if applicable
automated installation & deployment completed

For some reason we didn't discuss DoD for iterations, but I would say that it is:

Met DoD for all stories / backlog items in the iteration
Met the goal or theme for the iteration, if one was defined
Acceptance by the product owner and/or customer at the iteration demo/review

Productivity of distributed teams

2008-11-26T11:54:00.000-08:00

Jeff Sutherland and Guido Schoonheim recently gave a presentation on the experiences on forming a team for Xebia distributed between Holland and India. Their shocking conclusion: a fully distributed scrum team has more value than a colocated one.

They claim the benefits of distribution are cost reduction, availability of talent, and scaling up/down with knowledge retention.

They claim their approach accomplished the following results on a relatively complicated system with 100k+ lines of java code.

95% of defects found within the iteration a feature was developed
only 50 defects found in acceptance phase
less than 1 defect for kloc
15 function points delivered per developer per month. Note that Mike Cohn has published results of a small co-located team that achieve 16 FP/dev/month

How did they accomplish this state of hyper-productivity, as they call it? Here are some of the highlights.

Bring the Indian team to Holland for 2 iterations starting the 2nd iteration. Establish personal relationships, shared agile value system, common mindset, mutual respect. My team at Envysion successfully used a similar approach; we had the Chinese team members on-site with the client for 2 months at the beginning. See my previous post for more info.
A few people from each site travel to the other location every 2 months or so to maintain the bond.
Staff with equally skilled people in all roles in both locations
Use video conferencing for daily stand-up meetings, retrospectives and planning meetings. The sprint demo was done only by the Holland team because stakeholders wanted it presented in Dutch language.
They started with a single team, then when they expanded to multiple teams, they seeded those teams with people from the first team to maintain the culture.
They maintained discipline to follow XP practices, including pair programming
Had a clear and disciplines Definition of Done for each sprint, which included 90% unit test coverage and fully automated functional and regression tests

Independent test teams

2008-11-18T10:43:00.000-08:00

The agileprojectmanagement discussion group on yahoo has a recent thread titled "Testing Sprint Advice" that caught my attention. (Note that you'll need to be a member of the group to follow the link.)

I believe the ultimate goal of agile testing/QA should be the complete and unambiguous definition of acceptance criteria. The ultimate manifestation of that would be executable requirements defined by the product owner. Of course the product owner would create these in collaboration with testers, and including developer input as appropriate.

There is often debate over whether test teams are more effective when they are independent from the development effort or when they actively collaborate with developers. I personally believe the collaborative approach is better. If the product owner and QA collectively define unambiguous acceptance criteria, then the developers are usually in the best position to automate the validation of those criteria through automated tests of various kinds. Since it's hard to formally define every single acceptance criteria, testers can focus much of their effort on exploratory testing - which of course may leading to a better understanding of additional acceptance criteria.

Steven Gordan makes some great points in the post, arguing for the collaborative approach which I'll summarize below..

1. If the testers effectively come up with their own version of the requirements based on an "independent" understanding of the product, then inevitable disputes over the actual requirements for the iteration ameliorate much of the potential value of independent testing....

2. If the testers actively participate in the development of the acceptance criteria for the iteration, then where is the independence? If they are going to actively collaborate with the team and customers from day 0 of each iteration, everyone would be better off if they were actually part of the development team.

3. If the testers passively utilize the acceptance criteria for the iteration without actively voicing their independent opinion about what is missing from the acceptance criteria, then we get the worst of cases 1 and 2. The testing is not truly independent, yet what independence remains is not being leveraged to improve the acceptance
criteria.

Inevitably, independent testing finds problems later than
collaborative testing would.

Thanks to Steven for the great insight.

The SEI addresses Agile

2008-11-15T05:46:00.000-08:00

The SEI recently published a paper that asks the provocative question: why not embrace both agile and CMMI? In an earlier post, I wrote that Perficient achieved level 5 using an agile approach, so I know it is possible.

Both CMMI and agile have a long history, it turns out. Although modern agile methodologies mostly emerged in the 1990's in the context of small teams, maybe the origins of agile and CMMI really weren't so different after all.

The paper traces the roots of agile back to Iterative and Incremental Design and Development (IIDD), a technique developed more than 75 years ago by engineers including W. Edwards Deming. It's also noteworthy that Demind is one of the fathers of the lean movement who is credited in part for bring lean to Toyota many decades ago.

An early progenitor of IIDD was Dr. W. Edwards Deming who began promoting Plan-Do-Study-Act (PDSA) as the vital component of empirical engineering. Early adopters of Deming’s teachings in the aerospace industry include NASA (National Aeronautics and Space Administration) and the US Air Force, each of which developed entire systems using time-boxed, iterative, and incremental product development cycles.

Origins of CMMI

everyone working to develop the initial CMM was looking for the solution to the “software problem” from the perspective that software is a component of a larger system and that if it failed, lives would be lost (e.g., aircraft, ships, weaponry, medical devices).Systems were evolved using careful and deliberate development paths according to lower risk,standardization-heavy and contractually-driven relationships between the developer and the customer.

The paper does a good job of explaining some of the reasons why the 2 camps are often at odds, which includes the fact that both approaches are often misused, which adds fuel to the fire.

Here's a paragraph that summarizes the paper's conclusion.

Agile methods provide software development how-to’s, purposely absent from CMMI, which work well on small co-located projects. Meanwhile, CMMI provides the systems engineering practices often required on large, high-risk projects. CMMI also provides the process management and support practices (and principles) that help deploy and continuously improve the deployment of Agile methods in an organization regardless of organization or project size.

I tend to agree, except that agile approaches can also be successful when small teams aren't co-located; I know because I've done it.

A final quote, reminiscent of Rodney King's famous "can't we all just get along?"

If those of us in both the Agile and CMMI camps would understand and accept our differences and explore the advantages of the other, we will find new ways of combining the ideas of both to bring improvement to a whole new level. Our challenge to CMMI and Agile proponents alike is to learn the value of the principles and technology of the other and begin to use them in their work

In my opinion, CMMI is too often misued to force a heavyweight, waterfall methodology on an organization for the primary purpose of marketing the organization's supposed capabilities rather than truly improving the organization's capability to sustainably produce business value for customers. That's why I will probably always be leary of any CMMI initiative started by non-engineer management folks.

Can a customer support team be agile?

2008-11-11T11:16:00.000-08:00

Mattias Skarin recently posted a useful 2-page summary of his approach for managing a support organization using agile techniques and a kanban board.

I recently led a team of Solutions Consultants (who supported our customers development efforts) at IP Commerce. The challenge we had was that there was ALWAYS a full backlog of customer support tasks, and those tasks are virtually always a higher priority than project-based work that the team needs to get done - stuff like improving the process, creating sample applications, writing documentation.

What I did to solve this dilemma was to allocate 20% of each person's time for project work, with 80% for ongoing customer support. The team members scheduled their time to carve out 4 or 8 hour time blocks during the 2-week iteration where they could focus on project work.

For iteration planning purposes, I kept a backlog of project-based user stories only - no customer support tasks - and the team planned how much of that backlog they could knock out in 20% of their time. The customer support stories/tasks weren't considered during planning - we knew those would flow in steadily.

This worked well for us, but let me know if you have other advice on managing support organizations.

Agile in the Extreme

2008-11-05T21:05:00.000-08:00

What happens when you take agile techniques to the extreme? At the Agile 2008 conference, a couple of Aussies, Paul King and Craig Smith, gave a presentation titled Technical Lessons Learned Turning the Agile Dials to Eleven. You gotta love the reference to Spinal Tap!

Back in my days at BoldTech Systems (now Perficient), we did hard-core XP. I mean hard core. We all had tables on wheels, no cubicles, no walls - the whole team in a "bullpen". We strictly followed all of the XP practices, and if we didn't, the VP of Engineering who was the uber-XP-Coach, would quickly swoop in and set you back on the straight an narrow path.

Some valuable lessons we learned:

Pair programming for 100% of production code isn't optimal. Many programming tasks are just too routine to justify it. For those, do frequent (at least daily) code reviews. Save pair programming for the tough stuff.
100% pair programming can be exhaustively intense. How many people in this world do you really want to sit shoulder-to-shoulder with for 8 hours a day? Or 4 different people for 2 hours per day? Most developers I know need some time to work alone to keep their sanity.
Continuous integration requires an enormous amount of discipline. If you have to stop everything whenever a single test breaks, and you're writing lots of tests, you better not have very many tests fail.
Don't forget to do design. XP doesn't emphasize design, but it doesn't preclude it, either. Do just the right amount of architechture and design - at the beginning of a release and each iteration.
Test driven development is good. Period. Go ahead an turn the dial to 11 on this one!

Poll: Does your company call people "resources"?

2008-10-30T20:23:00.000-07:00

Thanks to a suggestion left on my previous post, I created a poll to find out if companies are calling their employees "resources". Please take a minute to answer the poll at my blog home page.

Resources vs. People

2008-10-29T07:36:00.001-07:00

Is it just me, or is anyone else out there bothered by the use of the word resources to refer to a company's employees? Webster's defines resource as:

That to which one resorts or on which one depends for supply or support; means of overcoming a difficulty; resort; expedient

I suppose employees fit within that definition, but so do the desks and copy machines. Maybe it's my ego, but I like to be distinguished from inanimate objects in the workplace. The word resource to me is just too cold and inhuman.

Lots of organizations preach that people are their most important asset. It's easy to say, but how do you demonstrate that value?

By putting significant effort into designing a thorough and effective interview process, to make sure you hire the best people to begin with. How well organized is your company's interview process? And how often have you had to let someone go because they didn't turn out to be a good fit?
By respecting people within the organization. Give them clear objectives and establish a culture that encourages them to innovate and excel working as a team. Establish a culture of continuous improvement where every employee is truly empowered and expected to improve quality, process, and customer satisfaction. In Toyota, every production line worker is expected to stop the line if they find any problem, get to the root cause, and then correct it.
By growing people within the organization. Give people a clear path for career growth. Give them opportunities to try different roles within the organization. When you have a position to fill, look inside the organization first before looking for someone new.

A good indicator of how well your company treats its employees is their longevity within the company. Be wary of any company where the average employee has only been around for 1-2 years.

Is Agile a Fad?

2008-10-27T21:27:00.001-07:00

I attended today's Agile Denver meeting - this time in Boulder - to hear Mary Poppendieck's presentation, Is Agile a Fad?

I'll summarize some of her material here, mostly in reverse order, starting with the key points and conclusions of the talk, followed by some of the contextual info she presented leading up to those conclusions.

The key to successful development organizations

The key to a successful development is for engineers (developers) to have a deep understanding of their customers - both internal and external. When new engineers start at Toyota, they spend their first 6 months on the production line assembling cars, so they fully understand their internal customers. Then, they spend 6 months working for a dealership - selling cars - so they know what customers really want.

Another key is building a culture that retains quality people for the 6-10 years it takes to build true expertise, and growing leaders from within the organization.

Fads vs. enduring principles

Why do we have these fads that fail?

silver bullet thinking. there is no silver bullet
trying to apply 1 solution in different contexts. different contexts require different solutions.
essential tensions in software. Don't swing too far toward one side or the other; rather find a solution that solves the valid concerns on both sides of the issue.

The principles behind systems engineering are robust over time. The concepts in project management are fragile over time.

Systems Engineering	Project management
low dependency architecture	complete requirements
quality by design	quality by testing (at the end)
technical disciplines	maturity levels
respect for complexity	scope control
skilled tech leaders	resource mgmt
learning & feedback cycles	timeboxes
success = accomplishing system's objective	success = accomplishing planning scope, cost, schedule

5 essential tensions in development

People. self-managed vs. managers. Answer: the servant leader facilitating self-organization.
Process: empirical vs. defined.Solution: relentless improvement, rigorous process for effective improvement. Identify the true root cause of problems, hypothesize solution, determine how to measure if it succeeds.
Product: development team vs. customers & biz operations. Solution: whole team philosophy. team talks to customers so they understand the problem deeply.
Planning: evolving plans vs. predictability. Solution: pull scheduling, set-based design (build multiple options), clear technical vision
Performance: concern only for the next iteration vs. long-term scope, schedule & cost. Solution: a team with pride & passion that delights customers - and has a deep knowledge of it's customers' needs, sustainable profit, breakthrough innovation

A brief history of software methodologies and the seeds of agile

What happened to all those methodology buzzwords? RAD, lean, structured programming, etc.? Sprinkled throughout the history of software, various people discovered and promoted practices that we call agile & lean today. They also promoted various practices that were unsuccessful fads.

1968

NATO conference on the software engineering crisis. Edsger Dijkstra said that programming became a problem in relation to the size and complexity of computer hardware. Douglas Ross of MIT said the most deadly thing is the assumption that you can specify what you're going to do, and then do it. The solution (compared to assembly languages): high level languages! (Cobol, Fortran, etc.). This removed drudgery, but increased the level of complexity possible, which led to the same problem all over again(Dijkstra).

1972

New York Times software project

structured programming made software more readable. Dijkstra proposed quality by design, as opposed to reliance on testing.
Dave Parnas devised information hiding, the concept of objects.
Top down programming was introduced by Terry Baker - basically this was the concept of continuous integration.
The "chief programmer team" concept introduced the tech lead, design review, pair programming, common code ownership. Result was 100 times more productivity (measured in LOC) and higher quality than typical at the time.

1976

Barry Boehm proposed that software maintenance was becoming the most expensive part of systems, and that the cost of changes got exponentially greater in later phases of the lifecycle. This famous (infamous?) curve was the key reason that everyone tried to nail down all the requirements at the beginning.

1982

Daniel McCracken & Michael Jackson wrote that the lifecycle concept (waterfall) was harmful and perpetuates failure by constraining thinking, and ignoring the reality that needs inevitably change over time.
James Martin wrote the 4th gen languages would allow application development without programmers. (A gross oversimplification of the inherent complexity.)

1984

Scott Schultz at DuPont introduced timebox development. 30 days for analysis & design, 90 days to develop. He called it rapid iterative production prototyping.

1988

Boehm introduces the spiral lifecycle model. More evolutionary model, but it's still a project management model, not a systems engineering model.
Watts Humphrey introduced software process maturity model (CMM). Attempt to bring in statistical process control and mandate maturity assessments. Focus on project management practices over system engineering practices.

1991

James Martin wrote book on RAD, facilitated by CASE tools. (Where are those CASE tools today? Anyone?) Problem: RAD often produced un-maintainable code. Didn't live up to the hype.

1995

Internet booms. J.C.R. Licklider serves as the technical visionary for several key internet organizations. Standards were developed.

Measurement and management

2008-10-27T14:14:00.000-07:00

I've been following a thread on the Agile Project Management mailing list on measuring productivity. One of the posts makes reference to the oft-repeated axiom, "if you can't measure it, you can't manage it." Sounds perfectly reasonable, doesn't it? It also seems perfectly reasonable that you would want to manage, and therefore measure, the productivity of software developers and software teams. But as several people in this thread point out, there is really no good measure of software productivity. We all dismissed the notion of measuring lines of code a long time ago - at least I hope. Agile methodologies encourage the measurement of team velocity - how many features, story points, or estimated task hours the team completes in a particular iteration. I would argue, as did some in the discussion, that this measurement is properly used only to estimate the workload for the next iteration - it's "yesterday's weather". If it's used by management over the long term to measure team productivity then human nature dictates that the team will skew the estimates to generate a positive outcome.

I would argue that the best measurements for success are (1) customer satisfaction and (2) profitability. While it's true that true customer satisfaction can't be measured until the end of a development project, the product owner can provide interim measurements of satisfaction - as each iteration is delivered.

Scrum and XP top list of agile methodologies

2008-10-19T20:45:00.000-07:00

VersionOne released the results of The State of Agile Development survey for 2008. The most commonly used agile methodologies are Scrum and XP. Scrum, Scrum/XP hybrid, and XP together represent almost 80% of agile software development. 1.9% of respondents reported using Lean Development.

Simplicity defined

2008-10-17T07:56:00.000-07:00

One of the most important but most elusive principles of agile development is simplicity. In XP, it's stated as Simple Design, and other contexts often times referred to as KISS (keep it simple, stupid), and closely related to YAGNI (you aren't gonna need it). Simplicity is a key to successful agile development because it's absolutely necessary to support other agile goals and practices:

short iterations
refactoring
collective ownership
pair programming
avoid premature optimization

In turn, simple design is enabled by test-driven development and refactoring.

The biggest problem with the simplicity principle is that people often disagree on what constitutes "simple", and I've always struggled to come up with a definition of simplicity that was, well, simple. I came across a quote today that I think sums it up pretty well, from Antoine Du Saint-Exupery.

Perfection is not when there is no more to add, but no more to take away.

Lean Thinking

2008-09-30T19:51:00.000-07:00

Lean Thinking, by James Womack and Daniel Jones, is a good introduction to Lean theory and presents some compelling cases studies of enterprises - from small and simple to huge and incredibly complex, such as Pratt & Whitney - that made the transformation to Lean. I haven't finished the book yet, but can summarize the primary thrust of lean thinking as eliminating waste (muda, in Japanese). The foundational lean principals are:

Value
Value stream
Flow
Pull
Perfection

Value can only be defined by the final customer, in relation to a specific product, service, or both. It seems like an easy concept, but I challenge you to define your organization's value in a single short statement.

The Value Stream is the complete set of activities required to design a product, produce the product, and manage orders for the product. The value stream typically extends beyond a single company to all of its suppliers. To achieve the ultimate goal a lean enterprise must be formed through cooperation with suppliers. Eliminate muda in this stream.

Flow refers to the practice making value-producing activities flow together to dramatically reduce the time required to produce product. It commonly requires removing departmental barriers and changing mentality from producing large batches of parts to producing complete products at the same rate that customers are ordering them.

Pull techniques dramatically simplify the planning and scheduling process. Don't build the product until the customer orders it by pulling it from production. Each downstream step in the production process in turn pulls from the step immediately upstream from it.

Perfection means, quite simply, that the goal is not settle for doing better than the competition, but to continually strive for improvement (kaizen). You can never reach true perfection, but you can get asymptotically close to it over time.

I'll dig deeper into these concepts in later posts.

5 levels of planning

2008-09-09T11:25:00.000-07:00

Yesterday I attended the Agile Denver meeting for a presentation by Hubert Smits on the 5 levels of planning. I was already aware of the 5 levels and I like to use all 5 levels, but it's always good to reinforce the principles behind the practices and hear from other agile practitioners out there. Here are the 5 levels, their suggested frequency, and the content of the plan.

Product vision: annually. A 1-2 sentence statement. The "elevator pitch".
Product roadmap: 1-2 times/year. The major themes of each release.
Release planning: 3-6 times/year. The user stories/features for 1 release.
Iteration planning: each iteration. With practice, most agile teams choose 1-3 week iterations.
Daily planning (stand-up or daily Scrum meeting): the 3 questions (what did I do since last meeting, what will I do today, and any impediments)

Training for product owners

2008-08-08T06:13:00.000-07:00

There are about a hundred Scrum Master courses for every one Scrum Product Owner training course. Yet the product owner is an absolutely key role in agile/Scrum projects. During the Agile 2008 conference, some people from Innovel presented on the product owner role and have made publicly available some great material for training product owners. It's a hands-on role playing exercise based on a hypothetical product called Beer Miles, a credit card that gives users reward points for purchasing beer from participating retailers. The product-owners-in-training are presented with a business case for the product and a set of product capabilities (aka user stories or features) that they must categorize and prioritize.

Cheers to the folks at Innovel that made this available!

The Goal

2008-07-10T20:25:00.000-07:00

I finished reading The Goal by Eliyahu (Eli) Goldratt. This is classic business novel about the Theory of Constraints (TOC), often cited in Lean and Agile literature. Written as a novel, it's an enjoyable read, and a must-read, I would say, for anyone who is serious about improving the way his business operates.

I always like to distill a good book down to it's bare essentials, so hear goes.

The goal is to make money. There are 3 fundamental measurements that express the goal, listed in order of importance.

Throughput: the rate at which the system generates money through sales.
Inventory: all the money the system has invested in purchasing things it intends to sell.
Operational expense: all the money the system spends turning inventory into throughput

The aim is to maximize throughput while minimizing inventory and operational expense.
Note that in software, inventory is any software or feature that is unfinished or not yet delivered to customers.

Stated differently:

Throughput is money coming in
Inventory is money stuck inside the system, or investments that potentially could be sold
Operating expense is money going out (to make throughput); any investment that can't be sold

Note: Agile software development reduces inventory by building software in small batches (iterations) that are quickly delivered to customers.

There are 2 types of resources:

bottlenecks (a.k.a. constraints): capacity <= demand
non-bottlenecks: capacity > demand

Balance the flow of product through the system, not capacity, with market demand. Make the flow through the bottleneck equal to market demand. A system needs to have excess capacity to handle the fluctuations in demand and variations in output from each resource in the system.

Activation vs. utilization of a resource:

utilizing a resource is using it in a way that moves the system toward the goal
activating a resource is using it whether or not there is any benefit from it's output.

Two rules of bottlenecks and non-bottlenecks:

The level to which you can utilize a non-bottleneck resource (without increasing inventory) is determined not by the capacity of that resource but by some other constraint in the system.
Activating a resource is not the same as utilizing it; activating a non-bottleneck to its full capacity is counter-productive with respect to the goal.

The implication: you must not seek to optimize every resource in the system. A system of local optimums in not an optimal system; often it is a very inefficient system. Optimize the whole system; not localized subsystems. [Lean principle: see the whole.]

The process for accomplishing the goal:

identify the system's constraints (bottlenecks)
decide how to exploit the constraints; maximize their utilization
subordinate everything else to the above decision. Operate all other components to maximize utilization of the constraint.
elevate the system's constraints; add resources or otherwise increase capacity of constraint resources
if in the above steps a constraint has been broken, go back to step 1. Do not allow inertia to cause a system's constraint. Whenever a constraint is broken, immediately re-examine conditions included the changes made in steps 2-4; they may now be problematic.

Effective management seeks answers to these 3 questions:

What should be changed
What should it be changed to
How to cause the change - without creating new problems, and with enthusiastic support

Perficient achieves CMMI Level 5 using Agile

2008-05-27T10:17:00.000-07:00

Perficient, the company where I recently worked (formerly BoldTech Systems), recently achieved CMMI Level 5 certification at their development center in China using an agile methodology. I worked in the China facility just over a year ago when they achieved CMMI Level 4 and wrote about it in a previous blog post. Perficient is now one of the very first companies to achieve level 5 using agile.

I don't advocate pursuing CMMI certification unless it's a business requirement. There are many offshore (primarily Indian) firms who tout their CMMI Level 5 certification, so it may be an important factor when competing for certain contracts.

Regardless, this is a great achievement and an important one for the agile community, whether you like CMMI or not.

Are you really doing Scrum?

2008-04-29T07:01:00.000-07:00

Jeff Sutherland, one of the founders of Scrum, has spoken about the Nokia Test - 8 questions to determine if a team is actually doing Scrum. A more succinct summary can be found here. Can your team pass the test?

Stealth agile and agile contracts

2008-04-28T20:45:00.000-07:00

I attended tonight's Agile Denver meeting, which was a presentation by Richard Lawrence from Avenade entitled Stealth Agile - how to implement agile techniques when you don't have full -- or any -- buy-in from management. It was a short preso, and the discussion after the formal part was informative. One audience member asked how to introduce agile practices on a project which is architecture-centric and project leaders tend organize developers' work around components or layers, rather than features that provide end-user value. The suggestion from the audience was to build a small feature as a proof-of-concept designed to expose risk. Architects generally favor proofs of concept, and they also generally like the idea of finding and reducing risk. Note that you can present this idea without even using the word agile or any of it's practices. Sneaky, eh? One caution though - when people see that your POC works, it'll likely end up in production, so built it with production quality, including automated tests.

One of the most interesting questions, I thought, had to do with a decidedly non-stealth agile issue, which is, how do you write a contract to be agile from the beginning? Richard's response was that traditional contracts are typical very scope-centric; they focus on fairly low-level details about what software features will be built. For an agile contract, he advocates one that focuses on the product vision, and just enough scope to specify who will own the intellectual property of different parts that get built - something the Avenade legal team insisted on for their consulting contracts. The agile contract specifies the vision, a team size, and a time frame in which the contractor will endeavor to achieve the product/project vision, allowing customer and supplier to collaborate on refining the specific features which best achieve that vision - through iterative development of working software and the feedback that results. I suppose this requires a fair amount of trust between the parties, but if enables success, I'd say a little trust is a small price to pay.

Lean foundation of Agile Methodologies

2008-04-21T19:10:00.000-07:00

Agile methodologies such as Extreme Programming and Scrum emerged in the 1990's as a radical departure from traditional, waterfall software methodologies. But were these agile methodologies really so new and radical? Many thought leaders have recently made the point that agile principles and practices are a software manifestation of the principles behind the lean product development strategies applied so successfully by Toyota starting decades before the agile software movement began in earnest.

Listed here are the seven principles of lean software development as identified by Mary and Tom Poppendieck in their book Lean Software Development: an Agile Toolkit and their agile counterparts from the Agile Manifesto (1), Scrum (2), and Extreme Programming (3).

Eliminate Waste

Working software is the primary measure of progress (1)
Simplicity - maximizing work not done - is essential (1).
Simple design - YAGNI (3)
The most efficient method of conveying information is face-to-face conversation (1)
business people and developers work together daily (1)
XP planning game (3)
Test-driven development (3)
Continuous integration (3)

Amplify learning (feedback)

early and continuous delivery of valuable software (1)
business people and developers work together daily (1)
Scrum sprint reviews held with all stakeholders (2)
XP - small, frequent releases (3)

Decide as late as possible

Welcome changing requirements, even late in the process (1)
Scrum product backlog - prioritized prior to each sprint (2)
Sprint planning / XP planning game (2) (3)

Deliver as fast as possible

deliver working software frequently (1)
potentially shippable software at the end of each short sprint (2)

Empower the team

Build projects around motivated individuals...trust them to get the job done (1)
The best...designs emerge from self-organizing teams (1)
Scrum self-organizing teams and Scrum master as servant leader (2)

Build integrity in (to delight customers)

Our highest priority is to satisfy the customer through early...delivery of valuable software (1)
Continuous attention to technical excellence and good design (1)
Design improvement / refactoring (3)

See the whole (optimize the whole system, don't sub-optimize)

At regular intervals, the team reflects on how to become more effective (1)
Scrum - sprint retrospective (2)
Design improvement / refactoring (3)
sustainable development - should be able to maintain a constant pace indefinitely (1) (3)
Design improvement / refactoring (3)

When agile is not a natural fit

2008-04-17T19:20:00.000-07:00

For a while now, I've been leading a team in a situation where textbook agile practices aren't a natural fit. My company, IP Commerce, has built a platform to enable electronic payments. Applications (built by third parties) connect to our platform to gain access to a variety of payment services such as credit card processing, electronic check processing, and e-commerce services such as PayPal. My team is responsible for integrating the various payment services with the IP Commerce platform. Each integration is called an adaptor, which translates messages (transactions) from the IP Commerce format to the service provider format & protocol. Why is it that this type of development doesn't easily fit into the typical agile model?

The scope of each adaptor is essentially fixed
Each adaptor must be certified before it can be deployed
The duration of each adaptor project is 6-12 weeks

Let's examine each issue in more detail.

Scope
Each adaptor translates messages, and to accomplish useful business functionality, there is a minimal set of messages that it must support. For example, in credit card processing, a customer must be able to do authorizations, voids, refunds, and settlements. Without all of those features, it doesn't meet any customer's minimal requirements. The only scope which is negotiable is minor features, such as corporate purchase cards or certain industry-specific features, for example those that support the lodging industry.

One of the core presumptions of agile is that scope is negotiable and features can be prioritized. If 90% of the scope is fixed and all the features (in this case, message/transaction types) have the same priority, the backlog isn't very interesting.

Certification
Agile methodologies, Scrum in particular, assume that a product can be deployed when the product owner judges it to have enough functionality completed. In our case, an adaptor cannot be deployed until the service provider to whom we're integrating certifies it. Not to mention that, as explained above, we need essentially all of the features working before it is useful to customers. This dependency on an external verification process is unavoidable, and unfortunately, it often takes a long time.

Duration
Each adaptor project lasts between 6-12 weeks. Most agile projects I've been involved with before now have been much longer, and have many more iterations that establish the all-important feedback loop and team rhythm.

Adapting agile to the situation
One approach we have tried is to treat each adaptor as a single coarse-grained feature in the backlog. This makes sense because these are the units of functionality that the business prioritizes, but on the other hand it doesn't make sense because agile features (user stories or backlog items) need to be small enough to fit into a single iteration (sprint). That disadvantage is a big one in my mind, so we have decided it's not a good approach.

Instead, we have decided to choose smaller backlog items, which for a single adaptor include a combination of true user-stories (message types and other user-identifiable features) and milestones such as 3rd party certification. In the past, I have been a big proponent of using a consistent sprint/iteration duration. In this situation, however, we've found it to be useful to first identify a concrete objective for each iteration (e.g. complete the first 2 message types, or achieve 3rd party certification) and then choose the sprint length based on the tasks required to achieve the objective - while keeping each iteration to 4 weeks or less.

It still feels awkward at times, but I feel we benefit greatly from relatively short iterations.

Rolling back offshoring

2007-05-29T07:49:00.000-07:00

For some companies, offshore development, particularly in India, is losing the financial advantages it once had. Software developer wages in India are overall about 30% of what they are in the US, but top performers are able to demand 50% to 75% of US salaries, with 15-30% annual increases, according to this story.

"If you're a startup and you're doing a lot of iterative design and you constantly need a higher level of agility, you might find [offshoring] more burdensome than the cost differential is worth," said Fran Karamouzis, a research vice president at Gartner who specializes in outsourcing issue.
On the other hand, "If you can give the Indian team things that are clearly delineated that they can own and drive on their own, then it works out pretty well," said Pejman Roshan, cofounder and senior director of marketing at Agito Networks, a wireless technology startup in Sunnyvale, Calif. that outsources software coding to a services firm in Bangalore.

In addition to the communication challenges introduced by different cultures and a 12-hour timezone difference, multiple power outages every day eat into efficiency of Indian teams.

An offshore agile success story

2007-04-05T21:14:00.000-07:00

In a previous post, I briefly mentioned the success my team had applying an agile approach on an offshore project. Here is some more about what we did that worked well for us.

In agile methodologies, a co-located team is critical to success - a prerequisite, some would argue. But what if you're stuck with a geopgraphically distributed team? Even worse, what if part of the team is offshore, 15 time zones away, it's members a part of a very different culture, and their English comprehension is limited? Is it possible to be agile in this situation? If so, how?

This is the situation I faced on my recent project. The onshore (US) team was colocated with the customer, and consisted of a project manager, a business analyst, and 2 architects/senior developers. The offshore contingent was located in Hangzhou, China and was a team of 8-10 developers and testers.

The project: build a leading-edge video streaming application using linux, MySQL, Java, javascript, and a few other technologies (Perl, PHP) thrown in for good measure.

The result: we delivered a high-quality application on time, to rave reviews from end users.

How did we make offshore agile work? To some extent, by being less agile, and less offshore. Here is how we organized the project for success.

Bring your offshore team onshore and have a single colocated team - at the customer site - as early as possible, and for as long as you can. We had our entire China contingent onsite for 3-4 months at the beginning of the project.

Note: obtaining visas can be tricky. We used B-1 visas, although we were pushing the limit on what can be done with B-1's. H-1B and L-1 visas are possbile in some cases. J-1 visas are another option, and there may be organizations that can help you arrange these. BoldTech worked with CAST, the Chinese Association for Science and Technology USA, to obtain L-1 visas for some people.

Share the vision. The entire team should know and be continually reminded of the vision and road map for the project. What is the end goal? Who are the end users? What are the key features in the next release?

The offshore team participates in - and drives - demonstrations to the customer. Delivering potentially shippable software each iteration has a very tangible meaning to developers if they know they will be personally demonstrating it to the customer. We got lax on this and stopped doing it after the team went back to China - looking back I kick myself for not making it continue.

Send US team members to your offshore location. Do this as often and for as long as you can. When the developers returned to China, I spent 4 months in China as the team lead, and brought my family with me. It's hard to find people willing to take that assignment, but you can send various US folks on a rotating basis for shorter visits if that's the best you can do.

Sufficient analysis and documentation. Once the offshore team was offshore again, scribbling a few notes on a note card wasn't sufficient to communicate requirements. The developers comprehension of spoken English was limited, so thorough requirements documentation became very important. And unless you like to answer the phone at 3 am to answer questions from China, I recommend that you have good-enough documentation. We combined screen mock-ups, narrative text, and use cases, as appropriate for each feature / user story.

Online collaboration tools. We used XPlanner, Mantis, and Mediawiki. For meetings and communcation, we liberally applied phone calls, web conferences, IM, and email.

Shift work schedules so working hours between locations overlap

Adjust your communication style. Listen closely to a conversation between two native English speakers, and you'll quickly learn that a huge proportion of what we say is idioms, expressions, and sayings. "Are we on the same page?" "Don't go down that rat hole." When you're talking to folks from China, you can't talk that way. Slow down, use common, simple words, and avoid the idioms.

New gig at IP Commerce

2007-04-02T17:35:00.000-07:00

I accepted a job offer today to be Director of Integration Delivery at IP Commerce, a company based in Denver. The company is building a software platform to enable and simplify electronic payments over the internet. So what will I be doing? Building and managing teams of developers that will build software to connect the system to various banks and payment service providers. I'm excited about the opportunity, and I'll be among friends - I've worked with many people there in the past.

I've enjoyed my 2nd tour of duty with BoldTech, and the unique opportunity I had to work in China and build a cool video web site at the same time while using agile techniques. This new position should be a great opportunity for me, though, and I look forward to it.

It's a bit premature for me to say how agile methodologies will figure into the work, but I will write more about it later.

A successful agile offshore project

2007-03-30T22:34:00.000-07:00

Since July, I've been working for BoldTech Systems as a consultant to Envysion, a start-up company in Boulder that provides web-based video management solutions. This is the project that brought me to China to lead a great team of developers in building Envysion's new web-based video application. We officially released the web site to the public this week, although several customers had been using it prior to then in a beta version.

I lead the team using agile principles and practices, and the result is a happy customer and very enthusiastic end-users.

Is "agile offshore" an oxymoron? To the hard-core agile fundamentalist, maybe so. With a distributed team and a remote customer, you need to try harder to be agile, and compensate for the obvious drawbacks of the situation as best you can. This project shows that a practical approach based on agile principles can yield fantastic results.

Check out the site here! Note - you will need to use IE on Windows XP. We'll be adding support for other browsers and platforms in the future.

Grass-roots Agility

2007-02-28T11:55:00.000-08:00

This Monday, Feb. 26, I attended the Agile Denver meeting for a presentation by Greg "Hap" Pearman entitled Being Agile in a Non-Agile Space.

The gist of the talk was, how can one be agile if the company you're working with isn't embracing agile methodologies? Greg advocates a grass-roots effort.

Here are Greg's suggestions.

1. Have courage
- many agile initiatives are grass-roots
- to know when it's best to quit or start over
- don't be afraid to fail

2. you need to really know your stuff to push agile
- how and why agile works
- sharpen the saw

3. Set expectations
- communicate often and early

4. Lead by example
Build trust

5. Start by fixing the biggest pain points of the process
- don't start with the easy/small stuff!
- biggest ROI

6. Understand dependencies between agile practices.
- If you don't do one practice, how will you compensate?

Based on his experience at US West / Qwest, Greg has the following advice if you will attempt to transition a large organization to agile methodologies:
- go fully agile from the start, and give team lots of support to succeed
- agile practices are interdependent

Agile CMMI blog

2006-11-24T07:38:00.000-08:00

From a comment on this blog, I discovered the Agile CMMI blog. Seems like a great resource to find out about the growing interest in reconciling these 2 worlds.

Agile CMMI - it is possible

2006-11-17T08:12:00.000-08:00

BoldTech Systems just achieved CMMI Level 4 at their Hangzhou, China office, where I'm currently working as an agile coach. And they did so legitimately, while following essentially agile practices. I say legitimately because many companies appear to finagle a CMMI via a smoke-and-mirrors appraisal process. In fact, BoldTech's Hangzhou organization went from level 1 to level 4 in just 2 years.

BoldTech was a pioneer in the adoption of agile methodologies, practicing hard-core XP when I starting working there in 1999. And I do mean hard-core, including 100% pair programming. Since then, the methodology has evolved from pure XP fundamentalism to a pragmatic adaptation of agile techniques based on years of hard lessons learned. Some of the practices in the trenches are not nearly as agile as I would like them to be, but on the macro level it's definitely agile.

I'm not a big fan of CMMI, but that's primarily because it's so often misapplied to enforce a rigid process for the sake of the process, and not because the process provides value to customers. If you keep the right focus and do a sensible tailoring of the CMMI practice areas, you can be agile and CMMI compliant at the same time.

The seating chart

2006-10-27T04:55:00.000-07:00

What formation is your team in during the daily stand-up meeting? Hopefully it's close to a circle, where each team member can make eye contact with every other team member. Many studies have shown that a significant amount of the information conveyed in a conversation is non-verbal, so make sure everyone is getting the full story.

Is everyone facing the leader/scrum master? This could be a problem. The leader should be a facilitator for the meeting which is held primarily to share information among all the team members. It's not just a status report to the leader/facilitator.

For longer meetings such as iteration planning sessions, pay attention to the seating; the same principles apply. Arrange chairs in a circular or semi-circular fashion.

The 3 Scrum questions and the language impediment

2006-10-18T06:39:00.000-07:00

"What is your name?"
"What is your quest?"
"What is the velocity of an unladen swallow?"

Wait, sorry - those 3 questions are for the bridge of death - they have nothing to do with agile software development.

One of the 3 questions each team member answers during the daily Scrum (daily stand-up meeting) is, "are there any impediments to your work?"

I am currently working in China with a team building a web app for a US-based client. One of our impediments is the language barrier. In fact, the word "impediment" itself isn't in the vocabulary of my Chinese colleagues. So how do you ask if there are any impediments?

I ask if they have any "problems or issues", but that's not quite the right. As the scrum master/agile coach, I've found that the following question hits the mark:

What can I do to help the team?

With those eight short words, I've found that I get much more insightful responses, and can root out some important impediments.

One developer per feature

2006-10-16T06:30:00.000-07:00

When you're planning an iteration, you start with user stories, break them into tasks, make estimates, commit to the iteration scope, and then begin working (as a collaborative, sel-organizing team, of course).

My past 2 project teams have both concluded that it's better to have 1 person (or pair of programmers) develop an entire feature than to have different people working on different sub-tasks. Avoid the temptation to break a story into layered tasks such as "build the DB", "build the domain objects", and "build the UI" -- with each task worked by different people. You may have people who specialize in only one of these layers, but your team will be more productive and flexible in the long run if everyone has broader skills. Besides, having 3 different people working on 1 feature requires lots of coordination and presents lots of opportunities for miscommunication. Instead, have one person (or pair, for you pair-programming zealots) build the whole shebang.

In OO development, we aim to minimize coupling between components and maximize cohesion within components. This is the same concept. Each developer is a component of the team, and you want to minimize the amount of coupling required between developers; when coupling (communication) is required, you use the most effective means of communication possible - face to face conversation. At the same time, you want to maximize developer's internal cohesion - by letting him/her focus on a single feature.

Of course, to do this, you need to have stories/features which are small enough for 1 person to complete within a single iteration. But that's a worthy goal for many other reasons, too.

The 7 Habits of Highly Effective Agile Teams

2006-08-12T17:10:00.000-07:00

After reading Stephen R. Covey's The 7 Habits of Highly Effective People, I was struck by the commonalities between some of the principles in his book and those of agile development.

This is how I see the two come together.

Habit 1: Be Proactive

The essence of this habit for me can be summarized with a single quote from Covey:

Our behavior is a function of our decisions, not our conditions.

When the team is subject to outside influences, changing priorities, and “scope creep”, it can complain, or it can:
1. choose to explain to these outsiders from management, marketing, etc., the consequences of changing direction in mid-iteration
2. choose to drop lower priority features
3. choose to abort the sprint/iteration and plan a new one
4. choose to accept that stakeholders, not developers, have responsibility and accountability for setting product priorities

A strong product owner and scrum master should provide the leadership and courage to make this happen.

Habit 2: Begin with the end in mind

Agile teams should start a project, a release, an iteration, and each day with a clear vision of the goal. The 5 levels of planning provide that vision:
- Product vision statement
- Product roadmap
- Release plan
- Iteration/sprint plan
- Daily plan (from daily stand-up/scrum meeting)

Habit 3: Put first things first

This habit emphasizes personal management to prioritize your life; allocate plenty of time for activities which are important, but not urgent (What Covey calls “Quadrant II” activities).

Scrum and XP both emphasize the importance of prioritizing. XP in particular, with it’s rigorous discipline around engineering practices, ensures that the team is investing time to achieve a high and sustainable velocity.

Particularly, the XP practices of refactoring and test-driven development are prime examples of Quadrant II activities. In Scrum, an effective product owner, scrum master, and team should collaborate to ensure that the product backlog and spring backlog address not only the highest priority business features, but also technical activities required to maintain a productive team environment.

Habit 4: Think win/win

Because agile teams are:
1. Self-organizing and self-motivated by a personal commitment to achieve the sprint goal
2. Not bound by a rigid and prescriptive process
3. Able to quickly adapt to changing business priorities
4. Empowered to insist on high quality
5. Working in a sustainable manner

Then they are free to seek solutions which meet the needs of customers and at the same time provide developers with opportunities to learn, grow, and produce the highest quality product.

Habit 5: Seek first to understand, then to be understood

An agile team’s number one focus is to deliver value to the customer. The team collaborates closely with customers to fully understand the product, it’s goal, and it’s features. Once the team has this understanding, and has proven that it can quickly deliver value, then it will build the trust required for those occasions when negotiating with customers and stakeholders is required. (We value customer collaboration over contract negotiation. But we still gotta negotiate sometimes!)

Habit 6: Synergize

Agile teams are deliberately cross-functional; they consist of people who specialize in fields such as database development, UI development, and testing. In agile development, we intentionally throw people with these diverse skill sets into close quarters and “shake well” to ensure that they collaborate closely. Agile teams truly value the specific strengths of each team member and seek to get the maximum synergistic benefit from the creative combination of everyone’s skills.

On agile teams, we don’t stifle creativity and cooperation by putting team members in competition with one another; we reward the entire team for meeting it’s collective goal. We don’t manage by command-and-control; we let self-organizing teams use their collective wisdom to find the best method to reach the goal.

Habit 7: Sharpen the saw

This habit is about the importance of physical, mental, spiritual and social self-renewal.

Agile teams have frequent retrospectives. This is the time when the team re-examines the effectiveness of it’s practices and reconsiders it’s activities in light of the project goal and agile principles. This is one form of renewal.

By creating a culture and environment which values sustainable development, individual self-renewal is possible. But it should also be encouraged. An agile team should also provide team members the personal time to research technologies, tools, and topics that interest them and will expand their knowledge and understanding. Even if the topic doesn’t have an apparent application on the current project or iteration, the investment of time will pay big dividends in the long term.

Managing your personal life with Scrum

2006-07-18T15:26:00.000-07:00

Pete Deemer posted this thought-provoking message on the yahoo scrumdevelopment group about using Scrum to manage his personal time. His "life backlog" consists of everything from chores, to exercise, to quality time with his family.

Sprint planning takes place weekly, where he typically commits to about 10 hours' worth of personal tasks. He always keep some free time unallocated, to maintain his spontaneity (and sanity). He also has a daily scrum meeting with himself each day, and tracks his personal burndown chart on the kitchen wall.

I love the idea of using Scrum's simple techniques for organizing one's life. What do you think?

Book summary: Collaboration Explained

2006-07-13T07:39:00.000-07:00

I read Jean Tabaka's book: Collaboration Explained: Facilitation Skills for Software Project Leaders. It describes many techniques an agile leader should keep handy in his/her toolbox - I recommend it.

Here is a brief summary of some of the salient points from the book.

Collaboration and servant leadership

In a collaborative culture, teamwork and self-organizing teams are highly valued. A collaborative culture is on the opposite end of the spectrum from a command and control culture. Leaders seek to get the best out of the team by optimizing their collaboration and protecting the team from outside influences which work against collaboration. The agile leader should be a servant leader:

...in a position of strength, they determine that the greatest power they can wield is in service to their teams as leader.

Some attributes of a servant leader

Goal setting to guide the team rather than prescribing tasks
Listening as a means to lead - ask questions!
Clear sense of values
Personal growth
Tolerance of imperfection - trust the team's decisions
Positive attitude

Evolving toward a high performance team

Teams go through 4 stages on the route to high performance, and they need different leadership techniques at each stage.

Forming
Storming
Norming
Performing

Characteristics of a collaborative team

Self-organizing (vs. title or role-based)
Empowered to make decisions
Members believe they can solve problems for themselves
Committed to success as a team
Motivated by trust, mutual respect and mutual accountability
Engage in participatory decision making
Decisions are consensus-driven
Constructive disagreement is healthy

Preparing for collaborative meetings
A good rule of thumb is for the leader/facilitator to spend 2 hours preparing for every 1 hour of meeting time.

Interview the meeting sponsor
Survey participants
Set the agenda

Conflict resolution and Consensus building
Teams should strive for consensus on major decisions, but it can take considerable time and effort to reach a consensus. Teams should decide in advance which decisions require consensus and which can be decided more quickly (by majority vote, for example).

Use the "fist of five" technique to measure consensus. When someone in the group has proposed a decision, ask everyone to simultaneously to show a number of fingers indicating their level of agreement:

5 fingers: I'm wildly enthusiastic about it
4 fingers: I like the idea and fully support it
3 fingers: It's not perfect, but I can support it
2 fingers: I don't like it, and can't fully support it
1 finger: I am very much opposed to this idea

Thomas-Kilmann conflict resolution mode
The following lists the modes and how often teams should strive to resolve conflict via that mode:

Collaboration: 65%
Compromise: 20%
Avoidance: 10%
Accomodation: 5%
Competition: never!

Leadership and facilitation techniques
The book is full of useful techniques in these areas:

Meeting organizing tools, e.g. the parking lot
Kicking off a meeting
Gathering information
Project & task estimating
Processing and organizing information
Retrospection and visioning
Managing meeting participants
Managing conflict
Closing a meeting

Action plans
"A bridge to each desk"

Practices for distributed teams

Selenium: the 10-minute test

2006-06-29T09:49:00.000-07:00

I spent some time today getting familiar with Selenium, an open-source web app test tool. Along with the Selenium IDE, which runs as a Firefox plug-in, it passed the 10-minute test with flying colors. It looks promising, and soon I plan to put it through a more rigorous test to see how well it handles javascript and DHTML.

Switching roles: scrum master to developer

2006-06-27T09:19:00.000-07:00

I'm the Scrum Master (team leader) for a team of 5 developers working on a complex enterprise application at StorePerform Technologies. I haven't worked very closely with most of the team members prior to this project, but most of them worked on the same project for a long time. I started the project hoping that I could serve both as the team leader (following the Scrum principle of servant leadership) and a part-time developer. I reluctantly came to the realization that it doesn't work, at least not with a new team which is still in the Forming & Storming stage. As my colleague Yuri Gadow helped me understand, the likely result is that I will be a bad leader and a bad developer - the worst of both worlds.

The team has taken the concept of self-organizing teams to an extreme, and they resent any suggestions from the Scrum master. My goal is to gain more trust and respect from the developers so they'll be more open to my suggestions. My solution was to switch roles for a few sprints (iterations) - I will be a full-time developer, and hand over the Scrum Master duties to Yuri. I believe this approach has several benefits:

I can focus on doing one thing well: writing code
I can keep my development skills honed
As an equal peer to the other team members, they'll (hopefully) learn to respect my technical skills and experience
I have the opportunity to "feel their pain" first-hand
The team (including me) will experience a different leadership style from Yuri

After nearly 4 weeks in the trenches, I think it's working well. Unfortunately, StorePerform cut the experiment short when they laid offthe entire US development team and outsourced it to India.

Outsourced!

2006-06-26T21:14:00.000-07:00

My job got outsourced to India. The rumors had been floating around StorePerform for a few days that something was up, and I figured there would be a few layoffs, but I got cold cocked. I didn’t expect that they’d offshore the entire product development, QA and support teams to India.

“I hope you got a good severance,” several people told me. Not a chance. Not at SP. Taking care of employees was never high on the executives’ list of priorities.

Fortunately, I’ve got a good network of friends and colleagues and it shouldn’t be too difficult jump back out of the unemployment line.