Becoming Lean

I've spent the past three years building up an internal software development team. This team was built with a foundation of principles instilled by XP-styled Agile (or as they were ingrained in me by ThoughtWorks). The team has changed dramatically over the three years, ever increasing in size, skill and agility. We are currently running with what I would consider to be a dream team. Each team member is a skilled and experienced generalist and everyone on the team either has 2+ years of agile experience or is an apt and enthusiastic participant in their own learning process. I couldn't be more excited about our future prospects and what we will deliver to the business in the coming months.

But even fielding this roster we experience some Agile anti-patterns that turned into pains for the team. An analysis of these anti-patterns is leading us towards a Lean Kanban style and away from the traditional Agile. I know there is not much out there about making this transition so I will try to document as much as I can to help others who are attempting the same thing.

The first of our pains was that we could not get a stable velocity. It was not uncommon for us to have a huge variance between iterations... embarrasingly so.

Iterations always seemed to result in hangover and the perpetual work in progress made testing very difficult. Not to mention cutting releases was difficult when they contained half completed work.

When we switching into maintenenance mode between phases everything went out the window. We moved into chaos mode and during this time there was some team churn (for the better) that changed the entire dynamics of the team so any historical metrics were rendered useless. Not to mention our rhythm, and I believe the inertia behind a strong rhythm is important to a dev team.

Finally, while our development team is Agile the business is hunkered down in a Waterfall mindset. They fear fast and cyclical releases opting for several big-bang releases per year. Admittedly we haven't done much to quell those fears since the work in progress problem plagues our own testing process. But even if those fears were gone the business has a gated approval process better suited for the space shuttle than it is for software.

We tried to band-aide these pains. For example we tried to fix the varying velocity by accounting for half-completed stories (Story A is %65 complete) to give a more realistic velocity. We injected empty buffer iterations to allow for work in progress completion and stabilization. We tried to estimate bugs and maintenenance work but unfortunately metrics taken during this span were used to estimate the schedule dates for future development work. As for the waterfall business problem there was unfortunately nothing we could do to reduce the release cycles. Perhaps the real problem is that changing the business is beyond the team's sphere of influence at this point and we have to earn the business' trust before that can be fixed.

After looking at these problems from a root-cause perspective we have decided to focus on the following principles:

1. Reduce work in progress
   
2. Zero defect mentality
3. Kaizen

We've also decided to switch to an iterationless Kanban system because it more closely represents the way we are working. In a subsequent post I will go through these principles and describe how we expect them to work in concert with Kanban to improve our process.

To be continued...