A recent post on the Scrum Development list detailed a way of dealing with bugs in scrum. While it’s a bit unconventional, in that the sprint backlog changes mid-sprint, I’ve got say I like it. Might even be adaptable to dealing with other priorities changing mid-sprint (genuine new, urgent, user stories) which is more likely to happen to me.
The background is this. 2+ years ago the organisation decides to re-develop the core software product. The CTO initiates several changes, the key ones being adoption of scrum and adoption of new language/platform in RubyOnRails.
This is a cultural change for everyone. CTO has spent time explaining agile methodology at board level and the board appear to have bought into it. Development team also spend time getting up to speed with scrum, rails and TDD, first on a test project then on the real project.
Product Ownership is an issue from day 1. Ownership goes back and forth several times over the life of the project, between board, senior operational staff, CTO, appointed project team etc.
Again we go through several approaches to user stories. We start with the idea of collecting user stories from outside development, but it’s hard to get people to give us user stories for what the existing product does, they only think about the new features the new product might include. In the end we fall back on the pragmatic approach of writing most (though not all) of the user stories ourselves.
For most of the life of the project (the run up to launch dates excepted) it’s hard to get the part of the organisation outside of I.T. to get involved in what we’re doing. There’s little or no attendance at sprint reviews above those people that must be there. It’s rare that anyone wants to find out how the new product works, how it meets the organisations needs, what it looks like, or what it’s like to use it. There’s generally a reluctance from the rest of the organisation to spare time even for formally organised reviews, acceptance tests and so on.
About two-thirds the way through the project we’re aiming to launch limited functionality to one key client. There’s a fixed deadline for this, and as the time approaches it becomes more and more difficult to see how we can cope with the complexity of what we need to do within the fixed two-week sprint cycle. The complexity increases because people are now finally looking at the product and finding fault with it. In essence, we had too much to do, and should have really moved the deadline. Since that was not an option we abandoned the scrum process and on a day-by-day basis updated check-lists with new tasks and tasks done till we launched.
(More about what happened here )
Then we reverted to scrum.
So, getting on for 21 months after the first changes were initiated, it becomes apparent that the board is very keen to launch, very soon. Again, we’re given a fixed time-frame. We (dev team) spend a couple of days fleshing out the things that need to be done, debating approaches to this and that, debating what compromises can be made to meet the fixed launch date. In the end we realise that we’re again in the realms of complex, team-generated tasks. (we know that we need to load test, no user story we’re given has has told us that). It seems artificial to try and shoehorn these task lists into self-generated user stories, so again we abandon the scrum process. Because we essentially made a small waterfall project at this stage, our estimates failed to take everything into account. Also the complexity increased because again people started looking at the product, and requested bug fixes, new features and cosmetic changes. And the deadline slipped.
And it stayed that way for nearly many months after the main product launch. Only now, following pressure from the dev team, are we using agile methods again within the team, and getting the agile message out to the rest of the organisation.
Summing up the issues:
So, apologies for rambling on a bit. If you’ve got this far, I’d be very interested in your comments.
(I wrote this in April ‘07 but for some reason didn’t post it. It’s become topical again, for me, so I’m posting these 10-month old thoughts unedited.)
Now it’s over, the truth can be told … we abandoned agile and went …. err …. fragile.
It was a fixed deadline that did it. With 3 sprints (6 weeks) to go before our app, or at least a few key features of it, was due to see light of day, everything seemed fairly straightforward. A couple of tweaks to existing functionality, a bit of UI facelift, and some load testing / app profiling, it didn’t seem too hard.
What happened though, was that the closer we got to to deadline, the bigger the divergence was between what we planned in sprint planning and what we did during the sprint. The last sprint in particular got really chaotic, that’s where we abandoned planned tasks and user stories, ignored scrumworks, and just did what we knew had to be done to get everything ready.
Now scrum as a methodology has worked pretty well for us so far, so what went wrong ? A couple of things I think.
First off, various team members spent more time than expected on supporting the legacy applications. One in particular had unique skills and specific sprint tasks (server setup) on which other team members depended. Since those tasks weren’t getting done, the other team members had to adapt and achieve their goals (load test on a cluster that resembles production) in other ways. We’ve had this problem all along, but all that’s happened is that tasks of this sort got planned, not executed, and deferred to later sprints. with the the looming fixed deadline, there was no alternative but to adapt and do our load testing / profiling on something other than a production-like cluster.
More generally, I think the inspect and adapt cycle was too long. It’s pretty much a two week cycle with a two week sprint. So at the beginning of the sprint the planning comes down to “where are we now, and what do we need to do next”. Then after the sprint we review and say, “well, did we get to where we wanted ?” and base the next sprint on that. Now, if there’s a fixed launch date, and it’s 2 days after the end of the sprint, then waiting till the end of the sprint to see if we’ve achieved our goals is waiting too long. We recognized that and just did what had to be done.
The other problem we faced was a classic project management one. We started each sprint with a fixed set of resources, a fixed amount of time and a known set of tasks that had to be completed. During the sprint however, what happened was that new tasks arose that were in business terms unavoidable (issues with the legacy app had to be addressed) but nothing else changed, no extra time to complete the sprint tasks (and sprint tasks could not be removed because of the fixed deadline), and no further resources were added.
