Agile Development: Sprint Retrospective

In my previous post in this Agile Development series I detailed the typical workflow for planning a sprint. In this post I will outline the process of the sprint retrospective. One helpful way to think about the sprint retrospective is as a mini postmortem. During the retrospective the team goes over what went well, what didn't go well, and what the team could have done better.

What Went Well

The reason we ask what went well is not so that we can pat each other on the back and inflate our egos. One of the keys to a successful agile project is consistency in the velocity of the team sprint over sprint. One of the best ways to do this is to identify what lead to success so that the team can capitalize on it in the upcoming sprint(s).

What Didn't Go Well

As with most growth being realistic, open and honest about our mistakes and failures leads to not repeating them. The purpose of identifying what didn't go well is NOT to lay blame on any particular person or people. Identifying what didn't go well helps us to avoid or correct them those mistakes.

What Could Be Done Better

At first this may not seem different from identifying what didn't go well. The real difference is that we recognize things that may have gone well but could have gone better. Asking what we could do better is an opportunity to identify gaps in our planning, our process, and our estimates. This question should lead directly to actions that can be taken to increase or maintain the teams current velocity.

Because the sprint review typically takes place before the planning of the next sprint asking these three questions provides the team with the opportunity to make changes that give them a higher likelihood for success before the next sprint starts.

Agile Development: Planning A Sprint

In my previous post in this Agile Development series I walked you through the daily stand-up. In this post I'll go outline the typical workflow for planning a sprint.

Identify The Features

The product owner, who is serving as the voice of the customer, proposes the highest priority features. The scrum team is responsible for getting clarification, pushing back on features, and proposing features or work that should be accomplished. Features should be prioritized based on their overall business value. It's important to note that working on tech debt can often provide down stream business value by enabling new feature work.

Prioritize and Estimate

Once the scrum team has identified the features to work on in the sprint, the team should then pull all of the stories associated with that feature into the sprint backlog. The product owner and scrum team should negotiate the priority of the stories. Once the priority of the stories has been determined the team needs to estimate the level of effort for each individual story.

One typical way of estimating stories during planning is to pick a story that does not have a lot of ambiguity which represents a medium (tee-shirt sized) amount of work. The team then assigns that story a point value (the middle value of their pointing system). The team then moves on to point the rest of the stories based on them being either more effort or less effort than the original story. Stories that are more effort are assigned a higher point value while stories with lower effort are assigned a lower point value.

Determine What Stories Make It Into The Sprint

Just because the product owner or scrum team has identified a feature (and it's related stories and bugs) as high priority doesn't mean that the team can actually complete that feature in a sprint. The team needs a way to estimate the amount of work that can be accomplished by the team during the sprint. This is typically done through velocity tracking.

Velocity in agile is typically determined by the amount of points the team was able to accomplish in the previous sprint. Historical sprints aren't to be taken into account as each sprint the teams goal is to get better and better as estimating the level of effort for story work.

The sprint backlog should only contain enough stories to meet the teams velocity. Choosing to few story points means that the team will likely have to poach more work (which hasn't been vetted) from the product backlog. Choosing too many story points means the team is not likely to finish all the work in the given sprint, setting the team up for failure from the start.

At first the teams velocity will be all over the place as they work out their estimation techniques. But over time the team should start to settle in to a consistent velocity. It's important to be aware that there are several common practices that can negatively affect a teams velocity. It's important for the sprint team to guard against these in order to accurately gauge their velocity sprint over sprint.

Under or over estimating the level of effort of a story

Often this is the result of ambiguity that is not resolved priority to planning. If the team doesn't have enough information to accurately gauge the level of effort for a particular story the product owner and scrum master should first work to resolve the ambiguities before a feature or story is considered for a sprint. This is an example of a valid reason for the sprint team to push back on a particular feature or story.

Unplanned sprint work or re-prioritization of the sprint deliverables mid-sprint

This is an Agile no-no and should be avoided at all costs. In Agile our sprint duration is typically short enough that when new work comes up we can prioritize it at the next sprint planning meeting. If new high priority work or re-prioritization mid-sprint is truely unavoidable the best way to make sure it doesn't impact the overall sprint deliverables is for the whole team to understand what the lowest priority work is and have that work drop out of the sprint.

Unaccounted for absence

People are going to take vacation, get sick, or be off for a holiday. It's difficult to account for sick time but it is very easy to account for holiday's and vacation time. Make sure in your sprint planning you understand how many days during the sprint people know they will be out and reduce that sprints velocity accordingly.

Other regular duties

Many software organizations use the DevOps model where the developer is on-call for the software and services they own. Not accounting for this type of work can have a negative impact on planning as the sprint team will sign-up for more work than they can actually accomplish.

Break The Stories down into tasks

Once the sprint backlog has been determined the team should break the stories down into smaller more granular tasks. This allows for more story work to be done in parallel. The team can swarm on a story together ensuring that the highest priority work is done first.

Once the tasks are identified the team is ready to start the sprint!

Agile Development: Stand-up

In my previous post in this Agile Development series I explained two ways to determine sprint duration. In this post I'll go into detail on what an agile stand-up typically looks like.

One thing you've heard me mention over and over in this series is that one of the primary goals of agile is to decrease the total time it takes to complete the feedback loop. One key way to accomplish this is with a daily stand-up.

In it's most basic form the stand-up is a time-boxed meeting (typically 15 minutes) where the scrum team gets together and answers three simple questions:

1. What did I do yesterday?
2. What am I going to do today?
3. What am I blocked on?

What Did I Do Yesterday?

In it's most basic form this question is intended to determine if progress was made towards the overall sprint goals. Knowing, as a team, what has already been accomplished helps the team formulate a plan for what needs to be accomplished over the next day.

I've seen this question often turn into a check-list of everything you did during the day. But it's important to keep in mind that your goal is not to provide an overview of everything you did. Instead, the purpose of this question is to make sure that others are aware of how much progress was made towards the work you committed to the day before. This is especially important when work items in the sprint are interdependent.

What Am I Going To Do Today?

There are two reasons that telling the team what you plan to accomplish for the day is important. First, it allows the team to validate whether the work is really the top priority for the sprint. During sprint planning many teams will prioritize the backlog for the sprint. But because our feedback loop is so short the priorities of any individual task may change throughout the sprint or even daily.

The second reason this question is important is that you are making a commitment to the team to accomplish a particular task. You can think of it as your daily contract with the team. And in turn the other members of the team are making a commitment to you to get other work done. This is how the team is able to increase and maintain a particular velocity. The team is able to time and coordinate work at a granular level that allows the team to maintain a constant throughput. 

What Am I Blocked On?

Making sure that the team is aware of any potentially blocking issues allows the team to react quicker and not allow blocking issues to affect the teams throughput. Because the scrum team talks about this daily, it is able to address potentially blocking or blocking issues immediately.

If a particular team member is blocked on a technical issue, I've often seen the scrum team swarm on the issue until it gets resolved. This has the benefit of making sure any blocking issue that is an upstream dependency of other sprint tasks gets resolved quickly and the team is able to dedicate more of the sprint to work that was planned.

If an issue cannot be resolved due to some unforeseen circumstance the sprint team has the ability to adjust and pull in additional tasks into the sprint while the blocking issue is being resolved. Because we talk about blocking issues daily, there shouldn't be a significant impact on the sprint schedule.

Parking Lots

It's often the case that during stand-up an issue comes up that requires a deeper dive than the 15 minutes allotted allows for. Fight the urge to go into detail during stand-up on the issue. Instead, announce that you have a parking lot and who should attend and then table the issue until after the stand-up is over. This allows less of the team to be randomized by an issue that isn't directly related to their daily commitment. 

Agile Development: Determining Sprint Duration

In my previous post in this Agile Development series I explained the anatomy of a sprint in terms of the three different sprint phases. In this post I'll detail the two main ways sprint duration is determined.

Fixed Time

Most sprint teams you come across will use a fixed time for their sprints. In my experience 2 to 3 week sprints are the most common duration. The goal of a fixed time sprint is to shorten the feedback loop with the customer in order to deliver the right features to them. By limiting the time to 2 or 3 weeks you can easily change course as the requirements change and provide continued business value to the customer without spending a lot of time on features that won't be used or whose requirements have changed.

Having a fixed time sprint allows the team to set expectations with your customer on what new features they will be building and when they will be available to the customer. I would strongly suggest that you don't have sprints that are longer than 3 weeks in duration. 2 to 3 weeks keeps the feedback loop small while also giving the sprint team enough time to build a feature out end to end. Increasing the amount of time it takes to complete the feedback loop means that the sprint team could spend more time building the wrong product and less time building the correct product should the requirements change.

It is occasionally the case with a fixed time sprint where the sprint will end and the team will have been unable to complete a particular feature. In this case the retrospective should identify the cause of the decreased velocity. Typical causes for a decrease in velocity (in my experience) have been:

• Failure to account for vacation or known leave of sprint team members.
• Failure to account for time spent on external work (like on-call duties).
• Failure to account for the unexpected work that grows the sprint backlog.
• Undefined or not well defined definitions of done.
• Stories that are too large and hide dependencies.
• Unknown external dependencies. 

Fixed Feature

Another, less common, way to determine sprint duration is to use a fixed feature approach. This approach says that a sprint is complete only when a particular feature is complete regardless of time. This is typically used when it is known that dependencies are hidden but that it is not easy to uncover those dependencies without first doing some work towards the end goal of the feature. In a system without a well defined or understood architecture, a fixed feature sprint duration is often used as it allows for the unknowns that go with the not well defined architecture.

Earlier in my career this was a more appealing way to determine sprint duration but as I've grown in my Agile planning I've come to realize that open-ended sprint duration's actually detract from shipping software more than they contribute to shipping complete software. Even with the most ambiguous problems it is usually better to first root out the ambiguity in a fixed time sprint and then plan and work on the feature.

Another reason that I've found for fixed feature sprints being less than ideal is that you have a harder time gauging your velocity as your velocity is determined by the amount of work you can complete in a fixed amount of time. Since knowing you're velocity is key for estimating when a feature is complete without it you cannot set your customers expectations as to when they will get a particular feature.

Fixed feature sprints also increase the time it takes to complete the feedback loop. And, as I mentioned earlier, increased feedback loops mean that it takes longer to respond to customer requests.

Agile Development: The Anatomy Of A Sprint

In my previous post in the Agile Development series I walked through the makeup of a sprint team. I explained the roles of Product, Dev, QA, Operations, and the Scrum Master. In this post I will go into detail on the anatomy of a sprint.

Planning

The first phase of the sprint life-cycle is planning. During the planning phase sprint velocity is determined, features are identified, features are estimated, and the backlog is created.

At it's most basic level the velocity of the sprint is determined by the number of points that were completed during the last sprint. When a sprint team first forms their velocity is very volatile. But as the team learns to estimate their story points more accurately there is less oscillation between the number of points completed sprint over sprint.

Determining which features are candidates for inclusion usually starts with product as the voice of the customer. Product will advocate for what the customer needs. The team will also identify what bugs and tech debt need to be addressed during the sprint.

Once the candidate features are identified the team will estimate the the level of effort for each story. At the formation of the sprint team a point system is determined and that point system is used for estimating work. The two most common point systems I've come across are using Fibonacci numbers (with 8 or 13 being the top end of a sprint) and using 1, 5, 10, 25, 50, and 100 as story points. Whatever your system of pointing there needs to be a way to determine small, medium and large stories.

There are many different estimating techniques but most teams I've seen use some sort of t-shirt size technique. They start by picking a story and assigning it an arbitrary point value. The rest of the stories are then pointed by determining if the story is larger or smaller than the initial story. When you have more than 3 assignments that you can use for point values you can get more granular and determine if the story is larger or smaller than the other stories in it's t-shirt size bucket.

Once the velocity is determined, potential features are identified, and stories are estimated you will be able to create your sprint backlog. Backlog creation begins by stack ranking the stories (taking dependencies into account) and cutting the sprint off after the last story that keeps the total number of sprint points at or less than the pre-determined sprint velocity.

Implementation

The second phase of the sprint life-cycle is implementation. During the implementation phase the team meets daily at stand-up and goes over what they worked on yesterday, what they're working on today, and any potential blocking issues. When a blocking issue is identified it's up to the scrum master to try to unblock the team member. During the implementation phase the software should be developed, tested, and integrated.

Review

The last phase of the sprint life-cycle is review. The review phase takes on two forms. The first is the sprint demo and the second is the sprint retrospective.

The sprint demo is the teams opportunity to showcase what they've built during the sprint. Sprint demos can actually be a very useful tool leading into the next sprint planning phase. Often when we think about planning with regards to software we think about what we're going to build from an architectural perspective. The problem with this line of thinking is that it typically leaves out integration, testing, and deployment. If we instead think about what we're going to demo during planning we'll have to take into account how we're going to demo it.

A note of caution about sprint demos. One mistake I've often seen teams make during sprint demos is to walk everyone through a bullet list of work that was performed. Walking others through a bullet list isn't a demo. If you find that your team is working on features that don't have a user facing component then figure out a creative way to tell the story of how what you worked on matters. Use an animation, a use case scenario, or maybe even a chart or some graphs to help illustrate what you worked on. 

The sprint retrospective is a mini postmortem. During the retrospective the team goes over what went well, what didn't go well, and what the team could have done better. The retrospective is a dedicated time for us to identify the issues that caused us to lose velocity, to call out our successes during the sprint, as well as identify what we could have done differently to have improved the sprint. Because the sprint review typically takes place before the planning of the next sprint we have an opportunity to make changes that give us a higher likelihood for success before the next sprint starts.

The retrospective IS NOT a finger pointing exercise. It's NOT a witch hunt or an opportunity to push blame around. The retrospective IS an opportunity to identify gaps in our planning, our process, or our estimates. The point of the retrospective is to maintain or increase our velocity.

Isn't there a phase missing?

You may be wondering where the "release" phase is. There isn't one. Releasing your software should be done as part of the implementation phase. Your sprint planning should account for what it takes to ship the software, if that's part of the definition of done for the sprint.

Agile Development: Sprint Teams

Sprints are the way that Scrum teams iterate over software. The goal of a sprint is to produce a working set of software that is potentially shippable. I say potentially because it's really up to the business to decide when the software is shippable. As long as the feature development is complete there is a choice. You can evaluate the number and severity of open bugs as well as the other features and make a call on whether you ship or not.

Sprint Teams

Sprint teams are comprised (at minimum) of a representative of product and the software development engineers. Though in practice you'll really want someone from QA (quality assurance, if your org has a separate QA group) and someone from operations represented.

Products Role

In Agile we favor collaboration over contract negotiation. The role of product management as part of the scrum team is to represent the voice of the customer. The product manager should be working directly with the customer to understand their needs. The product manager helps collaborate with the rest of the team on the requirements of the feature(s) being developed each sprint.

Developments Role

The developers are the ones actually building out the features. The role of the developer is to collaborate with product to turn the requirements into feature stories and tasks. The developer is also the advocate for the health of the code and architecture. The developer is responsible for making sure that unnecessary tech debt isn't accrued sprint over sprint. Often this means collaborating with product on the scope of work for each sprint to account for taking on bug fixes and technical debt.

Quality Assurances Role

Ideally all software we write includes unit, functional and integration tests. But that doesn't mean that all testing can be automated. There are some aspects of the user experience that may require manual testing. Or if your organization is new to agile you may have creating automated tests as tech debt. Whatever the reason, our goal of building potentially shippable software each sprint means including QA as part of the sprint team.

Operations Role

One other often overlooked member of the sprint team is operations. While continuous delivery and continuous integration are the ideals for deployment, many organizations have not shifted to that model yet. If your org is one that hasn't, including operations will help accurately account for the sprint work necessary to release the software.

Scrum Master

Each sprint has a scrum master. It is the role of the scrum master to facilitate scrum. This is done by managing the product backlog, facilitating the scrum meetings, facilitating the sprint planning meeting, facilitating the retrospective meeting, helping to define what done means for a given story or task, and removing roadblocks to the team. The scrum master is also responsible for removing roadblocks to the scrum team. This is usually done by fostering collaboration between teams and product.

Agile Development: The Need For Scrum

Over the last two decades the internet has become ubiquitous in our lives. The speed of our internet connections and performance of our computers have increased several orders of magnitude. These changes have had a cascading effect on the software industry. Software used to be run on hardware to expensive for the average person (or even company) to purchase and maintain. Bandwidth limitations required software to be distributed via a high cost and immutable media like the compact disk. As hardware prices decreased and the bandwidth and speed of our connections increased it became easier to distribute software over the web or even run it entirely in the browser using a combination of HTML, JavaScript and CSS. 

As software became easier to distribute our ability to innovate and provide our customers with fixes and new features became paramount to the success of a software product. The birth of the mobile computing industry with devices powerful enough to run both native and web based software intensified our need to respond to the ever changing needs of our customers.

One of the side affects of software distribution evolving has been that our processes, originally created to manage change over a long period of time, became inflexible and costly in a world where change was frequent. The software industry was plagued with process that couldn't keep up with the rapid multi-platform change the industry was going through.

This inflexibility in the software development process gave birth to the Agile movement. Agile recognized that responding to the changing needs of our customers has become as important or more important than following a long term plan. Collaboration on product requirements needed to take precedence over processes or tools. Agile understood that rather than formalizing requirements and managing those requirements as they flowed downstream that it needed to embrace iterative development and distribution to keep up with the ever changing needs of it's customers.

Agile understood that the cost of downstream change in a rigid process is greater than identifying the change upstream and pivoting earlier. In order to manage this change agile embraced the concept of Scrum. In traditional software development information flowed downstream from product to development, development to test, test to operations, and operations to the customer. If a change was needed it required a renegotiation of requirements and schedule due to the fact that any change restarted the process at the beginning.

Scrum increases the speed and flexibility of software development by bringing product, development, test and operations together into a cross-functional team responsible for adapting to change. Instead of requirements flowing downstream in a rigid process, requirements are collaborated on with product and test. Requirements are turned into features which are developed iteratively. At the end of each iteration the next set of requirements is defined. These requirements can build upon a previous iteration or completely change the direction of the software. Each iteration is time-boxed in order to allow for greater speed and flexibility in change.

Over the next few weeks I'll go in-depth into the different pieces of Agile Scrum with the goal of providing you a framework by which you can apply agile as part of your development process.