Agile Development: The Anatomy Of A Sprint

In my previous post in the Agile Development series I walked through the makeup of a sprint team. I explained the roles of Product, Dev, QA, Operations, and the Scrum Master. In this post I will go into detail on the anatomy of a sprint.

Planning

The first phase of the sprint life-cycle is planning. During the planning phase sprint velocity is determined, features are identified, features are estimated, and the backlog is created.

At it's most basic level the velocity of the sprint is determined by the number of points that were completed during the last sprint. When a sprint team first forms their velocity is very volatile. But as the team learns to estimate their story points more accurately there is less oscillation between the number of points completed sprint over sprint.

Determining which features are candidates for inclusion usually starts with product as the voice of the customer. Product will advocate for what the customer needs. The team will also identify what bugs and tech debt need to be addressed during the sprint.

Once the candidate features are identified the team will estimate the the level of effort for each story. At the formation of the sprint team a point system is determined and that point system is used for estimating work. The two most common point systems I've come across are using Fibonacci numbers (with 8 or 13 being the top end of a sprint) and using 1, 5, 10, 25, 50, and 100 as story points. Whatever your system of pointing there needs to be a way to determine small, medium and large stories.

There are many different estimating techniques but most teams I've seen use some sort of t-shirt size technique. They start by picking a story and assigning it an arbitrary point value. The rest of the stories are then pointed by determining if the story is larger or smaller than the initial story. When you have more than 3 assignments that you can use for point values you can get more granular and determine if the story is larger or smaller than the other stories in it's t-shirt size bucket.

Once the velocity is determined, potential features are identified, and stories are estimated you will be able to create your sprint backlog. Backlog creation begins by stack ranking the stories (taking dependencies into account) and cutting the sprint off after the last story that keeps the total number of sprint points at or less than the pre-determined sprint velocity.

Implementation

The second phase of the sprint life-cycle is implementation. During the implementation phase the team meets daily at stand-up and goes over what they worked on yesterday, what they're working on today, and any potential blocking issues. When a blocking issue is identified it's up to the scrum master to try to unblock the team member. During the implementation phase the software should be developed, tested, and integrated.

Review

The last phase of the sprint life-cycle is review. The review phase takes on two forms. The first is the sprint demo and the second is the sprint retrospective.

The sprint demo is the teams opportunity to showcase what they've built during the sprint. Sprint demos can actually be a very useful tool leading into the next sprint planning phase. Often when we think about planning with regards to software we think about what we're going to build from an architectural perspective. The problem with this line of thinking is that it typically leaves out integration, testing, and deployment. If we instead think about what we're going to demo during planning we'll have to take into account how we're going to demo it.

A note of caution about sprint demos. One mistake I've often seen teams make during sprint demos is to walk everyone through a bullet list of work that was performed. Walking others through a bullet list isn't a demo. If you find that your team is working on features that don't have a user facing component then figure out a creative way to tell the story of how what you worked on matters. Use an animation, a use case scenario, or maybe even a chart or some graphs to help illustrate what you worked on. 

The sprint retrospective is a mini postmortem. During the retrospective the team goes over what went well, what didn't go well, and what the team could have done better. The retrospective is a dedicated time for us to identify the issues that caused us to lose velocity, to call out our successes during the sprint, as well as identify what we could have done differently to have improved the sprint. Because the sprint review typically takes place before the planning of the next sprint we have an opportunity to make changes that give us a higher likelihood for success before the next sprint starts.

The retrospective IS NOT a finger pointing exercise. It's NOT a witch hunt or an opportunity to push blame around. The retrospective IS an opportunity to identify gaps in our planning, our process, or our estimates. The point of the retrospective is to maintain or increase our velocity.

Isn't there a phase missing?

You may be wondering where the "release" phase is. There isn't one. Releasing your software should be done as part of the implementation phase. Your sprint planning should account for what it takes to ship the software, if that's part of the definition of done for the sprint.

Agile Development: Sprint Teams

Sprints are the way that Scrum teams iterate over software. The goal of a sprint is to produce a working set of software that is potentially shippable. I say potentially because it's really up to the business to decide when the software is shippable. As long as the feature development is complete there is a choice. You can evaluate the number and severity of open bugs as well as the other features and make a call on whether you ship or not.

Sprint Teams

Sprint teams are comprised (at minimum) of a representative of product and the software development engineers. Though in practice you'll really want someone from QA (quality assurance, if your org has a separate QA group) and someone from operations represented.

Products Role

In Agile we favor collaboration over contract negotiation. The role of product management as part of the scrum team is to represent the voice of the customer. The product manager should be working directly with the customer to understand their needs. The product manager helps collaborate with the rest of the team on the requirements of the feature(s) being developed each sprint.

Developments Role

The developers are the ones actually building out the features. The role of the developer is to collaborate with product to turn the requirements into feature stories and tasks. The developer is also the advocate for the health of the code and architecture. The developer is responsible for making sure that unnecessary tech debt isn't accrued sprint over sprint. Often this means collaborating with product on the scope of work for each sprint to account for taking on bug fixes and technical debt.

Quality Assurances Role

Ideally all software we write includes unit, functional and integration tests. But that doesn't mean that all testing can be automated. There are some aspects of the user experience that may require manual testing. Or if your organization is new to agile you may have creating automated tests as tech debt. Whatever the reason, our goal of building potentially shippable software each sprint means including QA as part of the sprint team.

Operations Role

One other often overlooked member of the sprint team is operations. While continuous delivery and continuous integration are the ideals for deployment, many organizations have not shifted to that model yet. If your org is one that hasn't, including operations will help accurately account for the sprint work necessary to release the software.

Scrum Master

Each sprint has a scrum master. It is the role of the scrum master to facilitate scrum. This is done by managing the product backlog, facilitating the scrum meetings, facilitating the sprint planning meeting, facilitating the retrospective meeting, helping to define what done means for a given story or task, and removing roadblocks to the team. The scrum master is also responsible for removing roadblocks to the scrum team. This is usually done by fostering collaboration between teams and product.

Agile Development: The Need For Scrum

Over the last two decades the internet has become ubiquitous in our lives. The speed of our internet connections and performance of our computers have increased several orders of magnitude. These changes have had a cascading effect on the software industry. Software used to be run on hardware to expensive for the average person (or even company) to purchase and maintain. Bandwidth limitations required software to be distributed via a high cost and immutable media like the compact disk. As hardware prices decreased and the bandwidth and speed of our connections increased it became easier to distribute software over the web or even run it entirely in the browser using a combination of HTML, JavaScript and CSS. 

As software became easier to distribute our ability to innovate and provide our customers with fixes and new features became paramount to the success of a software product. The birth of the mobile computing industry with devices powerful enough to run both native and web based software intensified our need to respond to the ever changing needs of our customers.

One of the side affects of software distribution evolving has been that our processes, originally created to manage change over a long period of time, became inflexible and costly in a world where change was frequent. The software industry was plagued with process that couldn't keep up with the rapid multi-platform change the industry was going through.

This inflexibility in the software development process gave birth to the Agile movement. Agile recognized that responding to the changing needs of our customers has become as important or more important than following a long term plan. Collaboration on product requirements needed to take precedence over processes or tools. Agile understood that rather than formalizing requirements and managing those requirements as they flowed downstream that it needed to embrace iterative development and distribution to keep up with the ever changing needs of it's customers.

Agile understood that the cost of downstream change in a rigid process is greater than identifying the change upstream and pivoting earlier. In order to manage this change agile embraced the concept of Scrum. In traditional software development information flowed downstream from product to development, development to test, test to operations, and operations to the customer. If a change was needed it required a renegotiation of requirements and schedule due to the fact that any change restarted the process at the beginning.

Scrum increases the speed and flexibility of software development by bringing product, development, test and operations together into a cross-functional team responsible for adapting to change. Instead of requirements flowing downstream in a rigid process, requirements are collaborated on with product and test. Requirements are turned into features which are developed iteratively. At the end of each iteration the next set of requirements is defined. These requirements can build upon a previous iteration or completely change the direction of the software. Each iteration is time-boxed in order to allow for greater speed and flexibility in change.

Over the next few weeks I'll go in-depth into the different pieces of Agile Scrum with the goal of providing you a framework by which you can apply agile as part of your development process.

Minimizing the risk of bugs

Software development is a craft that requires practice, hard work and dedication. It's a craft that involves many edge cases and unintended consequences. As awful as it sounds, we've all got bugs in our code. The goal of writing software should not be to write software with no bugs as this is unattainable. Instead the goal should be to minimize the risk of high severity bugs.

Every change to your software is an opportunity to introduce new bugs. Following these tips will help you minimize the risk of introducing bugs into your system.

Test Your Software

Duh, right? Testing your software may sound like a no-brainer but you'd be surprised by how many times people break builds and introduce buggy software just because they didn't test their code. In my previous post on Testing Your Software Properly I provided a checklist of tests that you should run before you commit your code.

Without proper tests you can not have confidence that you aren't regressing an old bug or introducing a new bug into the system. Tests are the foundation for confidence in any change you make.

Reduce the surface area of your change

The more lines of code you change with every commit the higher the risk of bugs. This is where encapsulation, SOLID principles, and refactoring become so important.

It's important to encapsulate your software into individual loosely coupled pieces. If you make a change in a class and it causes cascading changes throughout the rest of your software in code un-related to your change then you're likely introducing new bugs.

If you follow the Single Responsibility Principle in SOLID your classes will have one reason and only one reason to change. This reduces the surface area of your change because you will not be changing code in unrelated modules.

Following principles like DRY and YAGNI will lead to more robust code that is flexible and easy to change.

Keeping your code simple is one of the keys to reducing the surface area of your change.

Reduce the complexity of the code

Overly complex code leads to bugs for many reasons:

• The code is fragile because the learning curve is steep.
• It's easier to do the wrong thing than it is the correct thing when making a change in the code.
• The code is not readable often causing you to make multiple context switches to understand a single workflow.

What are some signs that your code is too complex?

• The patterns in the code are not clear, obvious, and/or discoverable.
• You have multiple levels in indirection that support one workflow or use case.
• You have an abstraction that fronts a single concrete implementation.
• Your code does not have clear boundaries.
• You have highly interdependent modules.
• Your code is not highly cohesive.
• Your code has rigid rules that are not enforceable in their individual units but only as a whole.

What the new Microsoft needs to do to become relevant again

If you just look at the numbers for Microsoft under Steve Ballmer you'd have to draw the conclusion that Microsoft was successful. Their annual revenue more than doubled and the net income increased 215 percent. But my opinion is that Microsoft under Steve Ballmer was a disaster.

How can the company be a disaster when it was making money hand over fist? Microsoft's focus on it's closed eco-system (Windows, Office, etc) made it become irrelevant in three key areas that are the future of computing:

1. The Cloud
2. The Start-up
3. Mobile

It's hard to argue that those three areas aren't the most important areas in the technology industry. Not just in the short term either. The start-up culture, the cloud and mobile computing have changed the direction of technology for good.

We now live in a world where everyone (in developed countries) has a computer (or multiple) in their pocket at all times offering them instant and easy access to the internet. Gone are the days when people's primary access to the internet was the desktop. This "always on" access has changed how people consume both technology and hardware. That's why Microsoft's focus on the desktop has hurt them and their eco-system.

A lot has been said about a new culture forming at Microsoft under Satya Nadella. Microsoft has stated that it wants to be a mobile first company. It has started to make good on that by making applications like Office available on iOS and Android. But being a mobile first company isn't enough. In order for Microsoft to become relevant again they need to have a strategy for those three areas.

The Cloud

Microsoft's approach to the cloud has been narrowly focused on the wrong thing. They're trying to compete with cloud providers like AWS with Azure when, in my opinion, they should be trying to make Windows run better on other cloud providers.

Microsoft's biggest threat in the cloud isn't other cloud service providers, it's Linux. In the old days of the data center Microsoft could thrive. Their tools helped with scalability and reliability in a world where server up-time was a premium. In today's cloud we've learned to plan for failure and have built our services around the microservices philosophy, a philosophy similar to that of Unix; build small but very powerful pieces of software that can be strung together.

In order for Microsoft to compete in the cloud they need to focus not on providing cloud services but instead on making Windows more cloud ready. In order to do that they need to:

1. Make Windows run completely headless. I should never have to plug in a monitor or use a mouse to administer my server. Many people will try to tell you that this is achievable today but if you try it you'll see it's not true. The command-line interfaces for services on Windows are horrible.
2. Build SSH into Windows. SSH is the gold standard of remote access. Microsoft needs to buy in.
3. Ditch PowerShell and provide a POSIX compliant shell. This does two things. It's makes using Infrastructure as Code tools like Chef, Puppet, and etc much easier. Second, it allows for a lot of existing scripts and server knowledge to be ported to Windows. 

The Start-up

This is arguably going to be the most difficult area for Microsoft to change its focus on. This is also very much tied into the cloud strategy I mention above. Start-ups aren't using Microsoft products because:

1. It's not free.
2. Microsoft products are not standards compliant.
3. There's not as big an open source community built around core technologies that Microsoft is good at.

Free

In this day and age Microsoft needs to stop trying to make it's revenue off of Windows licenses. Linux is free and is available in many many many different flavors. One big reason start-ups are turning to Linux is simply the cost to get started.

So where can Microsoft make up some of this revenue loss? Support contracts. People buy support contracts when they're big enough to need the support and when they want to feel assured that an expert is available and willing to help. 

Standards

Start-ups need to move quick. They're competing against each other but also much more well established companies with a lot of money to burn. Standards help these new companies move quicker without getting locked into a proprietary system.

Start-ups can build email, spam filtering, calendar, contacts, file share, source control, and web servers using standards compliant software on Linux that is accessible on all modern OSs (Android, iOS, Linux, OS X, and Windows) and not have to worry about things like the OS X version of Outlook messing up calendar invites because it isn't as feature rich as the Windows version.

Standards compliant services allow people to consume those services using the client of their choice. The client they have become an expert in.

Open Source

Microsoft is moving in the right direction here. I applaud them in open sourcing .Net. But that's not good enough. Make Windows work better with the thousands and thousands of open source software projects that people use in the wild today. Explore GitHub and make the popular projects "just work" on Windows.

Mobile

Mobile is about choice, about personality, about uniqueness, about familiarity and about being distraction free as much as it's about being connected. I'm very encouraged seeing what Microsoft has done to pry Office out of their closed eco-system and make it available cross platform. But mobile is about more than just apps.

In order for Microsoft to succeed in Mobile they need to embrace HTML5, reactive design, and other browsers (specifically webkit). For example, there's no excuse for Outlook Web Access to work differently on a Mac or Linux than it does on Windows.

Mobile for Microsoft should be the bow on the package. Provide a set of micro-services and productivity software that people need and can consume from anywhere. People will pay for the software if it works and it's useful.