You Shouldn't Backfill Unit Tests

Recently I've found myself in several conversations about unit tests, and the lack of them on a given project. It seems that almost everyone on the team knows that writing unit tests is a good thing and it seems that most of them also wished they'd written them from the start. But, as is the case with most fast moving projects, the team finds themselves in the position where they don't have adequate test coverage. This leads to reduced confidence in the code and not being able to catch bugs far enough upstream to effectively prevent them from ever being released to customers.

What this usually leads to is people on the team making comments, in response to a bug or regression someone is working, that if they only had unit tests they would have found this bug earlier. What I've seen most often in that situation is someone finally getting fed up enough with bugs and/or regressions and picking up the unit test torch and leading a crusade to backfill all the missing tests. While I appreciate the spirit of this, it's not the right thing to do.

Why's it not the right thing to do? First, you're not actually solving your problem. Your actual problem is that you can't ensure that the code you're currently changing 1. does what it's meant to do and 2. doesn't do something it's not meant to do.  Second, depending on how old the given code is, you may not know how it was actually supposed to work vs how it actually does work. You may end up writing unit tests are aren't correct, and in the worst case scenario cause you to change working code. Lastly, when the team is done backfilling the tests, they haven't learned new behavior (i.e. to write the tests when you write the code) but, more likely, have grown a disdain towards writing unit tests.

Now before you start to think I'm advocating against writing unit tests I'll flat out tell you I'm not. In fact I've come to realize in my career that writing your tests first is the only way to assure that you have working code. So what am I advocating for then?

Make a decision as a team to write unit tests and then write unit tests for all the code the team writes from that point forward. You'd be surprised at how much better your code coverage will be after just a short amount of time. Why? Because you'll be writing unit tests for code that you're fixing bugs in. You'll be writing them for new features or changes to existing features. Essentially, you'll be writing tests in the hot spots of your code that are requiring change. The very code that is either causing bugs, or is likely to have bugs introduced in.

If you never change a particular set of code again then you don't really need unit tests for that code (remember I'm talking about unit tests and not integration, system, or other types of tests).  If you do change that code, then because you've decided as a team to write tests, you'll add the missing tests when you go in to make the code change.

So how do you get started? It's easy, just follow my 3 step plan.

1. Make a decision to not put up with having incomplete code (i.e. code without corresponding tests).
2. When you write your next set of code, write tests!!!! (preferably first)
3. Enjoy the fact that you now have less buggy code that you can have more confidence in.

Being a new parent; a software alagory

This weekend my son turned 1. This got me thinking about how so much of what I've had to learn as a new parent is similar to working in the software industry.

You don't understand the scope till your responsible for it

Prior to my son being born, we had no hands on experience with what it's like to have a child. Sure we babysat, a lot actually, and were comfortable around kids. We'd changed a million diapers. But we'd never sat up all night with a child with a fever. We'd never had to monitor a child's eating or sleeping routines. Babysitting, was just not the same as actually having a child. I've found that with parenting, most of the challenging stuff happens in the not-so-obvious areas of life. I think it's similar with engineering.

The challenge of working on a software product isn't on the surface. It's not about just understanding the language or platform. It's not about intellectually understanding the features and road-map. It's about understanding the nuance and not-so-obvious areas of the product. It's about understanding where your customers will be spending their time and what their pain points will be. You can understand software at an intellectual level but it's not until you try to change something, build a new feature, and/or take on operational ownership that you really start to understand the new product.

Everything has limitations

Every person has limitations. Newborns for instance can't talk. They can't crawl, can't feed themselves, can't really do much for themselves other than cry, sleep, and poop. As a new parent you learn the limitations of your child and (in the short term) help them compensate for their limitations and (in the long term) help them learn the skills necessary to overcome them.

A software product is much the same way. In order to help the system really thrive and succeed you need to understand it's limitations. Sometimes you have to put in short term fixes or mechanisms that allow them to work around these limitations. But in the long run, you need to be looking at how you can make the system better. How you can help the system overcome it's limitations. Sometimes this means refactoring. Sometimes it means prioritizing tech debt over new features. Sometimes it means recognizing a systems in ability to succeed and failing fast.

It doesn't grow unless you feed it

You're responsible for the care and feeding of the system. To do that you need to learning about the system as a whole unit, understand what change is necessary, and help to shepherd those changes through the system. Your software has upstream and downstream dependencies. Your job is to identify those dependencies and work to harmonize how your system runs in the context of the existing ecosystem.

Finding buggy code using git history and a binary search

Some bugs in software are easy to understand. Things like a null pointer exception are obvious and don't need a lot of context. But some bugs are more difficult. Some bugs just aren't obvious until you have the context of the change that was made that introduced the bug.

What I like to do when I'm hunting a bug down is to apply a binary search algorithm to my git history. This has two benefits:

1. You're able to find the exact commit a bug is introduced in
2. You've narrowed down the surface area of the code the bug could possibly be in

#1 above is important as it gives you the ability to reverse apply a patch and see if the bug goes away. If it works without breaking anything that's come to rely on that code it means that you've got a "quick fix" that can be issued to relieve the problem while you find a longer term solution.

Unless your team is a team of developers that goes dark #2 above is means you have found the handful of lines of code that contain the problem and the context of the code dependency. Simply seeing the inter-class dependencies may be enough to make an ambiguous bug make more sense. It's also a good place to look for violations of SOLID which make code less 'ible (maintainable, scalable, flexible, etc...).

So how do you accomplish this?

1. Find the starting point

Start with the git commit from your last release. Create a build from that and see if the bug reproduces. If not you've found your starting point. If it does reproduce and has been in the system a while, keeping going back release by release trying to find the commit of a build that does not contain the bug.

Hopefully you're tagging your releases or creating release branches.'

2. Use now as your ending point

3. Pick the mid point in time between your starting point and ending point

Using git log you:

1. Get the hash of the commit closest to the mid point in time.
2. Create a build using that commit
3. Check to see if the bug reproduces

If the bug does reproduce you've got a new starting point: Repeat step 3 with the new starting point.

If the bug does not reproduce: Repeat step 3 with the new ending point.

Using this binary search you'll find the source commit of your bug fairly quickly (limited only by your build and validation time) and you'll have the context of the commit to fully understand the change that introduced the bug.

What's the measure of success for your project?

When I first started in the software industry I believed that the measure of success for any project was whether or not it shipped anywhere. One of the biggest surprises to me transitioning into the industry was how much software never actually ships. It baffled my mind (and still does) that people could work for weeks, months, or even years on software that would never ship.

What I've learned though in my career is that shipped software is not the right measure of success. USED software is the correct measure of success. No, not re-sold software. I mean software that is being used in the real world by real people/processes to solve real world problems.

Simply shipping software is not enough. Why? Because shipping software that is not used is the same as not shipping the software in the first place. Anecdotally, the top three reasons for software not shipping in my experience have been:

1. The team never reaching feature complete.

2. The project running out of money.

3. The requirements changing so fundamentally that it made sense to just build something else.

#1 and #2 are closely correlated in that #1 eventually leads to #2. All 3 of those situations are symptoms of having a broken process. So how do you fix your process?

Get a product owner

All good (i.e. used) software has a product owner. Someone who is responsible for making sure that the software being built is solving an actual problem and has people who need the problem solved. A good product owner doesn't search for a customer for the software. Instead a good product owner builds software that has people who already need it and figures out how to get it in front of them.

Focus on building individual features rather than a breadth of features 

All too often in the software industry features get built because of some gut feeling of some higher up in the organization. Don't build software this way, it's the easiest way not to have real users. Instead focus on a small feature set (one feature if at all possible) and build it, ship it, and iterate on it.  

Once you've got people using a feature you'll know the correct next feature to build because your customers will ask you for it. The key is, your existing customers. The ones that rely on your software to accomplish some task, will ask you to build something that they need/want. 

Release early, and often

I've talked about this many times but it bears repeating. The only way to build the correct thing is to get feedback on what's being built as soon as possible. The only way to do that is get something in your customers hands as soon as possible, gather feedback, and pivot in response to customers needs. The only real way to do that is to release early and often.

A decade and a half with open source

Over the last 15 years I've been trying to move away from relying on 3rd parties for critical services I use. It started out as a way to learn what standards existed and how they worked in practice. It's one thing to intellectually know what's there, but a whole other thing to use what's there in your everyday life.

In 1999 I started with my own web server, rather than relying on a 3rd party host (like Geocities). I bought a decent desktop computer and installed Slackware and Apache httpd. This has been crucial in my career growth as it allowed me to learn new web technologies as they've come out.

In the early 2000's I decided to run my own mail server. I tried out several and finally landed on qmail. It worked well on Slackware and it supports SMTP Auth. I added IMAP support using Courier IMAP. But I've since switched to Dovecot. qmail also supports integration with SpamAssassin for SPAM detection and filtering. 

In the mid/late 2000's I heard about a new web based mail client called RoundCube. It supported drag and drop like a native email client and I was hooked because it allowed me to remotely access my email with a feature rich client similar to what I had been using via native clients. In the last several years it's added support for plugins which have brought CardDav and CalDav integration.

In 2007 I built my own RSS aggregator (which I have since open sourced) as a way to learn Ruby on Rails. This has been the single greatest learning tool for me. Every time I want to learn a new language or platform I write a client for my aggregator. Google's decommissioning of it's rss client had zero affect on me :)

About four years ago I decided to move from running my services on a standalone desktop box to running them on a server in my own cloud. The big reason is that I needed a server that I could tinker on that won't affect my email, calendar, contact, and file access. So I built my own server and installed XenServer. This has allowed me spin-up and play while also keeping my critical services running rock solid.

About 3 or 4 years ago I ran across Owncloud. I was looking for an alternative to Dropbox because I wasn't happy with Dropbox's model of having to use their software to access my files. Owncloud was revolutionary in allowing me to stop relying on 3rd party services for access to my critical data. After installing Owncloud I controlled my calendar, contacts, and files.

Sometime in the last 3 years I stumbled across GitLab, an open source alternative to Github.  I write a lot of software at home but much of it is very specific to things I want to do and aren't really interesting from an open-source perspective. GitLab allows me to work on private projects and access my source code using a remote git server. When something becomes useful to more than just myself I open source it. 

Senior Software Engineer

One thing I've noticed throughout my career that's been pretty consistent is the desire of software engineers to be given the title Senior Software Engineer. I've seen many people claim the title without truly understanding what it means or what the role entails.

The Misnomer

Often I've seen people given the title or claim the title based on the number of years of experience they have or based on the fact that they have more technical chops than their peers. Being a senior software engineer doesn't have anything to do with years of experience, but instead is about maturity as an engineer. While the two may be correlated (highly so in many cases) there is not a causation relationship. I've seen just as many tenured developers who were no more senior than their counterparts fresh out of college. It's also not only about your technical ability (while that is a large part).

If years of experience don't define a senior engineer and being the most technical person doesn't then what does? In my opinion a senior engineer is someone who manifests most, if not all, of the following characteristics.

Language/Platform Agnostic

A senior software engineer may have language or platform preferences but they're able to deliver on any platform. This is because they understand system metaphors, abstractions, and compositions well enough to know how to learn what they don't know quickly. They identify the correct metaphors, abstractions, and compositions needed to be successful and realize that the syntax and libraries are just implementation details.  

Able to identify and articulate trade-offs

Building software is always about trade-offs. They are able to take schedule and scope into account when building a solution. They understand and are able to clearly articulate risks. For example, a senior engineer knows that with a fixed schedule scope must be variable. They're able to identify and clearly articulate what the proper scope is and what features are at risk.

They're about shipping software

First and foremost, a senior software engineer is able to deliver software. They deliver software in large volumes. This is largely to do with their ability to understand and quickly get through boilerplate code in order to focus on the real problems. They understand that unshipped software isn't making the business better.

Able to get people to want to follow them

Solving complex and ambiguous problems is a requirement of being a senior software engineer. But being able to solve those problems isn't sufficient for the title. A senior software engineer is able to successfully lead a team. They're able to motivate people to solve a problem, hold people accountable to solving the right problem, and get down to the level of other less experienced developers in order to stretch them and help them achieve goals that are beyond their current abilities. 

Able to simplify a complex problem

Senior software engineers are able to see through complex problems and break them down into smaller, easier to solve, problems.

Understand the correct amount of engineering required to solve the problem

Senior engineers don't over engineer solutions and likewise they don't under engineer solutions. They understand that software is meant to solve a particular set of problems while needing to stay maintainable in order to increase the lifespan and usefulness of the product. They recognize that the elegant solution is not always the correct solution. They also have the ability to take a complex problem and identify the potentially fragile portions of a solution and design around them without over generalizing a solution or making the solution less useful in solving the problem at hand.

They recognize that everything doesn't have to be "built here"

This one is pretty straight forward but difficult in practice. Senior engineers recognize when a particular problem is not part of the groups core competencies and look for external solutions. While a particular problem may be interesting, unique, or fun a senior software engineer is able to recognize when solving that problem in house doesn't add any business value or add burdensome tech debt. 

Redefining federated identity and authentication

Recently I've been thinking a lot about Federated Identify. In the technology industry it feels like we're always on the constant hunt for the perfect solution when it's been sitting under our noses the whole time.

The Problem Space

At it's heart there are two things that are being solved for. First, how do I authenticate that you are who you are? Second, how do I share basic information in a secure way across multiple systems.

The first problem is a somewhat straightforward problem to solve. While in recent years we've seen advancements in bio-metric authentication, largely authentication is achieved in the form of some sort of username and password combination.

The second problem is not as straightforward to solve. There are a lot of systems that we use everyday ranging from social media to more crucial services like banks and utilities that need a variety of information about us on top of the typical authentication that we are who we say we are. Social networks need to know who your contacts are, what your calendar data entails, who you converse with and when you're available. Banks and utilities need to know more basic information about you such as your mailing address and phone number.

Having a single canonical source for all this data sounds like a great idea doesn't it?

The Problems With The Problem Space

Federated identity and authentication come with the promise of managing your authentication and personal information in one place. In theory that's achievable, but it practice it's a lie. Users still have the burden of managing multiple profiles, duplicating information across sites, and figuring out each sites different privacy policies, what they do with your information and what information they share (or sell) to others.

Again, if there were one true canonical source for this information this wouldn't be a problem, but history has shown that there isn't going to be one canonical source. In fact history has shown that we've made the problem worse. Instead of having one place to manage data we've got a dozen. To make matters worse, most of those places do not stay in sync and we're forced to either deal with inaccurate/out of date data or to spend a lot of time keeping all our data up to date.

Still aren't convinced, how many of these federated identity and authentication sites do you have an account with?

• Microsoft
• Google
• Yahoo!
• Twitter
• LinkedIn
• PayPal
• Foursquare
• Mozilla
• Amazon

My guess is that you have an account on at least half of those, if not all of them. That data point in and of itself is proof that federated identity doesn't solve for the you're only going to have to manage your data in one place problem.

Redefining The Problem

I'd like to propose that we redefine the problem in such a way that

1. We allow users to store their personal data wherever they want
2. We make that data accessible and 100% in the control of the users
3. We authenticate users in such a way that they only have one username/password combination to remember and changes to their password are instantaneous across all sites they authenticate with.

The Solution

Solving this problem isn't actually all that difficult. It can largely be done with existing technologies and open standards.

Authenticate users using their existing email service provider

If every email provider exposed an OAuth provider that was built on top of SMTP Authentication then we'd have this service today. The user would be redirected to their email provider and asked to authenticate. They'd give their email provider their username/password combo and then it would issue an OAuth token to the service.

The users email/password combination are never shared and users don't have to remember multiple usernames/passwords. It comes with the added benefit that when they change their email password it's instantaneously reflected in any service they use.

Again this could be built into every existing service now using existing standards.

Allow users to store data wherever they want

As a user I should be able to store my data in the service of choice. I should be able to store my data in Facebook, Microsoft, Apple, Google, Amazon, or <insert your favorite cloud provider here> service. If I want to use another service then I should be able to point it at where I store my data and it should be able to read or modify it as I choose.

This also could be done today using existing standards and protocols. If each cloud provider did the following we'd liberate our data and ourselves from having to choose how and where we store data.

1. Expose CalDav, CardDav, and WebDav APIs to read/write contact, calendar, and file data.
2. Provide a mechanism that allowed users to grant access (OAuth anyone?) to their data from another service.

Example of how this would work in the real world

Let's say I decide that I want to use GMail as my email provider. I go sign up and GMail creates an account for me and exposes an OAuth provider for anyone that wants to integrate with my GMail account.

Later, I get an iPhone and want to use iCloud and Apple's calendar. When I sign into my iPhone I tell Apple that I use GMail for email and it also authenticates me using GMail and OAuth and creates an account for me. Part of what I get is a calendar, contact management, and file storage.

I then decide I want to join Facebook. I go to Facebook and instead of creating a new account, I tell Facebook to authenticate me using my GMail email address. When it authenticates me I tell GMail to give it access to my calendar, contacts, and file storage. Facebook creates a new user for me, identified by my email address. Facebook then asks if I have a calendar and contacts and I point it at Apple, who has exposed a CalDav and CardDav server. Since Facebook has been approved for those services Apple is able to validate I am who I say I am and Facebook now has access to my contacts and calendar. In fact Facebook could also use my file storage as the primary place it stores photos and videos I upload.

As a user I get to control my data AND still get access to all the great services I've come to enjoy. We all win.

Agile Development: Demo Day

So far in this Agile Development series I've:

• Made the case for Scrum
• Defined who and what a sprint team is
• Walked through the anatomy of a sprint
• Provided tools for determining sprint duration
• Detailed what daily stand-up looks like
• Provided the typical workflow for planning a sprint
• Outlined the sprint retrospective

In my final post in this series I'll explain demo day and how it relates to agile development.

What Demo Day Is

As I've been saying all along in this series, one of the main objectives of Agile is to reduce the time it takes to complete the feedback loop. The loop is complete when the stakeholders have reviewed the feature(s) and signed off on them. Demo day closes the loop and provides the stakeholders with an opportunity to affect change.

I like to think of demo day as the bow that is tied around the sprint. It brings closure to the sprint process which started with sprint planning. At it's core the sprint demo is the teams opportunity to showcase what they've built during the sprint. Beyond that sprint demos can also be a useful tool leading into the next sprint planning phase as the product manager may change the plan for the next sprint based on the progress of the team.

There are two main parts to demo day. First is to review the work completed and not completed during the sprint. Second is to demonstrate the completed work to the stakeholders.

Reviewing Work

Reviewing the sprint work should be comprised of three things. First the team should very briefly call out what stories the team accepted into the sprint during planning. Second the team identifies what stories were added to the sprint after the sprint started. Finally the team goes over what stories were not completed during the sprint.

It's important to note that the review is not an exercise in excuse making or justifying not getting everything done. It's main purpose is to close the feedback loop with stakeholders and set expectations prior to heading into sprint planning.

The Demonstrations

Demonstrations are just that, demoing what the team built during the sprint. Typically the demo is limited to the user impacting aspect of the features the sprint team worked on during the sprint. The team reviews the features to ensure they are complete as well as provide product management the opportunity to get clarifications on behalf of the customer.

One mistake I've often seen people make is trying to "demo" spreadsheets or powerpoint bullets. This usually happens because the team was mostly in bug fix mode or works on non-user facing features. I would encourage teams in this situation to get creative and tell a story using charts, graphs, and metrics instead of showing data in excel or a list of powerpoint bullets.

Agile Development: Sprint Retrospective

In my previous post in this Agile Development series I detailed the typical workflow for planning a sprint. In this post I will outline the process of the sprint retrospective. One helpful way to think about the sprint retrospective is as a mini postmortem. During the retrospective the team goes over what went well, what didn't go well, and what the team could have done better.

What Went Well

The reason we ask what went well is not so that we can pat each other on the back and inflate our egos. One of the keys to a successful agile project is consistency in the velocity of the team sprint over sprint. One of the best ways to do this is to identify what lead to success so that the team can capitalize on it in the upcoming sprint(s).

What Didn't Go Well

As with most growth being realistic, open and honest about our mistakes and failures leads to not repeating them. The purpose of identifying what didn't go well is NOT to lay blame on any particular person or people. Identifying what didn't go well helps us to avoid or correct them those mistakes.

What Could Be Done Better

At first this may not seem different from identifying what didn't go well. The real difference is that we recognize things that may have gone well but could have gone better. Asking what we could do better is an opportunity to identify gaps in our planning, our process, and our estimates. This question should lead directly to actions that can be taken to increase or maintain the teams current velocity.

Because the sprint review typically takes place before the planning of the next sprint asking these three questions provides the team with the opportunity to make changes that give them a higher likelihood for success before the next sprint starts.

Agile Development: Planning A Sprint

In my previous post in this Agile Development series I walked you through the daily stand-up. In this post I'll go outline the typical workflow for planning a sprint.

Identify The Features

The product owner, who is serving as the voice of the customer, proposes the highest priority features. The scrum team is responsible for getting clarification, pushing back on features, and proposing features or work that should be accomplished. Features should be prioritized based on their overall business value. It's important to note that working on tech debt can often provide down stream business value by enabling new feature work.

Prioritize and Estimate

Once the scrum team has identified the features to work on in the sprint, the team should then pull all of the stories associated with that feature into the sprint backlog. The product owner and scrum team should negotiate the priority of the stories. Once the priority of the stories has been determined the team needs to estimate the level of effort for each individual story.

One typical way of estimating stories during planning is to pick a story that does not have a lot of ambiguity which represents a medium (tee-shirt sized) amount of work. The team then assigns that story a point value (the middle value of their pointing system). The team then moves on to point the rest of the stories based on them being either more effort or less effort than the original story. Stories that are more effort are assigned a higher point value while stories with lower effort are assigned a lower point value.

Determine What Stories Make It Into The Sprint

Just because the product owner or scrum team has identified a feature (and it's related stories and bugs) as high priority doesn't mean that the team can actually complete that feature in a sprint. The team needs a way to estimate the amount of work that can be accomplished by the team during the sprint. This is typically done through velocity tracking.

Velocity in agile is typically determined by the amount of points the team was able to accomplish in the previous sprint. Historical sprints aren't to be taken into account as each sprint the teams goal is to get better and better as estimating the level of effort for story work.

The sprint backlog should only contain enough stories to meet the teams velocity. Choosing to few story points means that the team will likely have to poach more work (which hasn't been vetted) from the product backlog. Choosing too many story points means the team is not likely to finish all the work in the given sprint, setting the team up for failure from the start.

At first the teams velocity will be all over the place as they work out their estimation techniques. But over time the team should start to settle in to a consistent velocity. It's important to be aware that there are several common practices that can negatively affect a teams velocity. It's important for the sprint team to guard against these in order to accurately gauge their velocity sprint over sprint.

Under or over estimating the level of effort of a story

Often this is the result of ambiguity that is not resolved priority to planning. If the team doesn't have enough information to accurately gauge the level of effort for a particular story the product owner and scrum master should first work to resolve the ambiguities before a feature or story is considered for a sprint. This is an example of a valid reason for the sprint team to push back on a particular feature or story.

Unplanned sprint work or re-prioritization of the sprint deliverables mid-sprint

This is an Agile no-no and should be avoided at all costs. In Agile our sprint duration is typically short enough that when new work comes up we can prioritize it at the next sprint planning meeting. If new high priority work or re-prioritization mid-sprint is truely unavoidable the best way to make sure it doesn't impact the overall sprint deliverables is for the whole team to understand what the lowest priority work is and have that work drop out of the sprint.

Unaccounted for absence

People are going to take vacation, get sick, or be off for a holiday. It's difficult to account for sick time but it is very easy to account for holiday's and vacation time. Make sure in your sprint planning you understand how many days during the sprint people know they will be out and reduce that sprints velocity accordingly.

Other regular duties

Many software organizations use the DevOps model where the developer is on-call for the software and services they own. Not accounting for this type of work can have a negative impact on planning as the sprint team will sign-up for more work than they can actually accomplish.

Break The Stories down into tasks

Once the sprint backlog has been determined the team should break the stories down into smaller more granular tasks. This allows for more story work to be done in parallel. The team can swarm on a story together ensuring that the highest priority work is done first.

Once the tasks are identified the team is ready to start the sprint!

Agile Development: Stand-up

In my previous post in this Agile Development series I explained two ways to determine sprint duration. In this post I'll go into detail on what an agile stand-up typically looks like.

One thing you've heard me mention over and over in this series is that one of the primary goals of agile is to decrease the total time it takes to complete the feedback loop. One key way to accomplish this is with a daily stand-up.

In it's most basic form the stand-up is a time-boxed meeting (typically 15 minutes) where the scrum team gets together and answers three simple questions:

1. What did I do yesterday?
2. What am I going to do today?
3. What am I blocked on?

What Did I Do Yesterday?

In it's most basic form this question is intended to determine if progress was made towards the overall sprint goals. Knowing, as a team, what has already been accomplished helps the team formulate a plan for what needs to be accomplished over the next day.

I've seen this question often turn into a check-list of everything you did during the day. But it's important to keep in mind that your goal is not to provide an overview of everything you did. Instead, the purpose of this question is to make sure that others are aware of how much progress was made towards the work you committed to the day before. This is especially important when work items in the sprint are interdependent.

What Am I Going To Do Today?

There are two reasons that telling the team what you plan to accomplish for the day is important. First, it allows the team to validate whether the work is really the top priority for the sprint. During sprint planning many teams will prioritize the backlog for the sprint. But because our feedback loop is so short the priorities of any individual task may change throughout the sprint or even daily.

The second reason this question is important is that you are making a commitment to the team to accomplish a particular task. You can think of it as your daily contract with the team. And in turn the other members of the team are making a commitment to you to get other work done. This is how the team is able to increase and maintain a particular velocity. The team is able to time and coordinate work at a granular level that allows the team to maintain a constant throughput. 

What Am I Blocked On?

Making sure that the team is aware of any potentially blocking issues allows the team to react quicker and not allow blocking issues to affect the teams throughput. Because the scrum team talks about this daily, it is able to address potentially blocking or blocking issues immediately.

If a particular team member is blocked on a technical issue, I've often seen the scrum team swarm on the issue until it gets resolved. This has the benefit of making sure any blocking issue that is an upstream dependency of other sprint tasks gets resolved quickly and the team is able to dedicate more of the sprint to work that was planned.

If an issue cannot be resolved due to some unforeseen circumstance the sprint team has the ability to adjust and pull in additional tasks into the sprint while the blocking issue is being resolved. Because we talk about blocking issues daily, there shouldn't be a significant impact on the sprint schedule.

Parking Lots

It's often the case that during stand-up an issue comes up that requires a deeper dive than the 15 minutes allotted allows for. Fight the urge to go into detail during stand-up on the issue. Instead, announce that you have a parking lot and who should attend and then table the issue until after the stand-up is over. This allows less of the team to be randomized by an issue that isn't directly related to their daily commitment. 

Agile Development: Determining Sprint Duration

In my previous post in this Agile Development series I explained the anatomy of a sprint in terms of the three different sprint phases. In this post I'll detail the two main ways sprint duration is determined.

Fixed Time

Most sprint teams you come across will use a fixed time for their sprints. In my experience 2 to 3 week sprints are the most common duration. The goal of a fixed time sprint is to shorten the feedback loop with the customer in order to deliver the right features to them. By limiting the time to 2 or 3 weeks you can easily change course as the requirements change and provide continued business value to the customer without spending a lot of time on features that won't be used or whose requirements have changed.

Having a fixed time sprint allows the team to set expectations with your customer on what new features they will be building and when they will be available to the customer. I would strongly suggest that you don't have sprints that are longer than 3 weeks in duration. 2 to 3 weeks keeps the feedback loop small while also giving the sprint team enough time to build a feature out end to end. Increasing the amount of time it takes to complete the feedback loop means that the sprint team could spend more time building the wrong product and less time building the correct product should the requirements change.

It is occasionally the case with a fixed time sprint where the sprint will end and the team will have been unable to complete a particular feature. In this case the retrospective should identify the cause of the decreased velocity. Typical causes for a decrease in velocity (in my experience) have been:

• Failure to account for vacation or known leave of sprint team members.
• Failure to account for time spent on external work (like on-call duties).
• Failure to account for the unexpected work that grows the sprint backlog.
• Undefined or not well defined definitions of done.
• Stories that are too large and hide dependencies.
• Unknown external dependencies. 

Fixed Feature

Another, less common, way to determine sprint duration is to use a fixed feature approach. This approach says that a sprint is complete only when a particular feature is complete regardless of time. This is typically used when it is known that dependencies are hidden but that it is not easy to uncover those dependencies without first doing some work towards the end goal of the feature. In a system without a well defined or understood architecture, a fixed feature sprint duration is often used as it allows for the unknowns that go with the not well defined architecture.

Earlier in my career this was a more appealing way to determine sprint duration but as I've grown in my Agile planning I've come to realize that open-ended sprint duration's actually detract from shipping software more than they contribute to shipping complete software. Even with the most ambiguous problems it is usually better to first root out the ambiguity in a fixed time sprint and then plan and work on the feature.

Another reason that I've found for fixed feature sprints being less than ideal is that you have a harder time gauging your velocity as your velocity is determined by the amount of work you can complete in a fixed amount of time. Since knowing you're velocity is key for estimating when a feature is complete without it you cannot set your customers expectations as to when they will get a particular feature.

Fixed feature sprints also increase the time it takes to complete the feedback loop. And, as I mentioned earlier, increased feedback loops mean that it takes longer to respond to customer requests.

Agile Development: The Anatomy Of A Sprint

In my previous post in the Agile Development series I walked through the makeup of a sprint team. I explained the roles of Product, Dev, QA, Operations, and the Scrum Master. In this post I will go into detail on the anatomy of a sprint.

Planning

The first phase of the sprint life-cycle is planning. During the planning phase sprint velocity is determined, features are identified, features are estimated, and the backlog is created.

At it's most basic level the velocity of the sprint is determined by the number of points that were completed during the last sprint. When a sprint team first forms their velocity is very volatile. But as the team learns to estimate their story points more accurately there is less oscillation between the number of points completed sprint over sprint.

Determining which features are candidates for inclusion usually starts with product as the voice of the customer. Product will advocate for what the customer needs. The team will also identify what bugs and tech debt need to be addressed during the sprint.

Once the candidate features are identified the team will estimate the the level of effort for each story. At the formation of the sprint team a point system is determined and that point system is used for estimating work. The two most common point systems I've come across are using Fibonacci numbers (with 8 or 13 being the top end of a sprint) and using 1, 5, 10, 25, 50, and 100 as story points. Whatever your system of pointing there needs to be a way to determine small, medium and large stories.

There are many different estimating techniques but most teams I've seen use some sort of t-shirt size technique. They start by picking a story and assigning it an arbitrary point value. The rest of the stories are then pointed by determining if the story is larger or smaller than the initial story. When you have more than 3 assignments that you can use for point values you can get more granular and determine if the story is larger or smaller than the other stories in it's t-shirt size bucket.

Once the velocity is determined, potential features are identified, and stories are estimated you will be able to create your sprint backlog. Backlog creation begins by stack ranking the stories (taking dependencies into account) and cutting the sprint off after the last story that keeps the total number of sprint points at or less than the pre-determined sprint velocity.

Implementation

The second phase of the sprint life-cycle is implementation. During the implementation phase the team meets daily at stand-up and goes over what they worked on yesterday, what they're working on today, and any potential blocking issues. When a blocking issue is identified it's up to the scrum master to try to unblock the team member. During the implementation phase the software should be developed, tested, and integrated.

Review

The last phase of the sprint life-cycle is review. The review phase takes on two forms. The first is the sprint demo and the second is the sprint retrospective.

The sprint demo is the teams opportunity to showcase what they've built during the sprint. Sprint demos can actually be a very useful tool leading into the next sprint planning phase. Often when we think about planning with regards to software we think about what we're going to build from an architectural perspective. The problem with this line of thinking is that it typically leaves out integration, testing, and deployment. If we instead think about what we're going to demo during planning we'll have to take into account how we're going to demo it.

A note of caution about sprint demos. One mistake I've often seen teams make during sprint demos is to walk everyone through a bullet list of work that was performed. Walking others through a bullet list isn't a demo. If you find that your team is working on features that don't have a user facing component then figure out a creative way to tell the story of how what you worked on matters. Use an animation, a use case scenario, or maybe even a chart or some graphs to help illustrate what you worked on. 

The sprint retrospective is a mini postmortem. During the retrospective the team goes over what went well, what didn't go well, and what the team could have done better. The retrospective is a dedicated time for us to identify the issues that caused us to lose velocity, to call out our successes during the sprint, as well as identify what we could have done differently to have improved the sprint. Because the sprint review typically takes place before the planning of the next sprint we have an opportunity to make changes that give us a higher likelihood for success before the next sprint starts.

The retrospective IS NOT a finger pointing exercise. It's NOT a witch hunt or an opportunity to push blame around. The retrospective IS an opportunity to identify gaps in our planning, our process, or our estimates. The point of the retrospective is to maintain or increase our velocity.

Isn't there a phase missing?

You may be wondering where the "release" phase is. There isn't one. Releasing your software should be done as part of the implementation phase. Your sprint planning should account for what it takes to ship the software, if that's part of the definition of done for the sprint.

Agile Development: Sprint Teams

Sprints are the way that Scrum teams iterate over software. The goal of a sprint is to produce a working set of software that is potentially shippable. I say potentially because it's really up to the business to decide when the software is shippable. As long as the feature development is complete there is a choice. You can evaluate the number and severity of open bugs as well as the other features and make a call on whether you ship or not.

Sprint Teams

Sprint teams are comprised (at minimum) of a representative of product and the software development engineers. Though in practice you'll really want someone from QA (quality assurance, if your org has a separate QA group) and someone from operations represented.

Products Role

In Agile we favor collaboration over contract negotiation. The role of product management as part of the scrum team is to represent the voice of the customer. The product manager should be working directly with the customer to understand their needs. The product manager helps collaborate with the rest of the team on the requirements of the feature(s) being developed each sprint.

Developments Role

The developers are the ones actually building out the features. The role of the developer is to collaborate with product to turn the requirements into feature stories and tasks. The developer is also the advocate for the health of the code and architecture. The developer is responsible for making sure that unnecessary tech debt isn't accrued sprint over sprint. Often this means collaborating with product on the scope of work for each sprint to account for taking on bug fixes and technical debt.

Quality Assurances Role

Ideally all software we write includes unit, functional and integration tests. But that doesn't mean that all testing can be automated. There are some aspects of the user experience that may require manual testing. Or if your organization is new to agile you may have creating automated tests as tech debt. Whatever the reason, our goal of building potentially shippable software each sprint means including QA as part of the sprint team.

Operations Role

One other often overlooked member of the sprint team is operations. While continuous delivery and continuous integration are the ideals for deployment, many organizations have not shifted to that model yet. If your org is one that hasn't, including operations will help accurately account for the sprint work necessary to release the software.

Scrum Master

Each sprint has a scrum master. It is the role of the scrum master to facilitate scrum. This is done by managing the product backlog, facilitating the scrum meetings, facilitating the sprint planning meeting, facilitating the retrospective meeting, helping to define what done means for a given story or task, and removing roadblocks to the team. The scrum master is also responsible for removing roadblocks to the scrum team. This is usually done by fostering collaboration between teams and product.

Agile Development: The Need For Scrum

Over the last two decades the internet has become ubiquitous in our lives. The speed of our internet connections and performance of our computers have increased several orders of magnitude. These changes have had a cascading effect on the software industry. Software used to be run on hardware to expensive for the average person (or even company) to purchase and maintain. Bandwidth limitations required software to be distributed via a high cost and immutable media like the compact disk. As hardware prices decreased and the bandwidth and speed of our connections increased it became easier to distribute software over the web or even run it entirely in the browser using a combination of HTML, JavaScript and CSS. 

As software became easier to distribute our ability to innovate and provide our customers with fixes and new features became paramount to the success of a software product. The birth of the mobile computing industry with devices powerful enough to run both native and web based software intensified our need to respond to the ever changing needs of our customers.

One of the side affects of software distribution evolving has been that our processes, originally created to manage change over a long period of time, became inflexible and costly in a world where change was frequent. The software industry was plagued with process that couldn't keep up with the rapid multi-platform change the industry was going through.

This inflexibility in the software development process gave birth to the Agile movement. Agile recognized that responding to the changing needs of our customers has become as important or more important than following a long term plan. Collaboration on product requirements needed to take precedence over processes or tools. Agile understood that rather than formalizing requirements and managing those requirements as they flowed downstream that it needed to embrace iterative development and distribution to keep up with the ever changing needs of it's customers.

Agile understood that the cost of downstream change in a rigid process is greater than identifying the change upstream and pivoting earlier. In order to manage this change agile embraced the concept of Scrum. In traditional software development information flowed downstream from product to development, development to test, test to operations, and operations to the customer. If a change was needed it required a renegotiation of requirements and schedule due to the fact that any change restarted the process at the beginning.

Scrum increases the speed and flexibility of software development by bringing product, development, test and operations together into a cross-functional team responsible for adapting to change. Instead of requirements flowing downstream in a rigid process, requirements are collaborated on with product and test. Requirements are turned into features which are developed iteratively. At the end of each iteration the next set of requirements is defined. These requirements can build upon a previous iteration or completely change the direction of the software. Each iteration is time-boxed in order to allow for greater speed and flexibility in change.

Over the next few weeks I'll go in-depth into the different pieces of Agile Scrum with the goal of providing you a framework by which you can apply agile as part of your development process.

Minimizing the risk of bugs

Software development is a craft that requires practice, hard work and dedication. It's a craft that involves many edge cases and unintended consequences. As awful as it sounds, we've all got bugs in our code. The goal of writing software should not be to write software with no bugs as this is unattainable. Instead the goal should be to minimize the risk of high severity bugs.

Every change to your software is an opportunity to introduce new bugs. Following these tips will help you minimize the risk of introducing bugs into your system.

Test Your Software

Duh, right? Testing your software may sound like a no-brainer but you'd be surprised by how many times people break builds and introduce buggy software just because they didn't test their code. In my previous post on Testing Your Software Properly I provided a checklist of tests that you should run before you commit your code.

Without proper tests you can not have confidence that you aren't regressing an old bug or introducing a new bug into the system. Tests are the foundation for confidence in any change you make.

Reduce the surface area of your change

The more lines of code you change with every commit the higher the risk of bugs. This is where encapsulation, SOLID principles, and refactoring become so important.

It's important to encapsulate your software into individual loosely coupled pieces. If you make a change in a class and it causes cascading changes throughout the rest of your software in code un-related to your change then you're likely introducing new bugs.

If you follow the Single Responsibility Principle in SOLID your classes will have one reason and only one reason to change. This reduces the surface area of your change because you will not be changing code in unrelated modules.

Following principles like DRY and YAGNI will lead to more robust code that is flexible and easy to change.

Keeping your code simple is one of the keys to reducing the surface area of your change.

Reduce the complexity of the code

Overly complex code leads to bugs for many reasons:

• The code is fragile because the learning curve is steep.
• It's easier to do the wrong thing than it is the correct thing when making a change in the code.
• The code is not readable often causing you to make multiple context switches to understand a single workflow.

What are some signs that your code is too complex?

• The patterns in the code are not clear, obvious, and/or discoverable.
• You have multiple levels in indirection that support one workflow or use case.
• You have an abstraction that fronts a single concrete implementation.
• Your code does not have clear boundaries.
• You have highly interdependent modules.
• Your code is not highly cohesive.
• Your code has rigid rules that are not enforceable in their individual units but only as a whole.

Testing your software properly

Here's a general checklist of the type of tests that you can use to ensure that you're testing your software properly before you ship it. This isn't an exhaustive list but can be used as a starting point for you to write a more exhaustive list that's right for your software.

Compile before you commit

Good testing starts with making sure your code actually compiles. Yes, people actually check in code without compiling it first. This is just a dumb mistake and is 100% avoidable. 

Run a clean build before you commit

A somewhat non-obvious thing to make sure when compiling is that you're not using a cached compiled object that has changed. Often compilers will cache objects and attempt to track when the dependency changes and only recompile the dependent object when the compiler believes it has actually changed. Cached objects will then be linked against your code and make your software appear to work but those objects may have actually changed and broken functionality. Doing a clean build before you commit will ensure that an object you depend on hasn't changed in such a way as to break your code integration.

Happy Path Tests

Happy path tests should be your minimum bar when committing code. Happy path tests ensure that your code works as intended when used in the way it was designed. These tests can be thought of as functional tests. They test the functionality of the software and ensure that the software meets the business requirement.

Negative Path Tests

Negative path tests ensure that your software is resilient to change. Negative path testing includes using your code in ways for which it was not intended. Common tests include sending in null object parameters and testing upper and lower bounds of parameters. Negative path testing also includes testing that your software properly handles exceptions and throws the proper exceptions.

White Box Tests

White box tests ensure that your code works from the outside in. These are a set of tests that ensure your objects work from the consumers perspective. These tests include making sure the object can be created and initialized, that method calls work according to spec, and that the code does not misbehave from the callers perspective.

Black Box Tests

Black box tests ensure that your code works from the inside out. These tests require access to the internals of the object. Black box tests usually test the fitness of particular private methods and algorithms. 

Life-cycle Tests

You should test how your objects function in various aspects of the objects life-cycle. The key to life-cycle tests is to make sure that your objects mange state properly. Life cycle tests are also useful in making sure that you don't have any memory leaks in your code due to life-cycle changes.

Life-cycle testing includes testing the creation, destruction, concurrency and serialization of your objects. Two life-cycle areas that tend to cause bugs are not properly testing when the object state is saved or restored or when the object is used in a multi-threaded environment.

Integration Tests

Do you understand how your software works in the context of the larger system of components that use and are used by your software? Integration tests allow you to make sure that your software works end to end in the system as a whole. 

When Not To Refactor

Refactoring software is a crucial part of extending the life of software. Refactoring contributes to enhancing the maintainability of the software by incrementally improving the design, readability and modularity of the components. But not much has been said about when not to refactor software.

Don't refactor code unless you need to change the code for a business reason.

One of the common mistakes I often see with regards to refactoring is when people refactor code that doesn't need it under the guise of making it better. The argument usually goes something like "this needs to be more abstract", "I wrote this code a long time ago and it is crappy", "this code is too complex" or something along those lines.

You should only refactor code when you are already in the code to make a change to support the business. That may sound counter intuitive but one of the worst things we can do is change code, however crappy, unreadable or complex that doesn't have a reason to change.

Valid business reasons to change code include (but are not limited to):

• Adding new functionality
• Extending existing functionality.
• Making measurable performance improvements.
• Adding a layer of abstraction in order to support a new use case.
• Modularizing a particular object so that it can be reused in another part of the system

Adding new functionality or Extending existing functionality

This is where the boyscout rule comes into play. If you are in already in the code for another reason then you should clean up the code even if you didn't make the mess.

Making measurable performance improvements

This one is probably self explanatory but it's important to note that performance improvements will usually require some level of refactoring. 

Adding a layer of abstraction in order to support a new use case

This is an important one to understand. Often people will over generalize code at the beginning. This leads to overly complex designs and less readable code. If we follow the rule of not creating a layer of abstraction until we have at least two or three use cases for the code then there will come a point when you need to refactor the code in order to provide a layer of abstraction that doesn't already exist.

Until that second or third use case comes about the code should not be generalized. You don't have enough information about future uses of the code to get the abstraction correct. You may get lucky and guess at the future abstraction but you don't want to run your business on guesses and luck.

Modularizing a particular object so that it can be reused in another part of the system

Code reuse is one of the most important tenets of object oriented programming. When we identify code that is not specific to a particular object or package AND is needed in some other part of the system we should refactor this code into it's own module. Its important to ONLY do this when the code is actually needed in another part of the system.

Don't refactor code without tests

In order to refactor code safely you should have unit and integration tests for the existing functionality. I would also argue that you should write tests for the new functionality as well before you refactor. This will help you to understand the proper way to refactor the code as it helps you define how the refactored code should be used from a consumers standpoint.

If the tests don't exist for the the existing functionality you should write them first before you start refactoring. This helps ensure that you don't cause a new bug in the code or regress an old bug when refactoring.

The Engineers Cloud

In my previous post in this series I explained the aspect of The Cloud that I like to call The Consumers Cloud. I explained how The Consumers Cloud breaks down into data management services, social media, and streaming media. In this post I'll talk about the second aspect of The Cloud.

The Engineers Cloud

I call this type of Cloud use The Engineers Cloud because this aspect of The Cloud isn't something you as a consumer interact with directly. Instead, engineers are taking advantage of Cloud services to enhance how you interact with their content and services.

What The Cloud Means To Engineers

While there are almost no limits to the things you can do in The Cloud from an engineers perspective, there are two main areas I'd like to focus on here. The first is as means of distributed computing. The second is better reliability of their services.

Distributed Computing

The Engineers Cloud allows you to take advantage of the virtual limitlessness of server resources in The Cloud. In the days before The Cloud server resources were finite. You only had the amount of resources you could afford to keep running all the time. These resources lived in data centers.

Companies like Facebook, Netflix, Amazon, and Google use The Cloud to do a variety of tasks that would be nearly impossible with a fixed set of resources. The ability to spin up an (almost) unlimited amount of servers running your services means that you can parallelize computing to a degree that was not possible a decade ago.

Some examples of engineers using The Cloud as a means of distributed computing:

• NASA's Jet Propulsion Laboratory (JPL) uses the cloud to capture and store images and metadata collected from the Mars Exploration Rover and the Mars Science Laboratory missions. They operate the mars.jpl.nasa.gov website out of The Cloud without building this infrastructure themselves
• Accuweather is using the cloud to serve over 4 billion requests a day.
• Evite is using the cloud to send more than 250 million party invitations each year.
• Netflix is using the cloud to stream videos to it's online streaming customers. It's able to take advantage of The Cloud's distributing computing to analyze a very large amount of data and turn them into recommendations and personalization.

Better Reliability

This is going to sound counter intuitive, but one of the reasons that The Cloud is more reliable is that when planning to put your software and services in The Cloud you have to plan for failure. The best example of this in practice that I'm aware of is Netflix's Simian Army.

The Cloud allows you to plan for failure and provide better reliability because it allows for:

• Redundency through geo-distributing services.
• Redundency through clustering your services.
• Reduced latency through DNS services.

Redundency through geo-distributing services

Most Cloud providers offer the ability to deploy your software and services to many different regions around the world. This allows you to keep your software and services running even if there are data center outages in a specific region like the Northeast blackout of 2003 by having your services fallback from one geo-graphic region to another if the initial region is down.

Redundancy through clustering your services

Most Cloud providers give you the ability to cluster your services behind some sort of virtual load balancer. Most of these load balancers will automatically stop sending traffic to a machine that is not responding or throwing a particular error for a predefined URL on the machine.

While clustering your services behind a load balancer allows you to remove or replace a machine that isn't functioning properly it also is the primary means by which you can quickly scale up your service to meet demand. If your service is experiencing a higher than expected load you can spin up a new server in your cluster and scale proportionally with your traffic. 

Reduced latency through DNS services

DNS is how the internet turns the name of service we go to into the address that the service resides at. For example when you type http://paul.oremland.net into your browser your computer is doing a DNS lookup of paul.oremland.net and being given an IP address. It then uses that IP address to talk directly with the service.

Many Cloud providers allow you to virtually control DNS based on characteristics of the request. Some services allow you to route traffic based on latency to or load on the receiving services. This allows you to distribute your traffic more evenly and provide a better customer experience. Instead of simply pointing your users at a specific machine, you can point them to different machines based on the current state of your system and what gives the users a great experience.

The Consumers Cloud

In my previous post in this series I gave a basic overview of what The Cloud is, its benefits, its high level infrastructure, and why you should care about it. In this post I will go into more detail of what I call The Consumers Cloud.

The Consumers Cloud

Often when people talk about The Cloud what they're really talking about are the applications that are built on top of, and enabled by the infrastructure of The Cloud. At a high level those applications are what I would call The Consumers Cloud.

The main purpose of The Consumers Cloud is to provide distributed access to your data and the services that provide that data. Your data is typically comprised of images, videos, and documents but really it can be any files you need to put or get from a variety of machines in a variety of locations.

In The Consumers Cloud you don't interact with The Cloud directly. Instead you interact with services that are built in The Cloud. Those services are the means by which your data is moved around and presented to you on a variety of devices (mobile, desktop, and etc).

The Consumers Cloud breaks down into three high level areas. The first is data management services, the second is social media, and the third is streaming media.

Your Hard Drive Is Everywhere

You can think of The Consumers Cloud as your hard drive that is everywhere. Services like Dropbox, Amazon Cloud Drive, Microsoft OneDrive, and Apple iCloud all provided the ability to store you files on their servers in order for them to be accessible from anywhere on almost any machine. They take your data and using very sophisticated algorithms distribute that data in such a way as to make reading and writing it from anywhere in the world possible and fast.

Nowadays when you purchase a mobile phone it usually comes with some sort of Cloud backup. That means that the pictures and videos you capture on your phone are uploaded to one of these services and made accessible to you from your many different devices. You can share this media with others much easier since it isn't stored locally on your phone, tablet, laptop, or desktop.

Your Social Life Is Nowhere

The second high level area that The Consumers Cloud breaks into is social networking. Facebook, Twitter, Instagram, Pintrest, and etc all exist in The Cloud. Sometimes these social networks need very few servers to serve the traffic of their users. Sometimes they need thousands of servers to meet their peak demand. Without The Cloud they wouldn't be able to efficiently scale down or up to handle the large volume of traffic they get in a cost effective way. The Cloud also allows them to distribute data and distribute load in such a way as to optimize connecting their users to servers that are closer to them or that have less load at any given time. The Cloud allows them to handle the ebbs and flows of their traffic patterns so that their services are always there.

Your Entertainment Is Just There

The last high level area that The Consumer Cloud breaks into is streaming media. Good examples of this are Amazon Prime, Netflix, and YouTube. All these services are major players in online entertainment business and all of them rely on The Cloud as the backbone of their services. They use The Cloud to optimize the distribution of media so that it can be accessed by millions of people without having millions of people each hitting their data stores for every piece of media every time.

The Consumers Cloud is about you, your data, and your online life. In my next post in this series I'll detail The Engineers Cloud.