Agile Development: Demo Day

So far in this Agile Development series I've:

• Made the case for Scrum
• Defined who and what a sprint team is
• Walked through the anatomy of a sprint
• Provided tools for determining sprint duration
• Detailed what daily stand-up looks like
• Provided the typical workflow for planning a sprint
• Outlined the sprint retrospective

In my final post in this series I'll explain demo day and how it relates to agile development.

What Demo Day Is

As I've been saying all along in this series, one of the main objectives of Agile is to reduce the time it takes to complete the feedback loop. The loop is complete when the stakeholders have reviewed the feature(s) and signed off on them. Demo day closes the loop and provides the stakeholders with an opportunity to affect change.

I like to think of demo day as the bow that is tied around the sprint. It brings closure to the sprint process which started with sprint planning. At it's core the sprint demo is the teams opportunity to showcase what they've built during the sprint. Beyond that sprint demos can also be a useful tool leading into the next sprint planning phase as the product manager may change the plan for the next sprint based on the progress of the team.

There are two main parts to demo day. First is to review the work completed and not completed during the sprint. Second is to demonstrate the completed work to the stakeholders.

Reviewing Work

Reviewing the sprint work should be comprised of three things. First the team should very briefly call out what stories the team accepted into the sprint during planning. Second the team identifies what stories were added to the sprint after the sprint started. Finally the team goes over what stories were not completed during the sprint.

It's important to note that the review is not an exercise in excuse making or justifying not getting everything done. It's main purpose is to close the feedback loop with stakeholders and set expectations prior to heading into sprint planning.

The Demonstrations

Demonstrations are just that, demoing what the team built during the sprint. Typically the demo is limited to the user impacting aspect of the features the sprint team worked on during the sprint. The team reviews the features to ensure they are complete as well as provide product management the opportunity to get clarifications on behalf of the customer.

One mistake I've often seen people make is trying to "demo" spreadsheets or powerpoint bullets. This usually happens because the team was mostly in bug fix mode or works on non-user facing features. I would encourage teams in this situation to get creative and tell a story using charts, graphs, and metrics instead of showing data in excel or a list of powerpoint bullets.

Agile Development: Sprint Retrospective

In my previous post in this Agile Development series I detailed the typical workflow for planning a sprint. In this post I will outline the process of the sprint retrospective. One helpful way to think about the sprint retrospective is as a mini postmortem. During the retrospective the team goes over what went well, what didn't go well, and what the team could have done better.

What Went Well

The reason we ask what went well is not so that we can pat each other on the back and inflate our egos. One of the keys to a successful agile project is consistency in the velocity of the team sprint over sprint. One of the best ways to do this is to identify what lead to success so that the team can capitalize on it in the upcoming sprint(s).

What Didn't Go Well

As with most growth being realistic, open and honest about our mistakes and failures leads to not repeating them. The purpose of identifying what didn't go well is NOT to lay blame on any particular person or people. Identifying what didn't go well helps us to avoid or correct them those mistakes.

What Could Be Done Better

At first this may not seem different from identifying what didn't go well. The real difference is that we recognize things that may have gone well but could have gone better. Asking what we could do better is an opportunity to identify gaps in our planning, our process, and our estimates. This question should lead directly to actions that can be taken to increase or maintain the teams current velocity.

Because the sprint review typically takes place before the planning of the next sprint asking these three questions provides the team with the opportunity to make changes that give them a higher likelihood for success before the next sprint starts.

Agile Development: Planning A Sprint

In my previous post in this Agile Development series I walked you through the daily stand-up. In this post I'll go outline the typical workflow for planning a sprint.

Identify The Features

The product owner, who is serving as the voice of the customer, proposes the highest priority features. The scrum team is responsible for getting clarification, pushing back on features, and proposing features or work that should be accomplished. Features should be prioritized based on their overall business value. It's important to note that working on tech debt can often provide down stream business value by enabling new feature work.

Prioritize and Estimate

Once the scrum team has identified the features to work on in the sprint, the team should then pull all of the stories associated with that feature into the sprint backlog. The product owner and scrum team should negotiate the priority of the stories. Once the priority of the stories has been determined the team needs to estimate the level of effort for each individual story.

One typical way of estimating stories during planning is to pick a story that does not have a lot of ambiguity which represents a medium (tee-shirt sized) amount of work. The team then assigns that story a point value (the middle value of their pointing system). The team then moves on to point the rest of the stories based on them being either more effort or less effort than the original story. Stories that are more effort are assigned a higher point value while stories with lower effort are assigned a lower point value.

Determine What Stories Make It Into The Sprint

Just because the product owner or scrum team has identified a feature (and it's related stories and bugs) as high priority doesn't mean that the team can actually complete that feature in a sprint. The team needs a way to estimate the amount of work that can be accomplished by the team during the sprint. This is typically done through velocity tracking.

Velocity in agile is typically determined by the amount of points the team was able to accomplish in the previous sprint. Historical sprints aren't to be taken into account as each sprint the teams goal is to get better and better as estimating the level of effort for story work.

The sprint backlog should only contain enough stories to meet the teams velocity. Choosing to few story points means that the team will likely have to poach more work (which hasn't been vetted) from the product backlog. Choosing too many story points means the team is not likely to finish all the work in the given sprint, setting the team up for failure from the start.

At first the teams velocity will be all over the place as they work out their estimation techniques. But over time the team should start to settle in to a consistent velocity. It's important to be aware that there are several common practices that can negatively affect a teams velocity. It's important for the sprint team to guard against these in order to accurately gauge their velocity sprint over sprint.

Under or over estimating the level of effort of a story

Often this is the result of ambiguity that is not resolved priority to planning. If the team doesn't have enough information to accurately gauge the level of effort for a particular story the product owner and scrum master should first work to resolve the ambiguities before a feature or story is considered for a sprint. This is an example of a valid reason for the sprint team to push back on a particular feature or story.

Unplanned sprint work or re-prioritization of the sprint deliverables mid-sprint

This is an Agile no-no and should be avoided at all costs. In Agile our sprint duration is typically short enough that when new work comes up we can prioritize it at the next sprint planning meeting. If new high priority work or re-prioritization mid-sprint is truely unavoidable the best way to make sure it doesn't impact the overall sprint deliverables is for the whole team to understand what the lowest priority work is and have that work drop out of the sprint.

Unaccounted for absence

People are going to take vacation, get sick, or be off for a holiday. It's difficult to account for sick time but it is very easy to account for holiday's and vacation time. Make sure in your sprint planning you understand how many days during the sprint people know they will be out and reduce that sprints velocity accordingly.

Other regular duties

Many software organizations use the DevOps model where the developer is on-call for the software and services they own. Not accounting for this type of work can have a negative impact on planning as the sprint team will sign-up for more work than they can actually accomplish.

Break The Stories down into tasks

Once the sprint backlog has been determined the team should break the stories down into smaller more granular tasks. This allows for more story work to be done in parallel. The team can swarm on a story together ensuring that the highest priority work is done first.

Once the tasks are identified the team is ready to start the sprint!

Agile Development: Stand-up

In my previous post in this Agile Development series I explained two ways to determine sprint duration. In this post I'll go into detail on what an agile stand-up typically looks like.

One thing you've heard me mention over and over in this series is that one of the primary goals of agile is to decrease the total time it takes to complete the feedback loop. One key way to accomplish this is with a daily stand-up.

In it's most basic form the stand-up is a time-boxed meeting (typically 15 minutes) where the scrum team gets together and answers three simple questions:

1. What did I do yesterday?
2. What am I going to do today?
3. What am I blocked on?

What Did I Do Yesterday?

In it's most basic form this question is intended to determine if progress was made towards the overall sprint goals. Knowing, as a team, what has already been accomplished helps the team formulate a plan for what needs to be accomplished over the next day.

I've seen this question often turn into a check-list of everything you did during the day. But it's important to keep in mind that your goal is not to provide an overview of everything you did. Instead, the purpose of this question is to make sure that others are aware of how much progress was made towards the work you committed to the day before. This is especially important when work items in the sprint are interdependent.

What Am I Going To Do Today?

There are two reasons that telling the team what you plan to accomplish for the day is important. First, it allows the team to validate whether the work is really the top priority for the sprint. During sprint planning many teams will prioritize the backlog for the sprint. But because our feedback loop is so short the priorities of any individual task may change throughout the sprint or even daily.

The second reason this question is important is that you are making a commitment to the team to accomplish a particular task. You can think of it as your daily contract with the team. And in turn the other members of the team are making a commitment to you to get other work done. This is how the team is able to increase and maintain a particular velocity. The team is able to time and coordinate work at a granular level that allows the team to maintain a constant throughput. 

What Am I Blocked On?

Making sure that the team is aware of any potentially blocking issues allows the team to react quicker and not allow blocking issues to affect the teams throughput. Because the scrum team talks about this daily, it is able to address potentially blocking or blocking issues immediately.

If a particular team member is blocked on a technical issue, I've often seen the scrum team swarm on the issue until it gets resolved. This has the benefit of making sure any blocking issue that is an upstream dependency of other sprint tasks gets resolved quickly and the team is able to dedicate more of the sprint to work that was planned.

If an issue cannot be resolved due to some unforeseen circumstance the sprint team has the ability to adjust and pull in additional tasks into the sprint while the blocking issue is being resolved. Because we talk about blocking issues daily, there shouldn't be a significant impact on the sprint schedule.

Parking Lots

It's often the case that during stand-up an issue comes up that requires a deeper dive than the 15 minutes allotted allows for. Fight the urge to go into detail during stand-up on the issue. Instead, announce that you have a parking lot and who should attend and then table the issue until after the stand-up is over. This allows less of the team to be randomized by an issue that isn't directly related to their daily commitment. 

Agile Development: Determining Sprint Duration

In my previous post in this Agile Development series I explained the anatomy of a sprint in terms of the three different sprint phases. In this post I'll detail the two main ways sprint duration is determined.

Fixed Time

Most sprint teams you come across will use a fixed time for their sprints. In my experience 2 to 3 week sprints are the most common duration. The goal of a fixed time sprint is to shorten the feedback loop with the customer in order to deliver the right features to them. By limiting the time to 2 or 3 weeks you can easily change course as the requirements change and provide continued business value to the customer without spending a lot of time on features that won't be used or whose requirements have changed.

Having a fixed time sprint allows the team to set expectations with your customer on what new features they will be building and when they will be available to the customer. I would strongly suggest that you don't have sprints that are longer than 3 weeks in duration. 2 to 3 weeks keeps the feedback loop small while also giving the sprint team enough time to build a feature out end to end. Increasing the amount of time it takes to complete the feedback loop means that the sprint team could spend more time building the wrong product and less time building the correct product should the requirements change.

It is occasionally the case with a fixed time sprint where the sprint will end and the team will have been unable to complete a particular feature. In this case the retrospective should identify the cause of the decreased velocity. Typical causes for a decrease in velocity (in my experience) have been:

• Failure to account for vacation or known leave of sprint team members.
• Failure to account for time spent on external work (like on-call duties).
• Failure to account for the unexpected work that grows the sprint backlog.
• Undefined or not well defined definitions of done.
• Stories that are too large and hide dependencies.
• Unknown external dependencies. 

Fixed Feature

Another, less common, way to determine sprint duration is to use a fixed feature approach. This approach says that a sprint is complete only when a particular feature is complete regardless of time. This is typically used when it is known that dependencies are hidden but that it is not easy to uncover those dependencies without first doing some work towards the end goal of the feature. In a system without a well defined or understood architecture, a fixed feature sprint duration is often used as it allows for the unknowns that go with the not well defined architecture.

Earlier in my career this was a more appealing way to determine sprint duration but as I've grown in my Agile planning I've come to realize that open-ended sprint duration's actually detract from shipping software more than they contribute to shipping complete software. Even with the most ambiguous problems it is usually better to first root out the ambiguity in a fixed time sprint and then plan and work on the feature.

Another reason that I've found for fixed feature sprints being less than ideal is that you have a harder time gauging your velocity as your velocity is determined by the amount of work you can complete in a fixed amount of time. Since knowing you're velocity is key for estimating when a feature is complete without it you cannot set your customers expectations as to when they will get a particular feature.

Fixed feature sprints also increase the time it takes to complete the feedback loop. And, as I mentioned earlier, increased feedback loops mean that it takes longer to respond to customer requests.

Agile Development: The Anatomy Of A Sprint

In my previous post in the Agile Development series I walked through the makeup of a sprint team. I explained the roles of Product, Dev, QA, Operations, and the Scrum Master. In this post I will go into detail on the anatomy of a sprint.

Planning

The first phase of the sprint life-cycle is planning. During the planning phase sprint velocity is determined, features are identified, features are estimated, and the backlog is created.

At it's most basic level the velocity of the sprint is determined by the number of points that were completed during the last sprint. When a sprint team first forms their velocity is very volatile. But as the team learns to estimate their story points more accurately there is less oscillation between the number of points completed sprint over sprint.

Determining which features are candidates for inclusion usually starts with product as the voice of the customer. Product will advocate for what the customer needs. The team will also identify what bugs and tech debt need to be addressed during the sprint.

Once the candidate features are identified the team will estimate the the level of effort for each story. At the formation of the sprint team a point system is determined and that point system is used for estimating work. The two most common point systems I've come across are using Fibonacci numbers (with 8 or 13 being the top end of a sprint) and using 1, 5, 10, 25, 50, and 100 as story points. Whatever your system of pointing there needs to be a way to determine small, medium and large stories.

There are many different estimating techniques but most teams I've seen use some sort of t-shirt size technique. They start by picking a story and assigning it an arbitrary point value. The rest of the stories are then pointed by determining if the story is larger or smaller than the initial story. When you have more than 3 assignments that you can use for point values you can get more granular and determine if the story is larger or smaller than the other stories in it's t-shirt size bucket.

Once the velocity is determined, potential features are identified, and stories are estimated you will be able to create your sprint backlog. Backlog creation begins by stack ranking the stories (taking dependencies into account) and cutting the sprint off after the last story that keeps the total number of sprint points at or less than the pre-determined sprint velocity.

Implementation

The second phase of the sprint life-cycle is implementation. During the implementation phase the team meets daily at stand-up and goes over what they worked on yesterday, what they're working on today, and any potential blocking issues. When a blocking issue is identified it's up to the scrum master to try to unblock the team member. During the implementation phase the software should be developed, tested, and integrated.

Review

The last phase of the sprint life-cycle is review. The review phase takes on two forms. The first is the sprint demo and the second is the sprint retrospective.

The sprint demo is the teams opportunity to showcase what they've built during the sprint. Sprint demos can actually be a very useful tool leading into the next sprint planning phase. Often when we think about planning with regards to software we think about what we're going to build from an architectural perspective. The problem with this line of thinking is that it typically leaves out integration, testing, and deployment. If we instead think about what we're going to demo during planning we'll have to take into account how we're going to demo it.

A note of caution about sprint demos. One mistake I've often seen teams make during sprint demos is to walk everyone through a bullet list of work that was performed. Walking others through a bullet list isn't a demo. If you find that your team is working on features that don't have a user facing component then figure out a creative way to tell the story of how what you worked on matters. Use an animation, a use case scenario, or maybe even a chart or some graphs to help illustrate what you worked on. 

The sprint retrospective is a mini postmortem. During the retrospective the team goes over what went well, what didn't go well, and what the team could have done better. The retrospective is a dedicated time for us to identify the issues that caused us to lose velocity, to call out our successes during the sprint, as well as identify what we could have done differently to have improved the sprint. Because the sprint review typically takes place before the planning of the next sprint we have an opportunity to make changes that give us a higher likelihood for success before the next sprint starts.

The retrospective IS NOT a finger pointing exercise. It's NOT a witch hunt or an opportunity to push blame around. The retrospective IS an opportunity to identify gaps in our planning, our process, or our estimates. The point of the retrospective is to maintain or increase our velocity.

Isn't there a phase missing?

You may be wondering where the "release" phase is. There isn't one. Releasing your software should be done as part of the implementation phase. Your sprint planning should account for what it takes to ship the software, if that's part of the definition of done for the sprint.

Agile Development: Sprint Teams

Sprints are the way that Scrum teams iterate over software. The goal of a sprint is to produce a working set of software that is potentially shippable. I say potentially because it's really up to the business to decide when the software is shippable. As long as the feature development is complete there is a choice. You can evaluate the number and severity of open bugs as well as the other features and make a call on whether you ship or not.

Sprint Teams

Sprint teams are comprised (at minimum) of a representative of product and the software development engineers. Though in practice you'll really want someone from QA (quality assurance, if your org has a separate QA group) and someone from operations represented.

Products Role

In Agile we favor collaboration over contract negotiation. The role of product management as part of the scrum team is to represent the voice of the customer. The product manager should be working directly with the customer to understand their needs. The product manager helps collaborate with the rest of the team on the requirements of the feature(s) being developed each sprint.

Developments Role

The developers are the ones actually building out the features. The role of the developer is to collaborate with product to turn the requirements into feature stories and tasks. The developer is also the advocate for the health of the code and architecture. The developer is responsible for making sure that unnecessary tech debt isn't accrued sprint over sprint. Often this means collaborating with product on the scope of work for each sprint to account for taking on bug fixes and technical debt.

Quality Assurances Role

Ideally all software we write includes unit, functional and integration tests. But that doesn't mean that all testing can be automated. There are some aspects of the user experience that may require manual testing. Or if your organization is new to agile you may have creating automated tests as tech debt. Whatever the reason, our goal of building potentially shippable software each sprint means including QA as part of the sprint team.

Operations Role

One other often overlooked member of the sprint team is operations. While continuous delivery and continuous integration are the ideals for deployment, many organizations have not shifted to that model yet. If your org is one that hasn't, including operations will help accurately account for the sprint work necessary to release the software.

Scrum Master

Each sprint has a scrum master. It is the role of the scrum master to facilitate scrum. This is done by managing the product backlog, facilitating the scrum meetings, facilitating the sprint planning meeting, facilitating the retrospective meeting, helping to define what done means for a given story or task, and removing roadblocks to the team. The scrum master is also responsible for removing roadblocks to the scrum team. This is usually done by fostering collaboration between teams and product.

Agile Development: The Need For Scrum

Over the last two decades the internet has become ubiquitous in our lives. The speed of our internet connections and performance of our computers have increased several orders of magnitude. These changes have had a cascading effect on the software industry. Software used to be run on hardware to expensive for the average person (or even company) to purchase and maintain. Bandwidth limitations required software to be distributed via a high cost and immutable media like the compact disk. As hardware prices decreased and the bandwidth and speed of our connections increased it became easier to distribute software over the web or even run it entirely in the browser using a combination of HTML, JavaScript and CSS. 

As software became easier to distribute our ability to innovate and provide our customers with fixes and new features became paramount to the success of a software product. The birth of the mobile computing industry with devices powerful enough to run both native and web based software intensified our need to respond to the ever changing needs of our customers.

One of the side affects of software distribution evolving has been that our processes, originally created to manage change over a long period of time, became inflexible and costly in a world where change was frequent. The software industry was plagued with process that couldn't keep up with the rapid multi-platform change the industry was going through.

This inflexibility in the software development process gave birth to the Agile movement. Agile recognized that responding to the changing needs of our customers has become as important or more important than following a long term plan. Collaboration on product requirements needed to take precedence over processes or tools. Agile understood that rather than formalizing requirements and managing those requirements as they flowed downstream that it needed to embrace iterative development and distribution to keep up with the ever changing needs of it's customers.

Agile understood that the cost of downstream change in a rigid process is greater than identifying the change upstream and pivoting earlier. In order to manage this change agile embraced the concept of Scrum. In traditional software development information flowed downstream from product to development, development to test, test to operations, and operations to the customer. If a change was needed it required a renegotiation of requirements and schedule due to the fact that any change restarted the process at the beginning.

Scrum increases the speed and flexibility of software development by bringing product, development, test and operations together into a cross-functional team responsible for adapting to change. Instead of requirements flowing downstream in a rigid process, requirements are collaborated on with product and test. Requirements are turned into features which are developed iteratively. At the end of each iteration the next set of requirements is defined. These requirements can build upon a previous iteration or completely change the direction of the software. Each iteration is time-boxed in order to allow for greater speed and flexibility in change.

Over the next few weeks I'll go in-depth into the different pieces of Agile Scrum with the goal of providing you a framework by which you can apply agile as part of your development process.

When Not To Refactor

Refactoring software is a crucial part of extending the life of software. Refactoring contributes to enhancing the maintainability of the software by incrementally improving the design, readability and modularity of the components. But not much has been said about when not to refactor software.

Don't refactor code unless you need to change the code for a business reason.

One of the common mistakes I often see with regards to refactoring is when people refactor code that doesn't need it under the guise of making it better. The argument usually goes something like "this needs to be more abstract", "I wrote this code a long time ago and it is crappy", "this code is too complex" or something along those lines.

You should only refactor code when you are already in the code to make a change to support the business. That may sound counter intuitive but one of the worst things we can do is change code, however crappy, unreadable or complex that doesn't have a reason to change.

Valid business reasons to change code include (but are not limited to):

• Adding new functionality
• Extending existing functionality.
• Making measurable performance improvements.
• Adding a layer of abstraction in order to support a new use case.
• Modularizing a particular object so that it can be reused in another part of the system

Adding new functionality or Extending existing functionality

This is where the boyscout rule comes into play. If you are in already in the code for another reason then you should clean up the code even if you didn't make the mess.

Making measurable performance improvements

This one is probably self explanatory but it's important to note that performance improvements will usually require some level of refactoring. 

Adding a layer of abstraction in order to support a new use case

This is an important one to understand. Often people will over generalize code at the beginning. This leads to overly complex designs and less readable code. If we follow the rule of not creating a layer of abstraction until we have at least two or three use cases for the code then there will come a point when you need to refactor the code in order to provide a layer of abstraction that doesn't already exist.

Until that second or third use case comes about the code should not be generalized. You don't have enough information about future uses of the code to get the abstraction correct. You may get lucky and guess at the future abstraction but you don't want to run your business on guesses and luck.

Modularizing a particular object so that it can be reused in another part of the system

Code reuse is one of the most important tenets of object oriented programming. When we identify code that is not specific to a particular object or package AND is needed in some other part of the system we should refactor this code into it's own module. Its important to ONLY do this when the code is actually needed in another part of the system.

Don't refactor code without tests

In order to refactor code safely you should have unit and integration tests for the existing functionality. I would also argue that you should write tests for the new functionality as well before you refactor. This will help you to understand the proper way to refactor the code as it helps you define how the refactored code should be used from a consumers standpoint.

If the tests don't exist for the the existing functionality you should write them first before you start refactoring. This helps ensure that you don't cause a new bug in the code or regress an old bug when refactoring.

Software Estimation: How to estimate software accurately part 3

In this series on software estimation I defined an accurate estimate as something that will:

• Give you an understanding of risk and unknowns.
• Quantify the known work.
• Be something that you can base a big decision on.
• Be refined as new information becomes available.
• Be a range with a 90% confidence interval.

In part 1 I explained how to account for risk and unknowns in your software estimation. In part 2 I explained how to quantify the work that is known. In the final post of this series I explain the last pieces needed to give an accurate software estimation.

Identifying Opportunity Cost

Let's say we estimate project X and determine that it will take 2 developers 2 months to complete. After estimating project Y we know that it will take 2 developers 1 month to complete (or 1 developer 2 months). Finally, when estimating project Z we determine that it will take 1 person 1 month. We now have a starting point from which to determine, based on cost to build and return on investment, if there is an opportunity loss associated with starting project X before projects Y or Z.

Refining As New Information Becomes Available

Most software estimates will start out as a range. Typically larger projects will have a wider range. This is because there are always things we know we don't know about the project as well as things that we don't know that we don't know.

It is easier to reduce ambiguity with information we know we don't know because we know where to start. As you practice software estimation you'll start to learn the correct questions to ask to identify the information you don't know you don't know.

Identifying new information should result in refining your previous estimate. The more ambiguity you are able to remove the more accurate your software estimate can be. The goal should be to continually narrow the range of the estimate.

More Accurate When Accompanied By A Confidence Interval

A confidence interval is a range of numbers from which we expect the real value to be contained. Having a good confidence interval allows us to determine the range of our estimate. This range can then be examined to validate our assumptions and to identify ambiguity.

For example, if I asked you to estimate how many marbles fit in a mason jar the actual number of marbles that would fit would be contained in your estimate. My 90% confidence interval may be 25 - 100 marbles. The actual number of marbles may be 87, in which case my interval contained the real value.

If you're interested in learning more about how to compute a confidence interval you should read How to Measure Anything by Douglas Hubbard.

Identify And Clarify Ambiguity

I'll wrap up this post with some questions you can ask to identify and clarify ambiguity in your software estimates.

• What assumptions does your estimate rely on?
• What are the risks associated with your estimate? Identify the things that could cause your estimate to be wrong.
• What external dependencies does your project have? For example OS updates, product launches, planned outtages, SDK updates, etc.
• What data do you need? Is it already available in a consumable format?
• What services are required? Do they exist? Do they talk the correct protocol?
• Are there any User Interface (UI) or User Experience (UX) dependencies that need to be resolved?

Software Estimation: How to estimate software accurately part 2

In this series on software estimation I defined an accurate estimate as something that will:

• Give you an understanding of risk and unknowns.
• Quantify the known work.
• Be something that you can base a big decision on.
• Be refined as new information becomes available.
• Be a range with a 90% confidence interval.

In part 1 I explained how to account for risk and unknowns in your software estimation. In this post I'll explain how to quantify the work that is known.

Quantifying The Known Work

All software has to be designed, architected, implemented, tested, debugged, and deployed. Each of these areas needs to be accounted for in the software estimate. Here are a set of questions that should be asked for each area to help you understand the level of effort associated with building your software.

Using these questions you can break out the known work into smaller buckets of work. You can then estimate each of these smaller pieces independently.

Design

Has the user experience and user interaction been defined? Do people interact directly via a user interface you provide or indirectly from another user interface? Have all the transitions and states been defined?

Will your software be available in multiple regions and/or locales? Does the design take regionalization and localization into account? Who provides the translations?

Architecture

What platform will the software run on? Are there platform specific requirements or constraints? Does hardware need to be purchased or provisioned?

What abstractions need to be defined? What is the hierarchy and/or composition of the abstractions?

Does the software rely on persistent data? How is it generated? How is it updated? What data specific workflows are there?

What metrics do you need to track?

How is the software going to scale? Are there single points of failure? Is the software going to be memory bound and/or CPU bound?

What is the software NOT going to do?

Versioning

Is your software released incrementally? How will the system handle data structure changes?

How are functionality updates handled in your system? Do updates need to be isolated from each other or can they co-exist. Is there a need to maintain backwards compatibility?

Implementation

What is the source control mechanism? Is it already setup? Does everyone have access? Is there a branching strategy?

Who is going to implement the software? Do they have all the hardware and software they need? Do you they know the platforms or technologies that your project uses? Do they require time to ramp up on the platform or tools? Is external training required?

Testing

Is your project going to use continuous integration? If not, why? Is your project going to use continuous deployment? If so, how do you deal with a failed deployment?

How are bugs going to be tracked?

How are the units of software defined? Are there tests for all units? A good software project has tests for each unit of work as well as tests for the integration of multiple units.

Are your tests automated? How often are the tests run and by whom?

Do you need black box testing, white box testing, or a combination of both for your software? Are you testing more than just the happy path or are you doing negative case testing as well? Does your software need stress or load tests?

Does your testing require specific hardware or software? Do these need to be procured?

Debugging

What tools are going to be used to debug the software? How are you going to profile performance?

What needs to be logged? How often and by what components? How are logs collected?

How are errors reported? What work is required to make errors in the system reproducible?

Deployment

How is the software going to be released? How is your software going to be monitored? How is your infrastructure going to be built out? Can you automate the infrastructure build out using tools like Chef or Puppet?

How is the software documented? How is the documentation updated?

What's Next?

In my next post I'll explain the last pieces needed to give an accurate software estimation.

Software Estimation: How to estimate software accurately part 1

In my last post Software Estimate: What you're doing wrong I explained why people tend to create inaccurate software estimates. In this post I'll outline what's necessary to create an accurate software estimate.

Previously I defined an accurate estimate as something that will:

• Give you an understanding of risk and unknowns.
• Quantify the known work.
• Be something that you can base a big decision on.
• Be refined as new information becomes available.
• Be a range with a 90% confidence interval.

Let's take a deeper dive into each of these areas with the goal of helping you to understand what information you need to gather to accurately estimate your software.

Identifying Risk and Unknowns

It is inevitable that there is risk associated with the software you're building. Risk in a software project is typically associated in one or more of the following areas:

• Blocking another project.
• Being blocked by another project.
• Requirements or scope that haven't been defined.
• Attrition.

If we understand the areas that pose risk to our project we can use that information to inform our estimate. Your project may need additional time to coordinate with another team. You may need to account for another projects schedule and/or risk.

Blocking another project

While blocking another project many not sound like something that needs to be taken into account when estimating software it can actually be the source of unexpected work. The other project may have requirements for your software that you're unaware of. You may have to make last minute changes to your software to account for these requirements. This is especially true if the other project is a higher priority project for the company than yours.

It's important from an estimation standpoint to understand if there are other projects that need to consume your software. If so, how well is the dependency between your project and theirs defined? Does the other project take direct dependencies on your software (for example by compiling against your library)? If you're providing a service how well is the service contract defined?

When you update your software what are the downstream effects on consumers of your software? Does your software require versioning?

Being blocked by another project

What other components, libraries, or services does your software depend on? Have they already been shipped? If so how do you deal with updates to the software? If the software hasn't already shipped, are their milestones known?

What happens if they can't deliver or aren't able to deliver in time for your project to meet it's goals? Do features in your software get removed? How long will it take to remove or update the affected components?

Requirements that haven't been defined

Blocking another project or being blocked by another project are not the only sources of undefined requirements. Additional requirements many come from your product owners due to under-defined user experiences or user interactions.

Are there mock ups of the user experience and user interaction? Have static web based mocks that people can touch and feel been built? Are all the transitions defined? What state does your application need or maintain?

Attrition

While attrition is often difficult to account for in the actual estimate, asking yourself some attrition related questions will help you identify work that should be accounted for in your estimate.

Does your software require a specialized skillset or tools? How long does it take to aquire the tools? What's the ramp up time to learn the skillset? What's the lead time that it takes to hire someone with the needed skills or platform knowledge?

What's Next?

In my next post I'll explain how to quantify the work that is known.

Software Estimation: What you're doing wrong

During the beginning of my career as a software development engineer one of things that I dreaded the most was the beginning of any project where I knew I was going to be asked to give an estimate for the software I was going to write. Often I thought "how can I estimate something I have no clue how I'm even going to build?"

Ultimately the dread I felt came down to this one simple fact. I had no clue how you could accurately estimate software. Over the years, as I've learned to estimate software development much more accurately, I've learned that what I thought of as software estimation was built on several assumptions that were incorrect.

Fallacies of Estimation

• Estimates require giving a specific date
• Estimates should be given on the spot
• Estimates can't change

Estimates Require Giving a Specific Date

This fallacy was probably the heart of the anxiety I felt about software estimation. I always assumed that an estimate was a fixed amount of time like 2 days or 1 month. It never occurred to me that an estimate could be a range. Fixed dates are rigid and inflexible and are easy to get wrong. Fixed dates cannot account for the fact that there are unknowns associated with the project. 

An accurate estimate will be a range. Ranges are more flexible and can help account for the unknown and any risks associated with the project.

Estimates should be given on the spot

This fallacy is based on the assumption that some big decision needs to be made right now and doesn't have time to wait for you to think about what unknowns you're dealing with. First off, if you're making a big decision on the spot based on a back of the hand estimate you shouldn't be in your role and shouldn't have the responsibility that you do. It's careless and dangerous to you, your product, and your customers to expect some engineer to accurately give you an estimate on the spot.

An accurate estimate will require you to do some research.

Estimates can't change

It may be true that the original estimate needs to be something that you can make a big decision or commitment based on. It's not true however that estimation stops once that decision is made or a project is put into motion. An estimate needs to be a living thing.

Estimates are about approximations. They're about dealing with ambiguity. As things become less ambiguous your estimate should be refined.

Where to go from here?

Now that we know the fallacies of software estimation we can define the characteristics of an accurate software estimate. An accurate estimate will:

• Give you an understanding of risk and unknowns.
• Quantify the known work.
• Be something that you can base a big decision on.
• Be refined as new information becomes available.
• Be a range with a 90% confidence interval.

In my next post I'll talk about the truths of software estimation and give you a set of resources to help you create accurate software estimations.

Transitioning to a professional software development role: part 3

In my first post in this series, Transitioning to a professional software development role: part 1, I started to outline some of the gaps I've seen in people's preparation for entering a career in the software development industry. I started off by focusing on what software development is not about.

In my second post in this series, Transitioning to a professional software development role: part 2, I took a look at what software development IS about. In the final post in this series I'd like to talk about the tools available that make us more efficient.

Being a good software developer means understanding how to apply agile

For a long time developing software was very much like developing a product on an assembly line. Assembly lines are very rigid and not well suited to respond to change. They run on the assumption that what happens upstream in the assembly line can be built upon and won't change. The moment change is introduced most of the product on the assembly line is ruined and must be thrown away.

Software's assembly line is called Waterfall. Overtime we've come to understand the downfall of waterfall and it's major flaw is that it's very rigid to change. Rigidity to change was okay when the primary delivery mechanism for software was the compact disk. But as software has grown to allow near real time delivery of features and functionality Waterfalls rigidity to change has become a hindrance to delivering high quality software in smaller but more frequent updates and features.

That's where Agile come in. Agile software development is about being able to respond to change in a rapid manner. It teaches us to think about software in a less monolithic manner but instead as a group of features that can be delivered in small chunks frequently over time.

I wrote a post several months ago called Software Craftsmanship: Project Workflow. If you're new to agile it's a good introduction to the anatomy of a project and what I've found useful. While the project workflow I've outlined isn't something you'll see in official Agile books, it is something that I have found extremely useful.

Being a good software developer means understanding how to use Lean

The concept of Lean Manufacturing was invented at Toyota. The primary goal was to reduce waste in the manufacturing cycle. This was done by re-thinking the manufacturing process to identify and remove waste. On example of waste could is parts sitting in a queue waiting to be processed. Toyota was able to show that by re-engineering their manufacturing process they could improve quality, efficiency, and overall satisfaction of customers.

The concepts behind Lean Manufacturing can also be applied to software development. Unfortunately these concepts often are applied incorrectly and have lead to many misconceptions and misunderstandings of Lean Software development. I wrote a post several months ago which outlined common misunderstandings in applying Lean to software development. 

As a professional software developer it's important to understand Lean and how to apply it to developing software.

Being a good software developer means understanding how to make trade-offs 

The last area I want to briefly cover is understanding how to make trade-offs. As a professional software developer you're going to be asked to make trade-offs all the time. Sometimes it will come in the form of quality (a bad trade-off IMO). Other times it will come in terms of features.

The key to understanding how to make trade-offs is learning to ask a few questions.

• What am I gaining by making this trade-off?
• What do I not get that I would gotten if the trade-off was not made?
• What downstream affects will this decision have on my long term strategy or road map?
• What additional work will be required later as a result of this trade-off?

The ultimate goal in software development is to provide business value in every part of the process. Understanding how to make trade-offs will help you provide the right business value at each step in the process.

The fallacy of the re-write

I've been in the software industry for a decade and a half and have worked on dozens of projects. Many of the systems that I have worked on were considered legacy systems. As with any system, but even more so with legacy systems, developers will get frustrated with the systems inflexibility. And inevitably this will lead to the developers decreeing that if they could only re-write the system all the problems will be solved. Unfortunately most product owners will eventually give in to these cries and will commission a re-write.

I'm here to tell you today (as both a developer and a manager) giving in to this urge IS NOT going to solve your problems. What it is going to do is grind your production to a halt and make your customers unhappy. This will have downstream effects on the team as the pressure to produce builds and builds and builds.

So why is a re-write not a viable solution?

Re-writes are usually based on a few commonly held (but false) beliefs in the software industry.

• If we start the project over from scratch we won't carry the problems from the old system into the new.
• If we start the project over from scratch we can use the latest and greatest technologies that are incompatible with our current technology stack.
• If we start the project over from scratch we can move faster and produce results quicker.

Why are these fallacies? If we dig a little deeper we will see that a ground up re-write means you are more likely to introduce problems in the new system than you are to solve problems in the old system. What is typically glossed over is the fact that the current architecture is doing a lot of stuff correct. How do I know this? Because it's the architecture that is in production right now running your business.

Let's take them at each of these fallacies one by one.

If we start the project over from scratch we won't carry the problems from the old system into the new.

This statement can really be broken down into two parts. The first part says that there are problems in the architecture that prevent you from extending the code and because you're now aware of those problems you can re-architect the software so that those problems no longer exist. The second part says that you won't carry over existing bugs into the new system. The second part of this statement is really related to the second fallacy, so we'll cover it when we cover that fallacy.

Because it is true that re-writing a system with knowledge of the current architectural problems can help you avoid current pain points most people are quick to accept this statement without challenge. There are many different times in the life-cycle of a product when problems arise. Some arise as bugs when writing the software. These can typically be rooted out with some sort of unit testing. The next class of problems crop up when integrating each of the pieces of the system together. You can create integration tests to help reduce the amount of integration bugs but often there are integration bugs that don't show up in pre-production environments. These tend to be caused by the dynamic nature of content. Because the new system is a re-write of the old system it will be more difficult to use real inputs/outputs from the old system to test the integration of the new system. Because of this you're likely to introduce problems in the new system that don't already exist in the old system. Because the new system won't be in production till it's done, these new architectural problems are not likely to be found till your new system is in production.

If we start the project over from scratch we can use the latest and greatest technologies that are incompatible with our current technology stack.

On the surface this statement is likely true. What this statement hides is similar to what's hidden in the previous statement. New technologies mean new bugs and new problems. Again it is likely that many of these problems won't surface till the new system is in production because, as anyone who has worked in the industry for at least a few years knows, production traffic is always different from simulated traffic. You run into different race conditions and bugs simply because of the random nature of production traffic.

If we start the project over from scratch we can move faster and produce results quicker.

The final fallacy is usually the one that most companies hang their hat on even if they acknowledge that a re-write from the ground up will introduce new bugs and problems and re-introduce existing bugs and problems. The reason is because they believe that their knowledge of the existing system should help them to only solve problems that need to be solved which leads to the system being built much faster.

The fallacy in this statement is more subtle but much more severe than the others. The reason is because until your new system performs all functions of your old system, the old system is superior from a business value perspective. In fact it isn't untill the new system has 100% feature parity with the old system that it starts to provide the same business value as the legacy system, not to mention more business value. Some will try to gain business value from the new system earlier by switching over to the new system before there is 100% feature parity with the old system. But by doing this you're offering your customers less value for the same amount of money, time, and/or investment.

This visual does a good job of illustrating the feature parity problem.

What is the solution then?

Are you saying I'm stuck with my current architecture and technology stack? NO! The best way to upgrade your technology stack is to do an in-place re-write. By doing this you help mitigate the problems presented in a ground up re-write. What does an in-place re-write look like?

By segregating and replacing parts of your architecture you're reducing the surface area of change. This allows you to have a well defined contract for both the input and output of the system as well as the workflow.

In-place re-write has another huge benefit over ground up re-write. It allows you to validate your new system in production as you would any new feature of the system. This allows you to find bugs sooner as well as validate the workflow and feature parity.

Another benefit of an in-place re-write is that you can decommission parts of the legacy system as you go without ever having to do a big (and scary) "flip of the switch" from the old system to the new system.

Most importantly, your customers do not suffer when you do an in-place re-write as you are not ever taking away features from your customers. Even better, you can prioritize giving your customers new features earlier by implementing them on the new system even before you've finished porting the entire old system over.

What's your measure of success?

In the software industry, or any industry for that matter, you're constantly making decisions as to where you're going to invest your time and resources. Maybe you're building a greenfield project and deciding where to start. Maybe you're 12 months into a project that isn't going as planned. Maybe your company or organization has pivoted it's business objectives and you're being asked to pivot with it.  Whatever the case, I've too often seen these decisions made without fully understanding what you're committing to and without having a clear plan to measure whether where you're investing your time and resources is actually being successful.

Measuring success isn't simply a matter of checking a box that something has been delivered. There's much more too it than that. You need to make sure you're delivering the right thing. You need to make sure that you're able to fail quickly if/when you discover that you're not on the right track. You need to have a clear definition of what's enough.

Here are five questions I like to ask before I invest in any project that allow me to have a clear understanding of what success looks like.

Do you have well defined user scenarios?

Well defined user scenarios allow you to know if you're building the correct thing or not. A well defined user scenario should clearly articulate what problem you're trying to solve, how you plan on solving it, and it should tease out some of the unknowns or external dependencies.

A great place to start with a user scenario is with a story board. Storyboards allow you to think about your product as it is used in the real world. Too often we start our user scenarios with an interface or UX design. These designs are great but they don't help us tease out the context surrounding our problem. For example, where users are when the problem occurs. Or what type of constraints the environment adds to the mix, like only having one hand available.

Can you show that you're providing business value now?

Does your new investment require that you wait until the product is complete before you can judge it's fitness? This should be a red flag that the structure and milestones of your project are not lined up correctly.

Agile using scrum and Kanban are two great ways to help you answer this question. Agile allows you to break things into smaller more iterable pieces. It gives you a way to measure your velocity as you go to help you adjust your milestones and scope. Kanban allows you to understand what work in progress is going on. It also allows you to understand and account for the unexpected work that inevitably comes down the pipe. Kanban allows you to make more informed resource decisions based on current resource allocations and throughput.

Is what you're investing in structured in a way as to allow you to change scope?

Do you have a way to show incremental progress? Do you have a way to see your work in action in the real world? If not, you should. Having a way to check-in and get a real world feel for how what your building works in the real world is crucial.

One way to do this is by setting up regular demo days. In agile, these tend to fall at the end of a sprint. It gives you a way to quantify what it is you're delivering and a framework from which trade offs can be discussed. I've seen pretty successful scrum teams decide what they're demo'ing at the next demo day BEFORE deciding what stories are in their sprint.

Having demo days allows you to break up larger work into smaller more consumable pieces. It also allows you to identify dependencies earlier in the process. Maybe your demo requires some work from another team? Maybe it requires some additional hardware or resources. Maybe it requires a deployment. Whatever the case, you're able to start asking the correct questions and unblocking your team before any work actually begins.

Are you able to measure success incrementally?

Demo days are one way to measure success incrementally but not the only way. Alpha and beta releases are another way. Getting something in-front of real world users (even just internally) will open your work up to a higher level of scrutiny which will inevitably make your product better. It will also force you to ask yourself the correct questions on how you measure success for individual features. For example, if you're adding a button to make peoples lives easier, how do you know if they're using it? How do you know if it was discoverable? How do you know if it actually solved the problem? Knowing the answers to this class of questions is paramount to the success of your project. Asking that class of questions helps inform the approach you take to solving your problems.

Do you understand the unknowns?

As you start asking the other questions it will become clear that there are always unknowns in your project that need to be fleshed out. It may be understanding the cost associated with the project and how it's going to be funded. It may be intra-team, inter-team, external, or 3rd party dependencies that are not apparent at the inception of the project. It may be that the level of effort is greater than the rewards for building any particular piece of your project.

Whatever the unknown, it should be your goal to root it out. You're process should make finding the unknowns easier than allowing them to crop up at just the wrong time.

Post Mortems

In my previous post on Continuous Delivery I mentioned using postmortems as a way to share responsibility and create action items to address and prevent failures as part of a change. I thought it would be helpful if I took the time to explain how I like to run postmortem meetings.

To me a good postmortem meeting has five distinct parts. Each part has a goal that's measurable and useful in understanding a failure and preventing that failure from being exhibited in the system again. It's important to go through these parts sequentially and in the order defined below.

Identify what happened

This may seem like common sense, but a good postmortem should always start with a chronological list of the sequence of events. These events should focus on the WHAT and not the WHY or the HOW. The purpose of this part is to get everyone on the same page as to what the sequence of events were that lead up to the incident, happened during the incident, and lead up to the resolution of the incident.

Identify what went wrong during the incident 

After the sequence of events has been identified and everyone understands the WHAT of the incident, it's important to understand what went wrong during those sequence of events. It's extremely important to try to keep this factual. If you can't back it up with data, don't include it. Again, that may sound like common sense but it's very easy to get into a finger pointing argument or start blaming other folks or groups during this part.

It's important to call out things like coordination failures, escalation issues, trouble shooting guide issues, or anything else that impeded resolution of the incident. The main goal of this is to take a constructive look at how the existing processes worked and identify holes or gaps in the process.

Identify what went well

After you've identified what went wrong during the incident it's also very important to identify what went well. This is important from more than just a moral perspective. It's important for the group to come to consensus and understand things that accelerated resolving the incident so that they can be included and continued in future incidents. It's also a good way to recognize the efforts of team members or other product groups that put skin in the game during an incident.

During this part it's important to call out things like issues that were identified by an automated system or that were recognized early by some standard operating procedure. It's also important to call out good coordination between teams, any heroic individual or team efforts, as well as any new mitigation strategies that were created or discovered during the incident.

Bring up what issues have been identified as a result of the incident

The main goal of this part is to analyze the data presented in part one in light of what went well or poorly. During this part you want to clearly identify gaps that need to be filled as a result of this incident. This is not the time to bring up HOW to resolve the issues but simply to identify the issues.

During this part it's important to call out gaps in communication or process around incident status notifications. If holes have been found in the trouble shooting guide this is the right time to bring them up. Other things like team members having incorrect permissions or access problems should come up during this part as should lack of training or improper knowledge of third party (or first party) systems.

Identify actionable improvements

By this point you should have a list of things that can be improved. This list may include process improvements, documentation improvements, or software improvements. It's important during this phase to make those improvements actionable. The goal of this part is to walk out of the postmortem with not just a list of what actions need to be taken, but also who is going to take the action. It's very important that one of the goals of the postmortem is that there is a clear understanding of what changes need to be made and accountability for getting those changes implemented.

Continuous Delivery

Continuous Delivery is an important topic in the software world, and particularly the DevOps movement. A lot of the talk around continuous delivery centers around the technical side of achieving it. But there's another aspect that needs to be considered first before it can truly be achieved. That aspect is the cultural changes that need to happen within an organization in order to truly achieve continuous delivery. These cultural changes are not a pre-requisite for continuous delivery, but you should be aware that they do accompany it.

The follow are the five areas that I believe are required for any organization to be successful at continuous delivery/deployment. Most organizations won't be starting at ground zero but being aware of the cultural change that will come can help accelerate a move to and adoption of continuous delivery.

Cross Discipline Teams

Development and operations need to work together. Ideally, they're both represented on the same team. One of the main goals of continuous delivery is to identify defects earlier in the pipeline and to provide business value as early in the pipeline as possible. The best way to achieve this is to think about delivery from the beginning. The project team should be planning the deployment even before the code is written that they'll be deploying.

Raise visibility

For every change the project team should understand how the change can be measured. Here are a few simple questions that you can ask as part of the dev cycle:

• What metrics are associated with this change and how?
• What are the risks?
• Is the performance profile expected to change?
• How do we measure success for this change?
• What are the signs that the change is not effective?

Build trust

Operations and development need to know that enabling the business is the #1 priority for BOTH units. No change should make it's way into the deployment pipeline unless it provides business value. If both units understand this then we have a shared understanding of the importance of each change and a shared desire to get that change out.

In order to build this trust the operations folks need to know and believe that the development folks will include them in feature discussions. Having operational discussions early and often in the conversation will help ensure successful deployment of the change.

On the flip side the development folks need to know that the operation folks are including them infrastructure discussions early and often as well. Again ideally, these folks work on the same team and these discussions are part of the normal standup discussions.

Minimize risks 

We need to understand that failure happens. It's how you deal with failure that leads to success or failure for the change. It's naive to believe that failure can be prevented 100% of the time.  Because we know it can't our goal is to minimize the risk of any change should it fail.

In order to minimize the risk to the overall system for any given change should focus on reducing the overall surface area of the change. The smaller the change the less risk there is of a failure associated with that change being catastrophic. The longer the time between releases the larger the risk of that change. The more code that's released with every change the larger the risk of failure or bugs.

In order to respond to change effectively we need to build a culture that responds to failure well instead of trying to prevent it. The easiest way to respond to failure well is to practice it. Build failure tolerance into your system. Understand how your system responds to failure and identify the expected and appropriate user experience when failure happens.

Shared responsibility

We all succeed or fail together. It's not possible for operations to succeed and development to fail (or vice versa) and have the product succeed. Both disciplines need to succeed in order for the business to truly succeed.

Once we've grasped the concept that we succeed or fail together we can create a culture where it's okay to talk about and address our mistakes. We try to avoid finger pointing and are able to do this because people own up to their mistakes and take responsibility for them. They participate in postmortems where they create action items to address the failure and prevent failure of that kind in the future.

Having shared responsibility also means that we treat rollback as a last resort. When a problem arises with a particular change we stop the line. We work together to figure out the problem when it happens and we work together to fix the problem before we move on to making additional changes in the system.

Sharing responsibility means that both the operations folks and the development folks should be on-call for incidents. Nothing provides better incentive to fix a problem than getting woken up in the middle of the night or interrupted on a Saturday.

Coding Standards Revisited: My Language Agnostic Coding Standards

In my previous post, Coding Standards Revisited: Writing Code That Lasts, I talked about how to approach a code review from a slightly different perspective. Today I'd like to talk about approaching coding standards from a slightly different perspective.

As I mentioned in my previous post I find a lot of value in framework, language, or platform specific coding standards. But I do not believe they tell the whole story. Here's a short list of non-traditional coding standards that I believe apply to all code on all (modern) frameworks, languages, and platforms.

Naming


Use meaningful variable, method, parameter, and class names. DO NOT use shorthand. A person that does not know how to read code should be able to tell you what the variable, method, parameter, or class is doing just from the name.

Separate out your concerns

A method or class should have one reason to change and ONLY one reason to change. If a particular method in that class has more than 10 - 20 lines it's probably doing too much. There are very few exceptions to this. Methods and classes should be distinct features that overlap in functionality as little as possible.

Cohesiveness

Write highly cohesive classes. The methods in a class need to have a lot in common. The methods in a class should act on the same data. The methods in a class should all be related to the data. I.e. an image manipulation class that has code to physically save the image to disk is not cohesive… it's coupled. There should be a class that deals with saving things to disk and a class that deals with image manipulation.

Dependency Management

Dependencies should be added by reference as much as possible and injected into the classes that depend on them. This allows the dependencies to be created at the correct level of abstraction. 

High-level modules should not depend on low-level modules. Both should depend on abstractions. Abstractions should not depend upon details. Details should depend upon abstractions.

Don't Start By Abstracting

Only abstract as you need too. The first (and I would argue second) implementation of something should come in the form of concrete classes. Only when you run into a scenario where you need to interchange these classes should you then abstract it.

Code Duplication

Follow the Don't Repeat Yourself (DRY) principle. Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

Commented Out Code

Don't comment code out and leave it. Delete it as you can always go back to it using source control.

Only Write Enough Code To Satisfy What You Need Now

Don't write a single line of code for something in the future (i.e. I'm gonna need a class that….). Only write code that is actually used right now for what you're doing.  Usually for me, this means starting with a test class. In the case of a UI I usually start by writing static UI Code and then working backwards making the code dynamic as I go and ensuring that each piece of dynamic code as a set of tests associated with it.

Coding Standards Revisited: Writing Code That Lasts

Over the last several weeks I've been writing a series of posts on Software Craftsmanship. While I don't think I'm completely finished with the topic (nor do I ever hope to be) I thought I would take the opportunity to switch gears and talk about coding standards.

Typically when people think about coding standards they may think of Google's Java Style Guide, GNU's C++ Coding Conventions, MSDN's C# Coding Conventions, Apple's Objective-C Conventions, Android's Style Guidelines for Contributors, or so on. There's practical guidance out there for most languages and frameworks. These conventions provide you with several advantages. They will help you write code that other developers in that language or on that platform will recognize and therefore be able to maintain easier. They help you code for compiler optimization or run-time optimization. And they help you navigate some of the trickier elements of the language. What these guides don't generally do is give you a good framework of how to write code that will last.

My Software Craftsmanship series was a good start down this path but now let's dive a little deeper and see if we can't identify some additional practicals tools that you can use to help you design and write code that will last. Ultimately that should be out goal; to write code that lasts. Code that lasts may not be elegant and it may have flaws but ultimately it has proven useful. So useful that it has stuck around. Sometimes you find code that lasts just because others are afraid to touch it for fear that it will break. I would argue that, while that code may have bad style or convention, it has lasted because it did it's most important task well; it did the one thing it was written for and has proven useful for that task.

Traditionally when we evaluate code fitness we look at how well the developer adhere'd to a certain style guideline or how well they adhere to a language or platforms best practices. We often look for places where patterns have emerged in the code and try to optimize our implementation around this pattern. We look for poor memory utilization or inefficiencies in dealing with large data sets. These are are great things that MUST be part of a code review. But these things alone don't tell the full story of whether or not the code will last.

So here are a few tips from me on what I like to also look for during a code review that I think identifies a pattern of code that will last.

What would a standard ______ expect?

Fill in that blank above with whatever framework, language, or platform the code was written for. So often it's the case that we let other framework, language, or platform coding standards creep into unrelated code. A standard developer for that framework, language, or platform should be able to open the code and see style that their familiar with. They should recognize patterns that are specific to that framework, language, or platform. The code should flow in a way that's optimized for the way that framework, language, or platform expects to run the code.

A few ways I often see this brake down are:

• Adding a dependency on a third party tool that auto-manages code style (like ReSharper)
• Adding a dependency on a particular IDE (like Eclipse) when the framework, language, or platform is IDE agnostic (like C++ or Java)
• Not using an IDE built for the framework, language, or platform. I.e. Xcode for iOS or Visual Studio for Microsoft .NET (though I would argue NOT for Mono)
• Curly braces, tabs, and spaces. Let any Java, C#, or Ruby developer read the others code and you'll hear be able to hear the complaints from down the hall.
• Programmatically defining UI elements in code when the framework or platform provides built in mechanisms to define such elements.

Often these dependencies are added with good intentions. They're added because the organization wants to increase productivity. Or they're added because the organization believes that conformance to a style is important for the cohesiveness of the code, which it is. Conformance to a style other than what a standard developer of that framework, language, or platform would expect is not cohesive. Cohesiveness should be determined both by how self cohesive it is as well as how cohesive it is within the ecosystem it is built for.

What external dependencies has this code been coupled to? Are they necessary?

Developers become very fond of particular pieces of code or particular third party libraries. We're taught in school and at work that we need to focus on re-use. Often this is misinterpreted to only mean reference other code. But occasionally, or often, it's the case that in order to reuse existing code without adding unnecessary dependencies we need to refactor the code we're referencing into a library or module. This allows us to manage the dependencies of our new code correctly. Often we reference the correct code but add dependencies on other packages or classes that are unnecessary because we fail to refactor the dependencies as their consumption changes.

When this code changes what affect will that have on other non-related code?

This is a smell that you've either got something messed up with your class encapsulation or code organization. You should expect direct consumers of your code to change as your interfaces change. But you should not expect transporters of your code to have to change. One way to test this is to simple change a public method signature and compile. See what will no longer compile. Are all the places you have to change reasonable?

How discoverable are the features of this code? Do I have to read the code to understand what it does?

This one is one of the more subjective items on the list. But I still think they're questions that absolutely must be asked. Your first pass during a code review should be to look at how the code is used. If during this pass you find yourself asking things like "why do you need to pass that?" or "why is this call necessary?" then you're probably looking at code that could use better method naming or is written with the wrong level of encapsulation.