Development teams should be deploying and mainintaining production

Most of the software development shops I've worked at in my career didn't have developers deploy their own code. Typically the developers would create a release and hand it off to QA who would do some combination of manual, automated, and performance testing. When bugs or performance issues were found they'd send the release back upstream to developers who were already working on the next feature.

This creates a culture of fear that the production system needed to be protected from the developers. This is unhealthy in my opinion. Software development teams should be deploying their own software for four main reasons:

1. Ownership
2. Accountability
3. Insight
4. Continuous Integration/Continuous Deployment

Ownership

Software teams that deploy and maintain their own software in production tend to have a higher level of ownership. This is because they are involved in the full end-to-end software development life-cycle rather than just building the software. They become better aware of the machines (whether physical or virtual) that their software runs on. They're more likely to insist and push for deployment automation as developers tend to hate repetitive manual tasks.

When the developer isn't responsible for deploying their code they move on to the next problem space after they've "shipped" their code to operations. This typically means they've moved on both mentally and emotionally from that release. When a developer is responsible for pushing to production they're invested in the release getting all the way to customers. They become responsible for diving in and solving problems that pop up during deployment.

Accountability

When you're responsible for maintaining software in production, you're more likely to be accountable to what happens in production. Specifically, the dev team becomes more accountable to:

• Bugs
• Test coverage
• Deployment failures
• Poor performing code

When you work at scale there are often bugs that only show up in production. When developers deploy to production they become more accountable for fixing bugs, performance issues, and deployment failures as they are directly exposed to them during the release cycle. One thing I've seen successful organizations do is, when a release fails, block all further releases until the deployment succeeds. Developers that are accountable for these failures will be encouraged to fix them because they're prevented from moving on to the next new thing until the current thing is successfully deployed. 

Insight

Understanding the health of your system means having the right metrics and validations in place. When software developers own deploying to production they're more likely to instrument their code well because they will need to validate that their deployments are successful. In my experience teams that deploy their own code better understand the performance characteristics and health of their systems. They are also better able to diagnose problems because they're familiar with the differences between their development and test environments and the production system.

Continuous Integration/Continuous Deployment

When a team owns their code end-to-end they are setup for success with respect to continuous integration and continuous deployment. They don't have to jump through hoops in getting their code into production and they can integrate deployment and test tools directly into their development pipeline. This encourages them to ship smaller features more often. By doing this, they decrease the time it takes to close the feedback loop and are better able to adapt to their customers ever changing needs.

Why you should learn multiple programming languages and platforms

In the first decade of my career I wrote software in C++, Java, C#.Net, Ruby, Python, Objective-C, VB 6 and VB.NET. Additionally I developed for ASP, ASP.Net, and PHP based Web Sites as well as scripted in Powershell and Bash. Learning multiple programming languages was very helpful for me during my career as a software engineer. 

Here are some reasons that you should learn multiple programming languages.

Trains you to separate the engineering from the language

Learning multiple languages teaches you to explore design patterns and engineering best practices that aren't platform or language specific. Being a great software engineer is about being able to identify the correct algorithms to solve your problem and being able to implement that algorithm in the language and platform that best solves the problem. Great engineers are able to build simple software and use design patterns that are language agnostic.

Enables you to learn other languages more easily

Once you've started to learn to identify design patterns learning a new language becomes less steep of a learning curve. Learning a new language is about learning the plumbing of the language, i.e the syntax, the libraries, and the run-time. Knowing what you *should* be building allows you to learn *how* to build it.

Exposure to different tool-sets and platform features

As you learn new languages and platforms you'll also be exposed to new tool-sets for building, debugging and testing your software. Using these different tools will help you to learn different aspects of interfacing with hardware and other software as the languages and platforms will likely have different layers of abstraction for different things. For example with C++ you'll learn better memory management. Whereas with Ruby on Rails you'll learn dependency management with tools like Bundler. 

Become better at picking the right tool for the job

Learning the ins and outs of different languages and platforms will help you learn to pick the right tool for the job. You'll understand what languages excel at what types of problems. For example, you may need to write a script to parse files and find that perl's built in regular expression capabilities  help you to write an efficient and simple script. You may want to build a cross platform game and decide that using C++ will help you port your game to multiple platforms without as much code duplication. 

How fear is influencing the software development industry

The software industry is largely made up of highly intelligent and highly analytical individuals. It's an industry where experience, ideas, and intuition help set people apart. We celebrate innovation, creativity, simplification, reuse, use of patterns and avoidance of anti-patterns but if not careful we can encourage a culture of fear.

Being Afraid To Be Wrong

Being wrong is often viewed as not being good at what you do. In an industry where your intellect and your intuition are your primary tools to be successful, being wrong can make you feel as though you're not as intelligent or as capable as your peers.

But this is a fallacy. The most intelligent and most intuitive people I've worked with in the industry are able to differentiate and admit when they're wrong. Being able to recognize when you're wrong and pivot to the right path is a skill that sets the highly capable apart. It's a sign that the person is truly looking for the best solution, rather than the best solution they can create.

Being Afraid To Try Something New

The software industry is constantly changing. This can be unsettling to people who feel like they're constantly trying to keep up with the latest and greatest in technologies, frameworks, and platforms. It's often the case that the latest and greatest is really just a passing fad. Getting caught up in one of these fads can have negative impact on your team or your project depending on how much you've embraced the technology.

New technologies and new ideas always come with risks. You risk adoption being so low that the tech will fail, security vulnerabilities, scalability problems, and a whole host of problems that can come with new tech or new ideas.

The problem with over-indexing on the risks is that you'll miss out on the game changing technologies and ideas. You miss out on things like Agile development, CAP theory in distributed computing, Linux, Node.js, native Mobile apps, and much much more.

Being afraid to try something new means you run the risk of being left behind and becoming irrelevant.

Being Afraid To Fail

Failure is a very important part of progress. Being afraid to fail is tantamount to being afraid to make progress. Often, people are afraid to fail because they believe that if they do they're not going to have another chance to succeed.

The real problem isn't failure, it's the scale of the failure. Waiting too long to validate your assumptions is a recipe for disaster. The key to failing well is to limit failures scope and to learn from it. If you can create an environment that leaves room for failure, you're creating an environment that can be successful.

Don't be afraid to fail, be afraid to not be able to fail.

Preparing for a software development interview

Software Engineering interviews can be stressful. They're typically 4 - 5 hours long meeting with 4 - 6 different people.  Being prepared can be the difference between getting that job you've always wanted or failing to truly demonstrate your capabilities and skill-set. Here are some tips for preparing for software development interviews.

Make Sure You Understand The Problem, Ask Clarifying Questions

One of the bigger problems I often see in interviews is the candidate just jumping in to solve the problem without making sure they understand what they're being asked to do. Even if you feel like you understand the problem 100%, you should ALWAYS repeat back what you think the problem is before jumping in.

Every problem comes with some level of ambiguity. As you repeat back the problem think about how to remove that ambiguity. Ask clarifying questions that help you understand what is in scope and out of scope for the problem.

Review Your Data Structures

One of the main things that you need to be able to demonstrate in any software engineering interview is how well you 1) know your data structures 2) know when to use certain data structures and 3) understand the systematic complexity for each one (e.x. how long do reads or writes take). Before every interview you should review the following data structures and understand how to apply them to solve real world problems.

1. Array
2. Linked List
3. Hash Table
4. Binary Tree
5. Queue

Give An Overview Of How You're Going To Solve The Problem 

To be successful as a professional software engineer you need to be able to communicate well. It's not enough to simply be able to code. Because you're going to work on a team and with people that aren't engineers (product development, program management, design, and etc) being able to communicate effectively is an essential part of the job.

Being able to talk through your solution helps you demonstrate that you're able to communicate effectively. Additionally, talking out loud about a solution often helps you identify gaps or potential bottlenecks before you get into your code. Essentially, it gives you a chance to validate your design and show the interviewer that you can approach a problem systematically.

Focus on Solving The Problem First Then Optimize

Another common mistake I see interviewees make is getting so focused on giving the most optimal solution that they never finish creating a valid solution. You're not likely to get an offer if you can't complete the exercise within the time allotted.

Your goal should be to create a working solution first. At that point, if you have time, you can optimize it. Along the way you can point out potential bottlenecks or un-optimized code.

Keep Things Simple

Simplicity in software is a gift. Complex software typically has problems with readability, understandability, and scale. You should focus on demonstrating that you understand how to keep things simple when solving your problem.

Some easy ways to do this are:

1. Limiting the number of classes/interfaces you create.
2. Use meaningful variable names.
3. Encapsulate the code into short simple methods, rather than one giant method.
4. Use built in language constructs rather than re-inventing the wheel.

Know How To Test Your Solutions

Professional software engineers test their own software. You need to be prepared to identify:

1. Test cases when the code should succeed
2. Test cases when the code should fail
3. Test cases that handle typical edge cases like NULL values, being passed 0, 1, or many objects, no connectivity, and etc.

Problematic Software Development Interviews

Interviewing in the tech industry is hard. Both for the interviewer and the interviewee. As an interviewer you've got 45 minutes to 1 hour to determine if someone has the right technical prowess and is a good fit for the company and for your team. As an interviewee you've got 45 minutes to an hour to show off what you know, how well you adapt to change, and how you respond when you don't know something.

In my 16+ year career I've been on my share of bad interview loops. The reason they were bad wasn't because of anything inherent about the person conducting the interview. They were bad because the person conducting the interview didn't know how to flush out what they needed to know about the candidate because they were asking the wrong questions or questions with the wrong focus. Here are four ways I've seen interview questions go down the wrong track. 

The Problem Requires More Than 20 Lines of Code

Writing code takes time. But even more so, solving the problem takes time. You need 3-5 minutes to go over the problem. The candidate then will likely 5 or so minutes to ask clarifying questions and make sure they understand the problem. Then there's another 5 or so minutes of them thinking through the problem and coming up with a solution that they can code.

Ideally, the candidate should be able to write the code for the solution in around 10 minutes. That's 30 seconds per line at 20 lines of code. If your problem requires more than 20 lines of code to solve, you're going to miss out on talking through their code with them. You'll also miss out on being able to scale your question to see how adaptable their code is to change.

The Problem Is Niche And Took You More Than An Hour To Solve

It's very rare that your problem space is so niche that you only want a candidate that has an EXACT set of knowledge in your problem domain. More likely, it's the case that you want to determine if the person knows her/his fundamentals well enough to be able to quickly adapt and learn a new problem space. Because of that, you should avoid niche questions or questions that require you to have super in depth knowledge about the problem space.

A better way to determine if the person is able to adapt and learn is to give them a problem to solve that requires them to critically think through an issue. Once they've satisfactorily solved that problem, change the parameters. You can do this by asking things like "how could you do this with less hosts" or "how could you solve the problem with half the memory usage" or something else that changes an external factor, an input, or an output.

Responding to change is what we do day in and day out as Software Engineers. It's more important to see how well they know their fundamentals and respond to change than it is to determine how deeply they already know your problem space.

The Problem Has Nothing To Do With The Problem Space The Job Is For

Unfortunately, too many times I've seen people ask questions that were not anywhere near the problem space that the job was in. I've seen front end web or mobile developers struggle trying to solve binary tree or problems involving Dijkstra's algorithm when the real problem domain is working with object life-cycle, state management, hardware constraints, and etc.

Make sure that your interview questions are able to tease out whether they will be able to be successful in your problem space. Not whether they know some arbitrary algorithm that isn't applicable or won't help them be successful in your problem space.

The Problem Determines How Clever They Are, Not What Their Skill-set Is

Trick questions, riddles, and brain teasers are not good interview questions. They help determine how clever a person is but not how likely that person is to succeed in your position. There was a time in the early 2000's when you couldn't attend a software development interview loop without being asked a question like "why is a man hole round" or some other riddle.

As an interviewer you want to determine if the person knows the fundamentals of software engineering, can think critically, can adapt to change, and fits in with the culture of your company or team. Your goal is NOT to determine of the person is clever.

Managing your software development career

One of the things I wish I had done better as a Software Engineer was manage my career. I was in the role of software engineer for over a decade but it wasn't until the last 5 years of that portion of my career that I started to take control. For the first 6 or 7 years of my career I naively believed that if I just did a great job my career would advance. I trusted that my management team and senior leaders would recognize my achievements and I would make my way up the leadership ladder.

While there's nothing wrong with having that much trust in your leadership team, in my opinion, it isn't the right way to manage your career because you have very little ownership of your success or failure. You're career should be a reflection of your capabilities and your desire to succeed. Not everyone wants to move up and not everyone is capable of moving up, but most are.

Here are some tips to help you understand how to navigate your software development career.

 

Start the conversation with your manager

No matter how good your manager is, you have to assume that he or she is unaware of your desire to make it to the next level. Starting the conversation with your manager helps them understand that you want to grow as an engineer. 

One important thing to be aware of is that when you have that conversation you're going to need to be prepared to hear about both the good and the bad about where you're currently at. You need to approach that conversation objectively and with a desire to close the gaps. Getting to the next level in your career is mostly about closing the gaps between where you're at and where a person at the next level already is.

Understand the responsibilities

No one is ready to move on to the next level until they're already over performing at their current level and have begun to perform the duties and responsibilities of the next level. Because of that, understanding the responsibilities of your current level and that of the next level is key to growing and achieving your goals. In particular you should understand:

1. What the expected technical capabilities should be
2. What the volume of code that you should be producing is
3. What the expected level of autonomy is
4. What the scope of influence should be. I.e. what influence you should have on your team, across teams, and within your org
5. What your role should be with respect to your roadmap and the teams day to day activities

Create an achievements doc

If you're anything like me you're not going to remember the important details about your everyday achievements. What I like to do is to keep an achievements doc where I can keep track of the details of my achievements. I don't write in it everyday but I try to re-visit it once a week and make sure that I put my big achievement of the week into it. I like to keep track of what I deliver, who I influenced, and other big decisions I made or contributed to.

I try to keep track of the following information for each achievement:

• What the problem was
• What the proposed solution was
• What the alternatives were i.e. what were the trade-offs
• Why the solution chosen was the correct one
• What the impact the solution had on the business, the team, operations, and/or efficiency

That last point is important. You're going to need a way to measure each of your achievements. Having data to back up what you do is the best way to have an objective conversation with your management.

 

Expand your influence

One thing common to every role is that the more senior you get the more influence you should be having. As a junior engineer you're sphere of influence is largely limited to yourself and your work. As you grow and develop you're sphere of influence expands from just yourself to your team. The more senior engineer you become your influence expands outside of just your team to your org. Expanding your influence means identifying what the sphere of influence should be at your current level and what it should be at the next level.

Get a mentor

As a growing engineer you need an outside influence. You need an objective voice to help you see both your strengths and your weaknesses. A mentor is someone that can help you see the obstacles to your career growth. They can help you grow your existing skills and identify new skills that you need to gain.

Create goals

The last step in career advancement is to create a set of goals. These goals should represent a mix of things it takes to be successful in your current role as well as things it takes to show that you're already working at the next level. Your goals should be SMART (specific, measurable, achievable, relevant, and time bound) and should be something that you review directly with your manager periodically (at least once a quarter). Goals are your best way to make sure you're on track for career advancement. 

Successfully Shipping Software

It's unfortunate how much software gets written and never actually ships and is used. This can't be wholly avoided because there are times when priorities do change. There are times when projects do need to get shelved due to more pressing needs. And if you're working at a company that values risk taking then projects will get canceled in an effort to fail fast. 

As a software engineer not shipping due to changing priorities or having to shelve something in favor of a more pressing project were two of the most demoralizing things. Especially when I had worked on a project for N weeks or (even worse) N months.

Being able to ship software fast is one of the big benefits of Agile software development. Agile encourages you to ship at the end of every sprint. In the ideal world this would mean putting something in users hands that can be used at the end of every sprint. In reality, this is usually not the case. Often, you ship a series of smaller end-to-end pieces into production which at some point you'll tell your users about.

Here are some things to consider which will increase your likelihood of shipping software your users will use and decrease the chance that you'll build the wrong thing.

Understand your problem

While this sounds like a no-brainer, it's actually one of the things that I see get missed the most. Understanding your problem is not about requirements gathering. You gather requirements about a known problem space. One reason software projects can fail and never ship is because the problem was based on an idea that was never validated. You should always be able to tie your problem back to something a user has asked for.

Identify your milestones

Many software teams identify milestones from the beginning and move forward linearly in time till shipping. This is not a pit of success in that it's easier to miss a requirement than it is to not miss one. It's better to start with the last milestone and ask the question "what needs to happen right before this to achieve this milestone".  Then you keeping going backwards in time asking that question and defining new milestones. Eventually you'll get to a requirements gathering milestone. You'll have achieved either defining all your milestones, or identifying ambiguity that would have likely derailed or delayed your project had you started defining milestones the other way around. 

Know your risks and how they can be mitigated

Every project comes with risk. Being able to identify what risks are associated with your project will help you determine how your project will succeed in the event that one or more of your risks are realized. Some common risks to software projects include:

• Upstream/downstream software dependencies (often internal software being developed by another team) being late or failing to ship all together.
• Hardware/software purchases that need to be made.
• Making sure you're able to deploy to production.
• Not having user documentation.
• Not having a marketing plan for a new feature or product.
• Getting started with performance testing or security reviews late (or not realizing you need to do them at all).

Make sure you know what you need to measure

Milestones are great to help you know whether your project is on track. But they won't help you once your software is released. You need to identify the ways that you should measure your software to understand it's health. You should understand both your customer facing metrics (like the number of failures, the latency of requests, or etc) as well as your internal metrics (cost, performance, system health, etc). Talk to your product owners and your sales teams to understand what they need to measure about the software in order to aid them in their jobs.

Have an exit criteria for success

What is your definition of done? This doesn't mean you won't maintain or make changes. It just means that you've successfully delivered what you wanted to deliver. This should be a checklist of items that's been validated on both the business and technical sides of the house.

Writing code that will last: Identifying Dependencies

In my previous post, writing code that will last: understanding bottlenecks, I talked about how bottlenecks can come in the form of vertical scalability, horizontal scalability, or statefulness. In this post I'll explore how identifying upstream and downstream dependencies in your software can make or break it from a longevity perspective.

Upstream Dependencies

There are several upstream dependencies that you need to consider in your code. These come in the form of your operating system, the platform your application or service runs on, the external applications and services you use, and the contracts you depend on for data.

Operating systems and software platforms can affect the performance, security, and ability of your service. Writing code that lasts means abstracting your code enough from the OS and underlying platform such that you can upgrade the OS or platform without changes to your application or service. This allows you to protect your self and your customers from security vulnerabilities as well as allows you to take advantage of performance enhancements that come from kernel or library updates.

You should also consider the dependencies you have an external applications or services. Do you depend on them in a version specific way? How will your application need to change if they change the schema of the contract you have or deprecate an API you use. Building code that will last doesn't mean prematurely optimizing your code to try to defend against these changes, but instead means understanding how to encapsulate them such that the effect these changes have on your code are minimal.

Downstream Dependencies

Downstream dependencies are people, applications, and services that depend on your software. From a downstream dependency perspective writing code that will last means understanding when you're making a choice that you can't go back and change after the fact. In order to write code that lasts we have to minimize the number of these choices that we make. These choices often come in the form of the APIs we expose, the contracts we create with other services, the way we represent objects, and etc. Once these choices are made, sometimes they can't be taken back without having some sort of negative side effect.

Writing code that will last: Understanding Bottlenecks

In my previous post, writing code that will last: avoiding potential roadblocks, I talked about how roadblocks can come in the form of cost, re-inventing the wheel, or under-defined requirements. In this post I'll explore how understanding potential bottlenecks in your software can make or break it from a longevity perspective.

Understanding Bottlenecks

The first bottleneck you should be considering is how your application scales vertically. Vertical scale typically has direct correlation with your applications performance characteristics. Vertical scale is achieved by increasing CPU, Memory, Network Capacity, or Disk your application has available to it. Is your application going to be CPU or Memory bound? Will you require a lot of Disk I/O? How much data will you be reading/writing over the network? You need to understand each of these areas in order to know how your application will scale vertically.

How your application scales horizontally is the next potential bottleneck you should understand. Horizontal scaling is the ability to add additional machines running your software into an existing pool to add additional computing. Horizontal scaling allows you to increasing the throughput of your application by having more capacity to process requests or jobs that the application performs. Your software needs to take this into account from the beginning or it won't last without requiring rework.

Another area to consider when understanding your applications bottlenecks is how stateful it is. Does your application require some external state in order to run? How the application gets and maintains this state can lead to potential bottlenecks. E.g maybe your application needs some specific information about the user making the request that it has to lookup somewhere. Or maybe it needs information about previous requests. An example of a stateful application is one that uses a shared database. The reason state cases bottlenecks is because your application is now dependent on something external that it does not control.

Writing code that will last: Avoiding Potential Roadblocks

I've written about writing code that will last before from the context of coding standards. Today I'd like to look at it from the non-code perspective. 

As a software engineer one of the best feelings was wrapping up a complex solution and feeling like I had written beautiful code. Code that sailed through peer review unscathed. Code that adhere'd to the standards of my team. Code that used best practices. Code that was simple and elegant.

One of the worst feelings as as software engineer was realizing that your beautiful code wasn't standing the test of time. Maybe it wasn't flexible enough to adapt to changing business requirements. Or possibly it created an unnecessary bottleneck. Or, despite the peer reviews, it wasn't as performant as expected.

  

Writing code that will last is about being able to look around corners and see the non obvious. Aside from the obvious, writing SOLID code, it's about avoiding potential roadblocks, about understanding bottlenecks, and identifying upstream and downstream dependencies. In this post I'll explore avoiding potential roadblocks.

Avoiding Potential Roadblocks

One of the biggest roadblocks to software engineering is cost.  Cost can come in many different forms; cost of development, cost of maintenance, cost of infrastructure. There are two areas to think about when considering the cost of software. First is ROI (return on investment). Does putting the software into production and keeping it in production save or make you more money in the long run than it will cost you to build and maintain? Make sure you're considering the operational overhead when determining ROI.

The second are to consider with regards to cost is opportunity loss costs, i.e. what software are you NOT building so that you can build this software. Your team could be building many different things. Why is this project the right one to build now? Is there something else you could be building that gets you a competitive advantage, a higher ROI, or reduces operational overhead?

Another roadblock to long lasting software is re-inventing the wheel. At it's simplest form it means writing software that already exists. It could exist as open source software, COTS (commercial off the shelf) software, or some internal library in your company's software portfolio. The latter is more often the case at larger companies as is having two different teams writing similar code at the same time. Sometimes in order to not re-invent the wheel, you need to change how you're approaching solving your problem so that you can re-use existing software. 

The last major roadblock to avoid is under-defined requirements. There's nothing worse than building a solution and realizing you it doesn't solve your problem because it didn't uncover the need for feature X. This is especially demoralizing to the team if they've just spent months building the software. The easiest way to avoid this is to have very clearly defined user scenarios up front coupled with short end to end milestones that allow you to get your software into production quickly. This allows you to reduce your feedback loop time and adapt to changing or unrecognized requirements.

The True Cost of Software Development

One mistake I've often seen made when planning for a software project or planning the road-map for a team is not taking into account the total cost of development. We often fall into the trap of planning our software based on one key data point; development effort. While development effort is likely the most volatile variable that goes into the cost of a project, it is by no means the only variable.

 

Planning isn't free

There are three major areas of planning that need to be accounted for when considering the true cost of a project. The first is in defining what the customer needs. This will involved someone (or multiple people) being the advocate for the customer. Likely this will involve a certain amount of research and a certain amount of customer interaction.

The second is in identifying upstream and downstream dependencies. Before you introduce change into your system, you need to understand how that change will affect people that depend on your system and on the systems you depend on. For instance, will your changes cause you to put more load on another system? Will they cause you to be able to handle less or more load on your existing system? Do they require your existing consumers to do additional work or is there work they can stop doing? It is likely that you will have to make some change in your system to account for these dependencies.

The last area is in defining key performance indicators. These are metrics that allow you to know if your software is working correctly or not. If you don't account for defining these then you won't be able to truly measure whether or not your software is successful.

System Architecture (or high level designs) aren't free

Often we think about a project start as the time when the developers start writing code. But this is incorrect. The old adage "measure twice and cut once" applies to software development as well. To build the right system, one that works for your customers, you need to have a solid design. True software costing accounts for the design and the time it takes to review the design.

Development

I don't think I need to state the obvious, but I will for completeness. You need to account for the actual development effort in the total cost. Usually this is done in dev weeks. This could be one dev for 12 weeks or 3 devs for 4 weeks, depending on the parallelization that the project allows for.  

Bug Fixes and Stabilization aren't free

Before you put any code into production you should be testing that code in a production like environment. Many teams have an integration or pre-prod environment which is built as a smaller scale replica of production. The very best teams have a way to drive production like traffic in this environment to help root out the bugs that seem to only be able to be found in production (like race conditions, threading issues, memory or other problems of scale).

Deploying Software isn't free

Hopefully by now your organization has started to, if not fully, embraced the DevOps movement. Irregardless of whether you have full continuous integration setup or a fully manual deployment there is some cost associated with the deployment of your code. In a fully automated environment it may be the cost of updating the automation. In a more manual deployment scenario it'll be the cost of validating and pushing the bits to production.

Maintenance and Operations isn't free

Once your software is in production, you have an implicit contract with your customers (if not explicit) to make sure that software is running. This involves routine maintenance as well as the operational cost associated with keeping the system running smoothly and responding well to changing usage patterns. Inevitably hardware fails, systems get more load than we expect, and bugs exist in the system that require intervention. Accounting for this cost is crucial in understanding the total cost of software development.

Documentation doesn't write itself

The last area that I want to call out is documentation. This is often forgotten about until it's too late. Your code should be as close to self documenting as possible, but your users will need a guide to the feature or the product. Don't forget to account for this in your costing.

Finding buggy code using git history and a binary search

Some bugs in software are easy to understand. Things like a null pointer exception are obvious and don't need a lot of context. But some bugs are more difficult. Some bugs just aren't obvious until you have the context of the change that was made that introduced the bug.

What I like to do when I'm hunting a bug down is to apply a binary search algorithm to my git history. This has two benefits:

1. You're able to find the exact commit a bug is introduced in
2. You've narrowed down the surface area of the code the bug could possibly be in

#1 above is important as it gives you the ability to reverse apply a patch and see if the bug goes away. If it works without breaking anything that's come to rely on that code it means that you've got a "quick fix" that can be issued to relieve the problem while you find a longer term solution.

Unless your team is a team of developers that goes dark #2 above is means you have found the handful of lines of code that contain the problem and the context of the code dependency. Simply seeing the inter-class dependencies may be enough to make an ambiguous bug make more sense. It's also a good place to look for violations of SOLID which make code less 'ible (maintainable, scalable, flexible, etc...).

So how do you accomplish this?

1. Find the starting point

Start with the git commit from your last release. Create a build from that and see if the bug reproduces. If not you've found your starting point. If it does reproduce and has been in the system a while, keeping going back release by release trying to find the commit of a build that does not contain the bug.

Hopefully you're tagging your releases or creating release branches.'

2. Use now as your ending point

3. Pick the mid point in time between your starting point and ending point

Using git log you:

1. Get the hash of the commit closest to the mid point in time.
2. Create a build using that commit
3. Check to see if the bug reproduces

If the bug does reproduce you've got a new starting point: Repeat step 3 with the new starting point.

If the bug does not reproduce: Repeat step 3 with the new ending point.

Using this binary search you'll find the source commit of your bug fairly quickly (limited only by your build and validation time) and you'll have the context of the commit to fully understand the change that introduced the bug.

Difference between a programmer, developer, and engineer

Coder, programmer, developer, software engineer and others were among the many different titles I had in my career while writing code. Some were sought after and some were not depending on where I was at in my maturity writing code.

When I first started out I wanted to be called a coder. I worked in IT divisions of small companies where some people in the group were in charge of configuring and maintaining the servers and some were in charge of writing the software that ran on the servers. In those early days  I wanted to distinguish myself from the former group and I put a heavy emphasis on distinguishing my self as a coder or programmer (usually the later). I wanted people to know that I was responsible for the logic of the programs that were running on the system, not responsible for using those programs to manipulate the system.

As I grew in maturity as a programmer I learned that good programming involved refinement and iteration. I started to learn that good programs, ones that were run for years on machines weren't simply written, they developed over time. Developing a system over time required recognizing tried and true patterns and best practices. It required abstractions in order to iterate over the system. Programs weren't simply the logic used to solve a problem. Programs were a set of requirements that grew and changed as the program became more complete.

As I started to master my trade I learned that the best developers were really engineers. People responsible for the design, implementation, and deployment of software. In my mind I started to realize that a great developer was really an engineer. Someone that was able to look at the requirements and design a system that met the need but was also resilient to change. I learned that the one constant in software development was change. Being adaptive to change required a solution to be engineered. You start with an idea, built a system around that idea, and put that system into practice.

As I honed my craft, I started to recognize the different between software programmers, software developers, and software engineers. Some key differences are:

Design

• Programmers think in terms of languages, SDKs, and APIs. They build systems based on the tools they know and are comfortable with. Programmers value creativity and cleverness.
• Developers think in terms of requirements, components, and interactions. They design systems using patterns and best practices that meet the requirements. They're able to evaluate, investigate, and determine if particular platforms, languages, and/or frameworks meet the need of the requirements.  Developers value completeness.
• Engineers are problem solvers. They solve problems regardless of platform, language, or tool. They think about architecture. That is dependencies, single points of failure, scalability, and resiliency change. Engineers value simplicity, flexibility, and adaptability. They recognize that a system is never fully complete.

Development

• Programmers value the most creative or clever code.
• Developers value the most abstract, generalized, and re-usable code.
• Engineers value the simplest code that can be maintained and changed.

Deployment

• Programmers don't typically think about deployment. They throw their code over the fence to operations folks who own and maintain the production systems for them.
• Developers think in terms standardized tools and processes. Developers value the ability to rollback.
• Engineers think in terms of automation, repeatability, and reliability. They think about the surface area of the change, and how to measure success.

Improving the efficiency of your team: cleaning up after yourself

If you're anything like me you're constantly trying to figure out a way to help your team become more efficient. One area that's often overlooked is how much time your team spends repeating the same mistakes or cleaning up after the same mistakes over and over again. I'm not talking about show stopping, bring your site down mistakes. If you're having those often you've got major problems in your underlying infrastructure and you should stop all feature development and get it fixed right away.

No I'm talking about small mistakes that over time compound the amount of time your team is taking to work around a problem or clean up after it. In order to help your team spend more time working on features and less time working on cleaning up after mistakes you should be doing two things.

Retrospectives

I've written before on Sprint Retrospectives before but it's worth mentioning them again as they are the easiest source of improving the efficiency of your team. Sprint retrospectives are a good way to understand what's randomizing your team. They're also a good way to find out why your teams estimates were off (i.e. what wasn't accounted for or what was over accounted for).

The key to the sprint retrospective not to just talk about the problems, but to create action items that will resolve the problems moving forward. After each sprint retrospective you should have an action item list. Each item in the list should have an owner and a due date. This will help ensure that nothing falls through the cracks.

Postmortems 

Postmortems help your team identify and correct problems with your process or places where there is no process where there should be. I wrote a good deep dive into postmortems a while ago, but the key to a postmortem is that you use the facts about an incident to see where your process breaks down and identify clear actionable steps to prevent the breakdown in the future.

Postmortems don't have to be used when something catastrophic happens. You can use them to help identify simple things that prevent your team from being more efficient. For example; you can do a postmortem on your sprint planning if your team wasn't able to accomplish all the work they scoped for the sprint. You can do a postmortem on why a feature wasn't built as expected or why a build broke. You can do a postmortem if your team misses a date. There are many reasons you can do a postmortem that don't have to do with a catastrophic event.

The goal is not to do a postmortem for the sake of doing one. But instead to add value where possible by identifying areas of inefficiency and digging in to find out where the problems are and creating actions to remedy them.

Are you using your own products?

I'm always surprised when I work on a software team that doesn't use the product they're building. It's a sign that your team isn't invested in the product. More importantly, it's a sign that your team isn't as invested in the customers.

Teams who dogfood their own software make better products. That sounds anecdotal, which it may be, but that's what I've seen in practice. Why is software better when those that make it use it?

Your team is always in the ecosystem

If you aren't immersed in the ecosystem that your app lives in, you won't ever truly understand how your app should work. You won't understand what metaphors are the right ones and which ones will feel foreign or fake. You can be sure your customers will notice if you use an iOS metaphor on an Android phone or a Windows metaphor on a Mac. Until you are immersed in the environment your customers are in you won't really understand your customers as well as you could.

Your team is your customer base

Relying on your own software in your day to day life builds empathy for your customers because you are part of the customer base. You'll find the workflows that need simplifying. You'll understand which features are missing. You'll cringe and curse every crash.... just like your customers do.

Your team feels any added friction

When your team is using your app you discover added friction before your customers do. For instance, if you make a change that's not backwards compatible, your team will find it first because they're already using the app and that feature. Add additional, and unnecessary steps to some workflow, your team more likely to find it than a paying customer.

Long and short, you find the mess in your app before anyone else does. When the app gets to your customers, they're delighted with how intuitive your app feels. 

The Unlearning Of Technology

I grew up in the Washington D.C. area. I grew up a Redskins fan (whose name I DO believe should be changed). That also meant I grew up hating the Cowboys. My family (and extended family) had season tickets and I could go to 2 - 8 games a year. When not at the games I would watch them at home. I was almost religious in my dedication. I could tell you the names of almost every starter and a good portion of the bench.

I moved to Seattle 10 years ago and now I couldn't name 5 people who play for the Redskins. I'm not quite as fanatic a Seahawks fan as I was a Redskins fan, but I watch every game, go to as many as I can, and can tell you their record, their standing in the league and who the key players are.

Did I set out to become a Seahawks fan and not a Redskins fan? No, I just like football and fell pray to convenience and repetition. Not having Redskins games readily available made it hard to stay up with the current happenings. Not watching games each week I wasn't able to learn what the team was doing well today, learning where their defense was weak and their offense strong. Essentially, I was spending time unlearning what I knew about the Redskins.

The same is true in the software industry. You're only going to know the ins and outs of platforms, frameworks, and code that you actually spend time with. If you don't spend time with it you'll evolve and change on separate paths.

Given enough time, eventually you'll grow apart from the technology. You'll stop recognizing the subtle changes from version to version. You'll stop being able to speak to the differentiation of the platform or framework. You'll make stupid, or novice mistakes when you do dive in. You'll miss important aspects of design, architecture, or code quality.

This isn't always a bad thing. Sometime, in order to learn something new, you need to unlearn something you knew well. But it's important to recognize that if you're not actively learning and growing with a platform, framework, or code base, then you're actively unlearning it.

What are you un-learning today that you shouldn't be?

Understanding Trade-offs In Software Development

I've worked in the software industry for a decade and a half as both a software engineer and software development manager. One thing that I've learned to do for the projects my teams work on is to anticipate and understand the possible trade-offs you're going to have to make downstream.

It's a given that something is going to change between the start of a project and when you actually ship that's going to force you to make a trade-off. As I've talked about before, Agile and LEAN allow us to mitigate the cost of making these changes by reducing the time it takes to get and incorporate feedback.

One thing I haven't talked about is how to make the right trade-offs for your product.

Minimal Shippable Product

Understanding your minimal shippable product is the foundation of any trade-off decision you're going to have to make. I'm not taking about a gut feeling about what you're customers may want. I'm talking about understanding what makes your product different.

Some questions to ask yourself to identify the minimal shippable product:

• Does the lack of feature X put my product at a significant disadvantage in the market place I'm entering or are already in?
• Will 80/90/100% of my customers use feature X?
• Will missing feature X make you look stupid? For example, a text editing product without copy/paste or an image processing app without the ability to adjust contrast. 
• Is feature X a key differentiator for my product?
• Can your customers achieve feature X in some other fashion? 
• Does missing feature X cause significant friction for your customers?
• Is feature X on the short list of reasons to buy your product?

Time To Market vs Feature Complete

Another very important thing to consider is how important is time to market vs feature completeness. Are you providing a service that no one else currently provides? Does the market you're entering have a sales cycle that requires you to make a specific date?

Being first to market has strategic advantages that cannot be ignored. But being first to market with a sub-optimal product may hurt you worse than not being first. Apple wasn't first to market with the smartphone, but they offered features that weren't available anywhere else. The converse is also true though, they were first to market with the modern day tablet and it took half a decade for anyone else to really break into that market because there wasn't a compelling enough reason to buy the alternative to the market leader.

Breadth vs Depth

The last major trade-off to consider is whether your product provides a breadth of features which are mostly polished or a sub-set of those features that are fully polished. I'm not talking about your minimal shippable product here. I'm talking about the things that didn't make your MSP list, but may be the right things to build for your customers. For example, if you're providing a parity product, you need to know how much parity you need to achieve to be relevant to your customers.