Writing code that will last: Identifying Dependencies

In my previous post, writing code that will last: understanding bottlenecks, I talked about how bottlenecks can come in the form of vertical scalability, horizontal scalability, or statefulness. In this post I'll explore how identifying upstream and downstream dependencies in your software can make or break it from a longevity perspective.

Upstream Dependencies

There are several upstream dependencies that you need to consider in your code. These come in the form of your operating system, the platform your application or service runs on, the external applications and services you use, and the contracts you depend on for data.

Operating systems and software platforms can affect the performance, security, and ability of your service. Writing code that lasts means abstracting your code enough from the OS and underlying platform such that you can upgrade the OS or platform without changes to your application or service. This allows you to protect your self and your customers from security vulnerabilities as well as allows you to take advantage of performance enhancements that come from kernel or library updates.

You should also consider the dependencies you have an external applications or services. Do you depend on them in a version specific way? How will your application need to change if they change the schema of the contract you have or deprecate an API you use. Building code that will last doesn't mean prematurely optimizing your code to try to defend against these changes, but instead means understanding how to encapsulate them such that the effect these changes have on your code are minimal.

Downstream Dependencies

Downstream dependencies are people, applications, and services that depend on your software. From a downstream dependency perspective writing code that will last means understanding when you're making a choice that you can't go back and change after the fact. In order to write code that lasts we have to minimize the number of these choices that we make. These choices often come in the form of the APIs we expose, the contracts we create with other services, the way we represent objects, and etc. Once these choices are made, sometimes they can't be taken back without having some sort of negative side effect.

Writing code that will last: Understanding Bottlenecks

In my previous post, writing code that will last: avoiding potential roadblocks, I talked about how roadblocks can come in the form of cost, re-inventing the wheel, or under-defined requirements. In this post I'll explore how understanding potential bottlenecks in your software can make or break it from a longevity perspective.

Understanding Bottlenecks

The first bottleneck you should be considering is how your application scales vertically. Vertical scale typically has direct correlation with your applications performance characteristics. Vertical scale is achieved by increasing CPU, Memory, Network Capacity, or Disk your application has available to it. Is your application going to be CPU or Memory bound? Will you require a lot of Disk I/O? How much data will you be reading/writing over the network? You need to understand each of these areas in order to know how your application will scale vertically.

How your application scales horizontally is the next potential bottleneck you should understand. Horizontal scaling is the ability to add additional machines running your software into an existing pool to add additional computing. Horizontal scaling allows you to increasing the throughput of your application by having more capacity to process requests or jobs that the application performs. Your software needs to take this into account from the beginning or it won't last without requiring rework.

Another area to consider when understanding your applications bottlenecks is how stateful it is. Does your application require some external state in order to run? How the application gets and maintains this state can lead to potential bottlenecks. E.g maybe your application needs some specific information about the user making the request that it has to lookup somewhere. Or maybe it needs information about previous requests. An example of a stateful application is one that uses a shared database. The reason state cases bottlenecks is because your application is now dependent on something external that it does not control.

Writing code that will last: Avoiding Potential Roadblocks

I've written about writing code that will last before from the context of coding standards. Today I'd like to look at it from the non-code perspective. 

As a software engineer one of the best feelings was wrapping up a complex solution and feeling like I had written beautiful code. Code that sailed through peer review unscathed. Code that adhere'd to the standards of my team. Code that used best practices. Code that was simple and elegant.

One of the worst feelings as as software engineer was realizing that your beautiful code wasn't standing the test of time. Maybe it wasn't flexible enough to adapt to changing business requirements. Or possibly it created an unnecessary bottleneck. Or, despite the peer reviews, it wasn't as performant as expected.

  

Writing code that will last is about being able to look around corners and see the non obvious. Aside from the obvious, writing SOLID code, it's about avoiding potential roadblocks, about understanding bottlenecks, and identifying upstream and downstream dependencies. In this post I'll explore avoiding potential roadblocks.

Avoiding Potential Roadblocks

One of the biggest roadblocks to software engineering is cost.  Cost can come in many different forms; cost of development, cost of maintenance, cost of infrastructure. There are two areas to think about when considering the cost of software. First is ROI (return on investment). Does putting the software into production and keeping it in production save or make you more money in the long run than it will cost you to build and maintain? Make sure you're considering the operational overhead when determining ROI.

The second are to consider with regards to cost is opportunity loss costs, i.e. what software are you NOT building so that you can build this software. Your team could be building many different things. Why is this project the right one to build now? Is there something else you could be building that gets you a competitive advantage, a higher ROI, or reduces operational overhead?

Another roadblock to long lasting software is re-inventing the wheel. At it's simplest form it means writing software that already exists. It could exist as open source software, COTS (commercial off the shelf) software, or some internal library in your company's software portfolio. The latter is more often the case at larger companies as is having two different teams writing similar code at the same time. Sometimes in order to not re-invent the wheel, you need to change how you're approaching solving your problem so that you can re-use existing software. 

The last major roadblock to avoid is under-defined requirements. There's nothing worse than building a solution and realizing you it doesn't solve your problem because it didn't uncover the need for feature X. This is especially demoralizing to the team if they've just spent months building the software. The easiest way to avoid this is to have very clearly defined user scenarios up front coupled with short end to end milestones that allow you to get your software into production quickly. This allows you to reduce your feedback loop time and adapt to changing or unrecognized requirements.

What's your reputation and what are you doing about it?

One thing that I always struggled with as a software engineer was reconciling what I knew about myself and what other thought about me. Every year during annual reviews (more often with the good managers) I'd get a glimpse into how others viewed me in the form of my performance review. I was not usually surprised with my overall performance rating, but there were usually things that came up that I wasn't aware others were paying attention to. Some good and some bad.

As I've shifted into a management role I've come to really appreciate how much your reputation can help you succeed or get in the way of your success. One of the things I try to work with my engineers on in making sure that the best of themselves comes out in how others view them. It's just easier to succeed in your role and in your career growth if others have an accurate view of your skills and capabilities, and if you're open to the areas you need to grow in. 

All too often I see engineers trying to change their reputation by telling others that they're not what they seem to be or making excuses for why they act a certain way. More often than not, this is not helpful. Instead you need to be doing something different to change your reputation. Here's some tips to successfully manage your reputation:

1. Decide that you care about your career and your own personal growth

Almost everyone I've ever managed or lead on a team has said that they care about their career and their own personal growth, but when confronted with challenges many have decided that overcoming those challenges required more work then they were willing to do. They give up and let their reputations dictate their opportunities and their success or failure.

The biggest hurdle I've experienced to making this decision is believing that you can be successful. You have to know that the road ahead is difficult, that you are capable of success and then make a conscious choice to put in the effort to grow. 

2. Understand what your reputation actually is

The first key to this is just preparing yourself that you're going to hear things you don't want to about yourself. You're going to hear that you struggle in areas that you either don't believe matter or that you don't believe you struggle in.

The unfortunate reality is that if you're hearing feedback that you don't believe to be true, there is something you're doing (or not doing) that is contributing to the reputation. This is not to say it's your fault. But you must acknowledge that you have room to grow.

Personally, I've heard my whole life (not just in my career) that I'm too combative. I see that trait as being passionate about what I believe. While that may be true (at least I hope) it doesn't change the reality that I am viewed through that lens and that I can't change that view until I actually understand that it is a filter that people are using when interacting with me.

3. Look for ways to capitalize on your strengths

One mistake I made at the beginning of my career was using a strength ineffectively. I'm a person that has always had good intuition and perception. I pay attention to little details and become aware of things that others don't recognize. The mistake I made with this was two fold. First, I thought that because my intuition and perception were very good that they were flawless. In other words I thought I was always right. Second, I thought this good intuition was a license to offer advice even when my advice wasn't sought out. These two mistakes compound and ended up building a reputation of being a know it all.

Through very successful coaching by managers who recognized this strength being used ineffectually I've learned to turn something that was making me ineffective into something that makes me more effective by asking questions. When I intuit or perceive something that I don't believe others are seeing I now ask questions instead of making statements.

This helps me in three ways. First it shows that I'm open to being wrong about my intuition or perception. If I'm open to being wrong and others are aware of it, they're more open to me being right. Sounds odd but it's true. What I'm doing is creating a culture where failure is okay and others don't feel like I'm judging them if I succeed or if they fail. Second, it helps others to feel like they're also apart of the solution. By asking a question you're inviting them to participate. By helping others to be involved they feel ownership in the result. Lastly, it helps me uncover the things that I don't know I don't know. Occasionally, these questions will uncover evidence to reinforce my initial intuition or perception. But more often they will uncover things I wasn't aware of that help me gain a more solid understanding of what we're talking about. 

The key here isn't just to make sure you have opportunities to showcase your strengths, it's to learn to make your strengths more effective.

4. Look for opportunities that allow you to growth in areas of weakness in a way that sets you up for success

Everyone has room for growth. Effectively managing your reputation with respect to these areas of growth means learning the skills necessary to be effective and then practicing those skills. Once you start to master these new skills you then look for opportunities to scale them.

One common example I run into is growing as a leader. Most ineffective leaders, in my experience, think leading is about directing others so they focus on being right and setting the direction (read: telling people what to do). The result is that these people get a reputation as micro-managers who don't trust their direct reports or peers. 

An example of growing in this area is by learning what motivates people and then learning to apply that motivation to what needs to be done. This results in leading by getting people to want to do the right thing.

The True Cost of Software Development

One mistake I've often seen made when planning for a software project or planning the road-map for a team is not taking into account the total cost of development. We often fall into the trap of planning our software based on one key data point; development effort. While development effort is likely the most volatile variable that goes into the cost of a project, it is by no means the only variable.

 

Planning isn't free

There are three major areas of planning that need to be accounted for when considering the true cost of a project. The first is in defining what the customer needs. This will involved someone (or multiple people) being the advocate for the customer. Likely this will involve a certain amount of research and a certain amount of customer interaction.

The second is in identifying upstream and downstream dependencies. Before you introduce change into your system, you need to understand how that change will affect people that depend on your system and on the systems you depend on. For instance, will your changes cause you to put more load on another system? Will they cause you to be able to handle less or more load on your existing system? Do they require your existing consumers to do additional work or is there work they can stop doing? It is likely that you will have to make some change in your system to account for these dependencies.

The last area is in defining key performance indicators. These are metrics that allow you to know if your software is working correctly or not. If you don't account for defining these then you won't be able to truly measure whether or not your software is successful.

System Architecture (or high level designs) aren't free

Often we think about a project start as the time when the developers start writing code. But this is incorrect. The old adage "measure twice and cut once" applies to software development as well. To build the right system, one that works for your customers, you need to have a solid design. True software costing accounts for the design and the time it takes to review the design.

Development

I don't think I need to state the obvious, but I will for completeness. You need to account for the actual development effort in the total cost. Usually this is done in dev weeks. This could be one dev for 12 weeks or 3 devs for 4 weeks, depending on the parallelization that the project allows for.  

Bug Fixes and Stabilization aren't free

Before you put any code into production you should be testing that code in a production like environment. Many teams have an integration or pre-prod environment which is built as a smaller scale replica of production. The very best teams have a way to drive production like traffic in this environment to help root out the bugs that seem to only be able to be found in production (like race conditions, threading issues, memory or other problems of scale).

Deploying Software isn't free

Hopefully by now your organization has started to, if not fully, embraced the DevOps movement. Irregardless of whether you have full continuous integration setup or a fully manual deployment there is some cost associated with the deployment of your code. In a fully automated environment it may be the cost of updating the automation. In a more manual deployment scenario it'll be the cost of validating and pushing the bits to production.

Maintenance and Operations isn't free

Once your software is in production, you have an implicit contract with your customers (if not explicit) to make sure that software is running. This involves routine maintenance as well as the operational cost associated with keeping the system running smoothly and responding well to changing usage patterns. Inevitably hardware fails, systems get more load than we expect, and bugs exist in the system that require intervention. Accounting for this cost is crucial in understanding the total cost of software development.

Documentation doesn't write itself

The last area that I want to call out is documentation. This is often forgotten about until it's too late. Your code should be as close to self documenting as possible, but your users will need a guide to the feature or the product. Don't forget to account for this in your costing.

Smart watches two biggest problems

Prior to purchasing my Pebble Time I hadn't worn a watch in almost a decade. I stopped wearing a watch after breaking several in day to day activities. While it was less convenient to look at my phone for the time, it was better then shelling out hundreds for a new watch every couple of years.

The Pebble Time was the first smart watch that actually made me want to wear a watch again. Not because it was a great fashion accessory, but because I hoped it would fill a couple gaps of convenience for me. I really liked the idea of being able to respond to text messages via voice, being able to see my calendar and get reminders without having to pull out my phone, and being able to do a variety of small tasks like checking the arrival of the next bus.

The input problem

Reading messages, calendars, activity tracking, and other types of small time bound information like sports scores and bus schedules is pretty easy and convenient on a smart watch.  But inputting information is incredibly difficult. There are three main ways to input information and all three of them suck. There's canned phrases, voice dictation, and keyboard input.

Given the small size of the smartwatch input, the only effective keyboards I've seen are T9 style. Anyone who remembers features phones remembers what it's like sending a text on a T9 keyboard. It's difficult and time consuming. A full keyboard on a smart watch is likely a non-starter unless someone can figure out how to make Swype like functionality useful on such a small screen. The only way I can see a keyboard being useful on a smartwatch is for it to learn how I talk and to predict what I'm going to say.

Voice dictation on a smartwatch isn't everything I thought it would be. There are times when it's incredibly useful but more often than not, it's just not convenient to dictate via voice. It's super frustrating when you dictate a long sentence and it gets 99% of what you said but changes some crucial word or phrase such that the entire dictation is meaningless. This happens often enough to make me not trust that it's going to be correct. Another problem with voice dictation is that often your surroundings aren't conducive to you being able to dictate via voice. A good example is when standing in line at the coffee shop or riding on an elevator. Speaking into your wrist in these situations just feels awkward. Whatever I'm trying to do, whether it's set a reminder or send a text message, the conversation feels too intimate for me to be saying out loud. It's also just weird to be talking into your wrist and not to another person.

This leaves canned phrases. I don't know about you but I can't seem to narrow down all the things I'm going to need to say at some point in the future to 4-5 canned phrases. It would be useful if the watch could predict what I wanted to say based on previous conversations. interactions, or time of day. For example, everyday when I'm leaving work (between 5-6:30pm) I send my wife a text saying I'm packing up. The likelihood of that being what I want to send in that time-frame Mon-Fri is pretty high. It would be nice if my watch allowed me, with one click, to send that message. For everything else, I've found that what I want to say is so context specific that canned phrases just aren't useful. The only thing I can think of that may work would be if the companion phone app was monitoring everything I sent and building a profile and creating dynamic canned phrases. But, that's just creepy.

The battery problem

The battery life on the Pebble Time is really good. I only have to charge my watch once a week. But even that feel too often. I couldn't imagine what it's like to own an Android or Apple watch where you have to charge it everyday. Smart watches need to perfect wireless charging.  I'd be more than happy to have a wireless charger setup at my desk at work and near my couch at home. I'm in both these locations often enough that my watch should be able to keep a charge indefinitely. I should not have to think about taking off my watch and charging it.

The Pit Of Success

Over the last decade or so I've worked on a lot of teams chartered with building APIs, web services, and/or micro-services that were consumed by external and internal software. When building something that's composed into another system you're inevitably going to have to make design decisions which, if used incorrectly, will not have the desired effect on the system.

A few years ago a former co-worker of mine introduced me to a concept that helped change how I approach solving for this problem. That concept was called the Pit of Success. In it's most basic form the pit of success can be defined as follows:

Pit of Success: being easier to do the correct thing than it is to do the incorrect thing?

So how do we create a pit of success with the products we build? Here's some simple tips.

Solve for the 80% case out of the box

By default the software should solve for the 80% use case. Often this can be accomplished by simply creating reasonable defaults for configurable parameters. The way I like to solve for this is by asking myself the question "what is the out of the box behavior?"

Make the developer do work to harm the system

Most of what we do on the web involves creating, updating, or deleting data. Occasionally we have to build things into a system that can have harmful if used incorrectly. In these particular cases you can create a Pit of Success by making the developer do work to enable the potentially harmful feature or use the potentially harmful API.

Familiarity

Be conscious of the platforms and frameworks that you're working with. Don't invent new conventions that don't add material value to the system. Use idioms and patterns that are natural to the developers of that platform.