The Pit Of Success

Over the last decade or so I've worked on a lot of teams chartered with building APIs, web services, and/or micro-services that were consumed by external and internal software. When building something that's composed into another system you're inevitably going to have to make design decisions which, if used incorrectly, will not have the desired effect on the system.

A few years ago a former co-worker of mine introduced me to a concept that helped change how I approach solving for this problem. That concept was called the Pit of Success. In it's most basic form the pit of success can be defined as follows:

Pit of Success: being easier to do the correct thing than it is to do the incorrect thing?

So how do we create a pit of success with the products we build? Here's some simple tips.

Solve for the 80% case out of the box

By default the software should solve for the 80% use case. Often this can be accomplished by simply creating reasonable defaults for configurable parameters. The way I like to solve for this is by asking myself the question "what is the out of the box behavior?"

Make the developer do work to harm the system

Most of what we do on the web involves creating, updating, or deleting data. Occasionally we have to build things into a system that can have harmful if used incorrectly. In these particular cases you can create a Pit of Success by making the developer do work to enable the potentially harmful feature or use the potentially harmful API.

Familiarity

Be conscious of the platforms and frameworks that you're working with. Don't invent new conventions that don't add material value to the system. Use idioms and patterns that are natural to the developers of that platform.

Transitioning well

Way too many times I've seen people leaving their jobs and transitioning poorly. Some see their decision as a free pass to express their frustrations. Some are fed up and want others to know about it. Some just aren't aware of what they're doing. Whatever your situation, here's a short guide to transitioning out of your current job, company, or role well.

Don't Sow The Seeds Of Discontent

There may be many reasons you're leaving your current role. You may be unhappy. You may feel slighted. You may disagree with the direction of the team. Or you may be leaving for some other reason entirely. It's important to remember that just because you feel this way doesn't mean everyone else should. It's not your job to make sure everyone else shares your grievances. That is a sign of immaturity. If you're having interpersonal conflict where you're at, chances are that you're at least a small contributor to the problem.

Be mature and don't be the poison pill that's trying to get everyone else to make the same decision as you. Transitioning is not the time to let everyone know what you've been secretly thinking about them, the project, or leadership. HR is the correct place to vent any of these frustrations.

Don't Burn Bridges

You've probably heard not to burn your bridges on your way out. And you've probably dismissed it as irrelevant to your situation. But there are two good reasons why you should take that advice to heart.

First, Though you may not be able to imagine ever working on that team, at that company, or in that role again you don't know what your future holds. There may come a time when you do want to come back. Maybe the leadership will have changed. Maybe they'll be solving a really interesting problem you're excited about working on. Maybe you've matured and seen the error of your ways. Whatever the reason may be, there may come a time when you find yourself wanting to come back to a job you've left. Leaving on good terms and being helpful on your way out leaves that door open to you.

Second, you don't know when you're going to run into your former colleagues again. You don't want the last impression you leave with those colleagues to be a negative one. They may be in the critical path for your success at some point in the future.

Do Your Best To Set The Team Up For Success

What can help your soon to be former team the most is to make sure that any tribal knowledge you have is documented and understood by the current team. 

• Document the things that are currently in flight by you.
• Make sure you state what the problem is, how you're solving it, and what's going to be left to solve.
• Go through your calendar for the past three months and look for any one off meetings you've had about current or future work and debrief the team and/or your manager on the problem space.
• Make an external point of contact list with the names and roles of every external team you interface with and share this list with your manager and current team before you leave.
• Make an internal point of contact list that can be shared with your external points of contacts and share this list with them before you leave. 

You Shouldn't Backfill Unit Tests

Recently I've found myself in several conversations about unit tests, and the lack of them on a given project. It seems that almost everyone on the team knows that writing unit tests is a good thing and it seems that most of them also wished they'd written them from the start. But, as is the case with most fast moving projects, the team finds themselves in the position where they don't have adequate test coverage. This leads to reduced confidence in the code and not being able to catch bugs far enough upstream to effectively prevent them from ever being released to customers.

What this usually leads to is people on the team making comments, in response to a bug or regression someone is working, that if they only had unit tests they would have found this bug earlier. What I've seen most often in that situation is someone finally getting fed up enough with bugs and/or regressions and picking up the unit test torch and leading a crusade to backfill all the missing tests. While I appreciate the spirit of this, it's not the right thing to do.

Why's it not the right thing to do? First, you're not actually solving your problem. Your actual problem is that you can't ensure that the code you're currently changing 1. does what it's meant to do and 2. doesn't do something it's not meant to do.  Second, depending on how old the given code is, you may not know how it was actually supposed to work vs how it actually does work. You may end up writing unit tests are aren't correct, and in the worst case scenario cause you to change working code. Lastly, when the team is done backfilling the tests, they haven't learned new behavior (i.e. to write the tests when you write the code) but, more likely, have grown a disdain towards writing unit tests.

Now before you start to think I'm advocating against writing unit tests I'll flat out tell you I'm not. In fact I've come to realize in my career that writing your tests first is the only way to assure that you have working code. So what am I advocating for then?

Make a decision as a team to write unit tests and then write unit tests for all the code the team writes from that point forward. You'd be surprised at how much better your code coverage will be after just a short amount of time. Why? Because you'll be writing unit tests for code that you're fixing bugs in. You'll be writing them for new features or changes to existing features. Essentially, you'll be writing tests in the hot spots of your code that are requiring change. The very code that is either causing bugs, or is likely to have bugs introduced in.

If you never change a particular set of code again then you don't really need unit tests for that code (remember I'm talking about unit tests and not integration, system, or other types of tests).  If you do change that code, then because you've decided as a team to write tests, you'll add the missing tests when you go in to make the code change.

So how do you get started? It's easy, just follow my 3 step plan.

1. Make a decision to not put up with having incomplete code (i.e. code without corresponding tests).
2. When you write your next set of code, write tests!!!! (preferably first)
3. Enjoy the fact that you now have less buggy code that you can have more confidence in.

Apple, Encryption, and the FBI's request

Encryption has been in the news a lot lately with the recent stand Apple has taken against the FBI's request to help them break into an iPhone. Whatever side you're on with this I think there's some interesting things that may be a bit misleading to the general public that I'd like to weigh in on from a technologists perspective.

I don't work at Apple or the FBI so all of my information about what's going on is from what I've read in the news, from Apple's and the FBI's public statements, and from what I know about how software is built and how operating systems work.

1. Apple is being a bit disingenuous when they say that they're being asked to create a new operating system. I think I understand why they're using this language. I can't find the source to cite here but I believe the legal guidance around the All Writs Act that the FBI is using to claim legal jurisdiction is that a company cannot be compelled to create something new in order to comply with the request. In Apple's case, saying that they're creating a new operating system makes then case stronger in their favor.

What Apple is being asked to do is create new code or remove existing code (likely somewhere in between) in their existing operating system that would weaken or bypass the current security of the system. The reason I say this isn't a new operating system is because an operating system at it's core is a system that interacts with hardware, sensors, and peripheral devices by providing tools and services that applications can build upon. Operating systems do things like interact with your hard-drive, memory, graphics card, and etc. While security is part of the operating system, 99% of what Apple would be delivering with a change would be the same as what's in their operating system now.

Regardless of whether what Apple would need to create is a new operating system or not, they're being asked to create a product that does not exist today by writing (or changing) code that would allow for a brute force attack. So I think Apple is being genuine in what they're trying to say, but not in the way they're saying it.

2. Apple's correct that you can't just write something and then throw it away. 

The only way to achieve what the FBI is asking without having any trace of the source code would be to have a machine from which the source could be changed and the operating system build that didn't have a hard disk or memory attached to it. If there is a hard disk or memory there is a "record" of the change at the lowest levels from which the changed code could be re-constructed. This isn't likely as it would require physical or network access to the machine from which the code was written. You could prevent network access by never plugging the machine in to the network. And you could prevent physical access by destroying the machine immediately after creating the modification to the operating system. Neither of those two are really feasible in today's modern software development ecosystem.

Virtually all software written today, likely including Apple's, is written using source control. Meaning, there is a system that contains every line of code and tracks all changes to that code. This is done for many many reasons. For instance, as software engineers we often have to re-create a build at a given point in time in order to find and fix bugs in a particular release. By the time software is released, we're already working on the next thing. Being able to recreate a build exactly as it was allows us to create targeted changes (or updates) as well as gives us the ability to reproduce errors. Additionally source control gives us an audit mechanism for changes introduced into the system. We can see who, when, (and often) why a change was introduced. With a source control system it is possible to go back to a snapshot of any code in the system at any point in time.

Trying to make changes that never get saved up stream to the source control system isn't that tough. But what is likely tough is getting a fully releasable build created using a build system that assumes the build is being created from source control. It's likely that the Apple's code that builds Apple's operating system is done so through a pipeline of tools that gets the source code from source control. Meaning, all of Apple's processes are likely setup around the use of a source control system.  So if engineers at Apple were to create the requested software it's likely that it would have to be added to the source control system in order to create the requested build of the operating system. Once these changes are added to source control, they're forever there and forever accessible (to an extent).

3. This is about more than just 1 phone. As much as the FBI wants us to believe this is just about one phone it's not. One of the ways our legal system works is through the setting of precedent. Once an act has been performed that can serve as an example for future similar cases precedence has been set. If Apple complies with this request there's precedence that can be used to compel other hardware/software manufactures to do the same.

4. Security is about trust and at the end of the day that is what this is all about. Apple, rightly so, does not want to lose the trust of those in it's ecosystem. It doesn't matter whether Apple thinks this is the correct thing to do from a moral, legal, or ethical standpoint. If Apple loses the trust of those in it's ecosystem that would spell imminent doom for the company.