Monday, January 5, 2015

Breaking Into The Software Industry Without A CS Degree

I want to start out by saying that having a CS degree will be very very very helpful in a career as a software engineer. I don't want anyone reading this post to confuse what I am about to say as me believing that a CS degree is not helpful, it is. A CS degree will introduce you to Boolean logic, data structures, code optimization, (hopefully) the ability to debug code, and at least one language. It will also make sure you understand more than just the basics of mathematics. But having a CS degree is not the only path to a successful career in software engineering.

Over the years I've worked on everything from criminal investigation software, enterprise content management systems that serve millions of pages to millions of users a day, and mobile applications for iOS, Android, and Windows Phone. I entered the industry without a CS degree; I have a B.A. in Philosophy with minors in Communications and Religion.

I also want to clarify that this post is geared towards those wanting to become professional software engineers. If you're looking to becoming a hobbyist developer, while useful and applicable, most of this will be overkill for you.

If you don't have a CS degree and you want to be a professional software engineer you're going to have more work to do than your counterparts with a CS degree. You're going to need to start by learning the basic's of software engineering. While not an exhaustive list, a good place to start would be learning the following.

  • Math, specifically algebra, calculus, and geometry.
  • Boolean logic.
  • Data structures (learning what they are and learning to identify when to use them).
  • How memory works and is managed as well as the difference between a stack and the heap is.
  • Debugging and profiling code.

I've often been asked how to go about starting to learn the above list of concepts. With the exception of the math you can actually start learning those concepts by diving right into a software development language. I would suggest learning 4 types of languages. Each language will give you a different perspective on the above list.

Start with assembly, move on to C++, from there learn Java or C# .NET, and finally learn HTML and JavaScript. Learn them in that order. Optionally you'd be well off learning a scripting language like perl or python. While scripting languages can be used for more heavy lifting they're very useful as languages for building tools and utilities to automate redundant and often error prone tasks.

Assembly will give you a deep understanding of the connection between a processor and code. It will help you to be aware of how much work computers need to do to use things that are very simple in code (but expensive) like loops and case statements.

C++ will teach you about memory management better than any other language while encouraging you to use object oriented concepts. C++ has a much steeper learning curve but it will teach you more about what's going on with the computer than a high level language will.

Java and C# .NET are both managed code, meaning they run in a virtual machine. The virtual machine will take care of *most* of the memory management for you. These languages will allow you to focus on learning the life-cycle of your software and software design. Higher level languages also make it easier to identify and use data structures correctly. When learning data structures you'll want to pay attention to learning not just how they work but also learning to identify when they should be used. This is going to be a requirement if you want to work for any of the big players in the industry.

I've lumped HTML and JavaScript together as one thing even though technically they're not a single language. I'm comfortable lumping them together as one because they've become so interdependent on each other for web development that you're not going to be able to be truly competent as a web developer without understanding both. Learning HTML and JavaScript will help you learn *the web* as well as help you learn the concepts of User Interface (UI) design and workflow.

This list won't teach you everything you need to know about computers or even software. But it does give you a place to start. As you get into the industry you'll find certain niche's that interest you more than others and you'll naturally dive deeper into that technology stack. 

Monday, December 29, 2014

The (Unfortunate) Reality Of Open Source

I've been thinking a lot about open source software recently. It's amazing how much of our lives are run by open source software, how little people are aware of, and the downstream affect it has on our lives. Are you an Android, iPhone, or Kindle user? Do you use Gmail or Google search? If so, you're using open source software. In fact Amazon, Google, Samsung, Apple, Netflix, TiVo, Comcast, Wikipedia, WordPress, and many many many more companies have built core parts of their businesses using open source technologies.

In a way, you not being aware of open source software in your everyday life is a win. It means that open source is overcoming some of the usability problems that plagued the community through most of the 90's and 2000's. The fact that Android is the dominant mobile operating system means that people are learning to build usable graphical user interfaces (GUI) on top of Linux. It also means that by using the products we love we're making products that we aren't even aware of better simply because they're based on the same open source technologies.

Yet in another way, you not being aware of open source software in your everyday life is bad. When people aren't aware of the possibilities of what they're using they don't know what to expect or even demand. Simply using open source software doesn't mean that your favorite app or website or service is interoperable with anything else. In order to make software work together we also need open standards.

Here's an analogy to make this more tangible. What if in order for you to fill up your car with gas you could only go to your manufacturers shop and you couldn't pump gas from a competing manufacturer because the nozzles were different or the cars used different fuel? That would make it very difficult for you because your manufacturers decisions and processes would have a much bigger effect on your everyday life. You'd have a much more rigid schedule for getting gas. But because there are agreed upon ways that gas pumps work you are able to use any number of competing services. Your manufacturer is free to innovate and make their car better but you have flexibility in how you consume and use gas. The competitors have to earn your business.

Now let's use a technology example. Do you use dropbox? What would happen if tomorrow you wanted to move everything in your dropbox to Amazon, Apple, Microsoft or Google's cloud? Would you be able to have that stuff migrated automatically just by choosing a new service? Or would you have to do all the work yourself?  There are open standards for folder sharing like WebDav. There's no reason that you couldn't just enter your other services credentials into Dropbox and click a button to migrate your data other than them not wanting to make that easy for you.

Right now in the software world it's like we're driving around in cars that all use gas but because the nozzles are all different shapes and sizes we can only get gas from one manufacturer. As a society we seem to have accepted owning the burden of interoperability in our technology. So we choose the services that cause the least amount of pain rather than choosing the services that cause the most amount of joy.

It doesn't have to be this way. If we demanded that our data be migratable and we were easily able to move from Brand A to Brand B then both would be forced to earn our trust. Both would have to work hard to provide us with services and support that delight us. Both would have to listen to us when we say that using their products are difficult or unintuitive.

How do you make these demands you may be asking? With your wallet. Buy software and services from companies that use open source and open standards. Don't buy from those that don't. Talk to your friends and family about this problem and educate them. Ask your friend who is computer literate to help you chose products that will remove some of the burden from you.

Monday, December 22, 2014

2014: The year for streaming media

Personally, 2014 was a good year for me. I switched jobs mid-year and have been working on enjoyable projects with really really smart people. My wife and I traveled to Budapest and Prague, which was absolutely amazing as well as San Francisco, Portland, Iowa, and Richmond (VA). We got to see one of my lovely cousins get married and meet the new 2nd cousins. We spent Thanksgiving with my sister-in-law and got to met our new and very beautiful niece Eleanor. I've kept up with blogging once a week which, honestly, I wasn't sure I would actually be able to do. But the best thing that happened in 2014 for me was that my wife and I are found out we are expecting our first child. I'm both terrified and excited at the thought of being a dad, but I can probably predict the future and tell you that having a child is going to be the highlight of 2015.

2014 was an interesting year for technology and there were a lot of new products announced and released. But from the perspective of what affects my everyday life most I've been interested in what's been happening in the streaming media segment.

While Apple has had a media streaming solution for several years their competitors (other than Roku) have struggled to come up with any interesting alternatives. There have been may solutions over the years for home media centers like Windows Media Center, XMBC on a Raspberry Pi (Raspmbc) or XBox to say the least. But in my opinion, none of these solutions other than Apple TV or Roku have had the teeth to take off in the mass market. And, unfortunately, without mass adoption the quality of content is not very good.

2014 saw two entries into the streaming market which I believe will help drive competition and innovation in a category of software and hardware which has been pretty stagnant. Amazon announced it's Fire TV, which was received very well and Google announced the Nexus Player which seems to be a legitimate reboot of their efforts to get into the streaming TV market. Amazon also announced the Fire TV Stick.

We've got a Roku, Chromecast and Fire TV Stick in our household. We don't have an Apple TV because we don't own any Apple mobile products and Apple is a pretty closed ecosystem and I really really really don't want to encourage that. That's not to say they don't make beautiful products because they do.

The Roku is a simple and easy to use device with a pretty decent interface. But to me something just doesn't feel right. When I use the Roku I put down my everyday tech (phone/tablet) and pick up their remote and use their software. I'm aware that I'm using something that isn't customized to me. Their interface doesn't feel like "home" to me like my own tech does. The Roku feels slightly foreign. Also, streaming media from my phone or tablet directly to the Roku has been clunky at best.

Up until December the Chromecast has been my favorite device of 2014. I really like how easy it is to use. I can just open the YouTube app on my phone and start queuing up clips. Or I can open Netflix, WatchESPN, Comedy Central or HBO Go, find what I want to watch and then just fling it to my T.V. Really my biggest complaint with the Chromecast is that the app developers have to integrate it directly into their mobile app and apps have been slow to adopt this additional API.

I say that the Chromecast has been my favorite device up until December. That's because I got my Fire TV Stick this month and so far it's been pretty incredible. The interface is great and very intuitive if you're already familiar with Amazon Instant Video. I downloaded the free remote app and the voice search is very very accurate and fast. One of the things I really liked about the Roku is that the apps are right on the device and the Fire TV Stick followed a similar path. It has all of the apps I use today with my Chromecast except HBO Go. But I can even stream HBO Go directly from my phone to the Fire TV Stick using display mirroring.

While display mirroring is a battery drain on your phone/tablet it's pretty useful for me. Traditionally I've used it to stream to the Chromecast when an app hasn't implemented the Chromecast API. It's been nice being able to use it with the Fire TV Stick out of the box. I think my biggest complaint with it is that I have to enable it on the Fire TV each time I want to mirror.

I have big hopes that 2015 will bring more innovation to streaming media.

Monday, December 15, 2014

DIY Xen: Shrinking a Linux Disk

If you've been a reader of my blog for a while you've probably picked up two important details about me. First, I'm a huge fan of open source. Second, I'm a big DIY'er when it comes to software and services. Not that they have to, but I think the two tend to go hand in hand. My guess as to why that is may be because most people who gravitate to open source seem to do so out of a desire to learn.

I've been running my own mail and web server(s) for well over a decade now. Not because I think I can do it better than what's out there. But because I was truly interested in understanding the nitty gritty of what makes the internet run. Historically I had always done this on a single Slackware Linux box. This served it's purpose but did come with a few side affects. I was using my email service as my primary email, my RSS aggregator as my primary source for news, and my CalDav server as my primary calendar.

One big problem I started to run into with my single server setup was that every so often while tinkering with some new software I wanted to learn about I'd inadvertently take down my server for a bit. Which basically meant I was dead out of the water in terms of my email, calendar, and RSS feeds. So I decided to let my curiosity about "The Cloud" turn into working knowledge by setting up my own Xen server.

My initial impression (which still holds true today) is that Xen is awesome. Within just a few hours I was able to get Xen running on my hand built server (16GB RAM, 700GB hard drive, Intel Quad Core i5). Slackware has never let me down so I decided to stick with it for my guest OSs and I setup separate servers for my production services and my tinkering. It's been great.

One problem I ran into while I was trying to find the optimal setup was shrinking a Linux disk that I made too big to start. So I thought I would document the process in case anyone out there was running into the same issue.

  • Shutdown your existing Linux Virtual Machine (VM).
  • Create and attach a new storage device in XEN.
  • Start the Linux VM.
  • Create a partition on new drive.
  • $ sudo /sbin/fdisk /dev/sd[b,c,etc] (i.e. sdc or sdb or etc)
  • Create a filesystem on the new partition.
  • $ sudo /sbin/mkfs.ext4 /dev/sd[b,c,etc]1
  • Create temporary mount point so that you can copy the existing partition over.
  • $ sudo mkdir /temp_mount
  • Mount smaller partition you just created.
  • $ sudo mount /dev/sd[b,c,etc]1 /temp_mount
  • Copy the contents of the existing partition that you want to shrink onto new smaller partition.
  • $ sudo cp -rax /old/drive/* /temp_mount/
  • After the copy has completed unmount the new smaller partition.
  • $ sudo umount /temp_mount
  • Shutdown your Linux VM.
  • Detach the original larger drive from the VM in XEN
  • Restart the Linux VM and verify everything was copied and is working as expected.
  • Delete no-longer-needed storage device in XEN.

Monday, December 8, 2014

Willing, capable, and nearby

As I was getting ready to graduate college and enter the career world my grandmother gave me possibly the best piece of advice anyone has ever given me. She told me to always remember that there's someone that lives nearby that's just as capable and willing to work for less.

This piece of advice may sound cold or negative on the surface, but in reality it was meant to bring perspective, inspire humility, and make me ask myself why I am doing what I'm doing. My grandmother, who worked 40+ years at The Washington Post, recognized that given enough time we all feel unappreciated at work. She also realized that when we feel unappreciated we tend to over inflate our value and contribution.

Part of her point is that money is a means to an end. If money becomes an end in and of itself and your only motivation for feeling appreciated then you're going to be disappointed. Maybe it's getting less of a raise or bonus than expected or finding out that your co-worker, who works half as hard as you, makes more than you. Whatever the reason, relying on money to provide motivation at work will eventually fail and you'll find yourself unsatisfied and unfulfilled.

So what's the key then? The folks over at RSA have a great 10 minute video explaining how research has shown that money isn't a good enough motivator. That's not to say that money isn't important, it's just that there's a point where money as a motivator peaks. Once people make enough money that they're not constantly worrying about it then there are three main motivators; autonomy, mastery and purpose.

I really think that's the underlying point my grandmother was trying to make all those years ago. I don't know that she'd have been able to name those three areas specifically but I am absolutely sure that she understood that you need a combination of those three to feel appreciated and valued and to be motivated in your career.

I believe that she wanted me to understand that if I didn't search out and understand what it was about my job that motivated me then I would never really be happy in my career. For me this has translated into asking myself the question of whether or not I would do my job outside of work in my spare time.

When I was an individual contributor the answer to this question for me was really simple. It was a resounding yes. I would work 9 - 12 hours a day writing software at work to come home and write more software for personal use for another 4 - 6 hours. Writing software was, and still is, a hobby. It's a way I relax. It's something that helps me grow and keep my mind sharp. I really like solving problems and I like adding utility.

But once I entered middle management I had to ask myself this question "what motivates me now?" I think the answer to that question is actually one of the same reasons I started this blog. I really like investing in people. I enjoy mentoring and helping others grow. Not because I believe I know more than them or that I have all the answers. Actually, it's quite the opposite, in my career one thing I have learned is that I don't know it all and there is always more that I can learn. What motivates me is going through the process of learning with someone else.

Monday, December 1, 2014

Why is getting your data on a new phone so much work?

Recently my wife upgraded her phone after finishing her two year contract with our mobile provider. She transitioned between phones on the same carrier made by the same manufacturer.

For some context, my wife's primary email comes from a standard IMAP server. She gets her calendars from a standard CalDAV enabled server. She gets her contacts from a standard CardDAV enabled server. She downloads her music and files from a standard WebDAV server. She installs her applications from two app stores, Google Play and Amazon Appstore.

It took us over 4 hours to transition everything from her old phone to her new phone. Why in 2014 is this still so cumbersome?

What transferred/setup without any work

  • The applications installed from the Google Play Store.
  • GMail.
  • Home screen background image.

What we had to manually transfer/setup

  • Applications that were NOT installed from Google Play Store.
  • IMAP email.
  • CalDAV calendars.
  • CardDAV contacts.
  • Lock screen background image.
  • Phone PIN.
  • Phone home screens
    • Widgets.
    • Application Shortcuts.
  • Alarms.
  • Application Settings.
  • Her camera pictures.
  • Her downloaded music.
  • Her downloaded files.
  • 3rd party application data (Instagram, Facebook, Pintrest, and etc).

There's nothing on the second list that couldn't have been automatically transferred. I'm not sure what the right solution is to this problem, but I do know this shouldn't be as much work as it was.

As technologists we put way too much on the shoulders of our users. We expect them to do the heavy lifting for things that we can do easily through software. I think part of this problem is that we, as an industry, don't think enough about the import/export scenarios for our mobile products. But that's sad given that most people are on 2 year contracts with their carriers and they have an opportunity to upgrade their phones if they can financially afford it.

In my opinion this is real opportunity lost.

Monday, November 24, 2014

Ubiquity of data

I've been thinking a lot lately about ubiquity of data, or really the lack of it in today's modern technology.

In the 90's and 2000's most of our information lived on our primary machines, whether that was a desktop or a laptop. As an industry we spent a lot of time and resources trying to make that information portable. In the 80's it was the Floppy drive. In the early/mid 90's it was the Iomega Zip Drive. In the late 90's/early 2000's it was the recordable compact disk. In the mid to late 2000's it was flash memory in the form of a USB stick. All of these technologies focused on one thing, making it easier to move data from one place to another.

In the late 2000's/early 201x's we started to talk about shifting our data to the cloud. The thought was that if we put our data in services like Amazon S3, Amazon Cloud Drive, Dropbox, Box, Microsoft One Drive, and etc. that our information would be ubiquitous. In a way we were right in that we can now access that information in the cloud from anywhere. But fundamentally we're still thinking about and interacting with data as something that we move from place to place.

I think as an industry we need to stop thinking about data as a thing that we move from place to place and instead solve the problems that prevent us from accessing our data from anywhere. So what are the problems that we need to solve to make this a reality? This list is by no means exhaustive, but it's where I think we need to start.

  • Federated Identity Management.
  • Data Access Standards.
  • Networks that do not model their business on whether you're a data consumer or provider.

Federated Identity Management

In the world we live in today, each service (i.e. company) owns authenticating who we are. I.e. they keep a proprietary set of information about us that they use to test us with. If we pass the test they considered us authenticated. Most of these tests come in the form of two questions, what's your name and what's your password.

The problem with this is that it takes identity authentication out of the hands of those being identified and puts that into the hands of those wanting to authenticate. There's nothing inherently wrong with wanting/needing third party validation. The problem comes when we have hundreds of places we need to authenticate with, each with it's own proprietary method of authentication. Not to mention that it passes the buck to the user to remember how each one of these services authenticates them.

Tim Bray has a good discussion on federation that you should read if you're interested in the deeper discussion of the problems of identity federation.

Data Access Standards

We need data access standards that any group (for-profit or not) or individual can implement on top of their data that allows any other system (using the federated identity management) to interact with it. These standards would define CRUD operations (create, retrieve, update, and delete) in such a way that any other system and interact with the data on that system on the users behalf.

We have a good start to this with standards like OPML, RSS, WebDAV, CalDAV, CardDAV, and etc but these standards aren't cohesive. On top of that we don't have a real way to query a service to see what type of CRUD operations it supports. If we had the ability for the service to state what it serves then the clients could more intelligently interact with that service. Currently we put the onus on the user to know what a service offers.

Networks that do not model their business on whether you're a data consumer or provider

Right now the people who provide us access to the internet think about us in two categories. The first category I'll call "data consumers" and the second category I'll call "data providers".

Data consumers have the ability to get things from the internet and put things somewhere else on the internet. But data consumers don't have the ability to provide things to the internet without putting it somewhere else.  A good example of this is email. A customer with a standard "data consumer" internet connection cannot run a mail server for two reasons.

First, they get a dynamic IP address from their their ISP (internet service provider). This means that the address from which they connect to the internet is always changing. Think about this analogy to a dynamic IP address. What if your home address was constantly changing either daily, weekly, or monthly. It would be impossible for anyone to contact you via the mail reliably because anytime your address changed mail sent to the previous address would be delivered to the wrong house. It's the same way on the internet. If you want people to be able to talk to you you need to have a static address for them to contact you.

Second, ISPs block the ports necessary for others to talk to you. Even if you had a static address, often your ISP blocks standard email ports (25, 993, 143, 587, and 465) because they're trying to stop spammers from easily distributing their spam. But as anyone with an email address knows, the spammers are doing just fine even with the ISPs not allowing incoming connections. So I don't buy this as a valid reason to block these ports.

Data providers have all the same access as data consumers except they pay more to have static IP addresses and to not have the ports blocked. Notice anything wrong with this situation? The ability to fully participate in the internet is based on how much you pay your ISP. ISPs hide behind the fallacy that they're trying to protect you in order to be able to charge you more for the ability to truly participate on the internet. Does that extra money you pay actually protect you or anyone else on the internet better? No. Most ISPs will probably tell you that your also paying for more reliability. But you're running on the same system as the data consumers, so I don't buy that argument either.

I truly believe that we're not quite moving in the right direction when it comes to solving these problems. Until we do, you will constantly be battling moving your data from one place to the next when any new interesting service comes into existence.