Tuesday, June 18, 2013

Why great software artists steal rather than borrow

"Good artists borrow, great artists steal", Pablo Piccasso

It's not a perfect analogy for software, but it sure gets your attention.   A different way to say this is "Don't borrow the ideas of other products, steal the building blocks themselves instead of rebuilding them."

Software teams are able to get new products and updates to market much faster than they were 10 and 20 years ago.   Much of the productivity gains have come from "stolen" building blocks that have been made open source or integrated into the development stacks that we use.

When I helped build Netscape Navigator nearly 20 years ago we had to start at a very low level and build everything up.    The networking libraries started with TCP sockets and even then there was not a consistent API.   The UI was built up from fonts, pixels, primitive menu API's and drawing commands.    When we wanted to display a GIF or JPEG we wrote all the code ourselves.   Everything was built using plain old "C" and we had to build all of our support libraries from scratch.    Simple list structures, hashing functions, time functions, threads, better memory allocation to deal with 16 bit machines.  We eventually built a cross platform support infrastructure called the Netscape Portable Runtime (NSPR) that encapsulated all the support code into one library and let anyone build tools on top of it.   We also gave it away, so anyone could use it.     Thousands of people over the last 20 years have been building similar libraries and making them available through open source for others to build upon.

Today when we build an application we can code the UI in HTML and let the browser figure out how to render it.   We can build the business logic in a high level scripting language that handles threading, memory management and a whole host of other complex low level problems for us.   We have access to thousands of open source libraries that can do incredibly complex tasks.    On top of all that we can even host our applications in the cloud and let someone else build, maintain and monitor all our hardware for us.

New code is still hard
UI's are easier, infrastructure is easier, memory management is easier, but coding is still pretty much the same.     We have lots of new buzzwords for software development these days:   Scrum, Agile, Extreme, Waterfall.   It all sounds very exciting, but I can tell you that team programming is still pretty much the same.    I have been doing versions of Agile programming for more than 20 years and we have now codified the methodology significantly, but the nature of the game has not changed very much.    New code is still hard.   It takes longer that we estimate and getting it bug free is very hard.   Keeping it bug free while extending it is even harder.

Don't build anything you don't have to
Every single thing you build will take about 10x the time in maintenance than the time it takes to build.   The things we build tend not to be as fully featured or as well thought out as a successful open source library and likely won't have half the features of the components you could have "stolen".   

So when do you decide to build something?
Deciding when to build something new versus using something that exists is really tricky.   One of the hardest parts is knowing what is out there.    There are literally tens of thousands of open source libraries that we could grab that do virtually anything.    If you want to make smart decisions about when to "steal" a component you need to educate yourself and get to know what is available.     It may seem like a lot of time spent researching and reading, but imagine how much time you are going to save by not having to build a major component for your development project.    And how much time you will have later on by not have to fix the bugs and add features because the component you found is already feature complete and stable.    Get on the internet and ask people if searching doesn't yield results.   

Spend all your time wisely
In a start-up, and in any company, but especially a start-up, you should spend 100% of your time building unique value.    Time spent reinventing the wheel does nothing to generate unique value.    Investors won't care that you built the coolest new version of crontab to control your back end processes, they care how many users you have and how much money you can make.  

Use your components wisely
Don't try to use components in ways they were not designed to be used.   First, you will likely find bugs that no one else has encountered because they have never used that component in that particular way.   Second, it probably won't scale and perform the way you expect.   

Don't try to second guess the design unless you are an expert
Understand and follow the best practices for each component.   You might not understand all the reasons for the best practices, but they are there.    If you want to use the Zend PHP framework you are going to have to use a model-view-controller paradigm.   You may think that it's too complicated for the simple website that you are trying to get done right now and you don't want to spend the time to learn how Zend's M-V-C model works, but you need to do it.    There are really good reasons why you should do these things, but they really don't manifest themselves until you start maintaining the code or decide to change the UI later in the product life cycle.   Or you might decide that SQL is way too complicated and design an intermediate language to sit on top of it.   If you think that's a good idea and your not a bona fide SQL and relational DB expert you are going to get in trouble.     SQL got complicated for a reason, trying to second guess 30+ years of iteration is very dangerous.

Don't use components that suck
I have a whole separate blog article on this one.  "For software development tools it's the ecosystem that matters most".   Make sure all the components and tools you use are bulletproof.    Time spent fixing someone else's code takes away from your product.

Give back
On a final note, I would like to encourage you to give back when you can.   All of the code that we "steal" comes from somewhere and wouldn't be there without the hard work and generosity of those who put it there.   Recognize that and give back to the open source community by contributing new libraries and enhancements and bug fixes.    I have open sourced almost all the code I have written and it has been a highly rewarding experience.