Abstraction: The Sunshine of your Code

As part of a "back to basics" kick I'm currently reading Structure and Interpretation of Computer Programs. This book is a gem and although I'm only through the first two chapters I've been fascinated several times by fundamental and obvious programming concepts that I seem to have overlooked as my career progressed beyond those carefree University days. I'm particularly absorbed in the author's consideration of abstraction.

Abstraction is something I have personally taken for granted even in University. Recalling lectures where profs used analogies of the abstraction of cars or TVs I approached the topic with the blasé "Ya, ya, ya we get it... lets get to the code!" attitude. Now seven years later this book has caused me to reflect upon abstraction and realize that it is the fundamental ingredient of nearly every advancement in the construction of software.

The core idea is in creating Abstraction Barriers to speak a different language in each layer. In doing so, we can work most efficiently with the software itself and more importantly with other humans (of which most are computer language illiterate). To steal the authors' example at the highest level we can all speak about rational numbers, create them, add, subtract and multiply them. Anyone can "program" a calculator to make rational numbers dance. But there's a special class of people we try to avoid at parties who know that a rational number is comprised of a numerator and a denominator. Further there are the computer scientists who would look at a rational number as some kind of pairing of two integers. Now consider that each computer scientist may choose to implement a rational number a different way. Does this change the way the mathemetician's work with numerators and denominators? Does it change the way the average person does math? Absolutely not. But it may be the reason us computer science nerds don't even get invited to those parties.

In essence Abstraction Barriers are the driving force beneath Design by Contract and Interface Driven Design. Object Oriented languages are based on abstraction as there is no inheritance or polymorphism with out it. Even the often abused principle of encapsulation is about protecting the abstraction barrier. Abstraction is the foundation of every OO design pattern ever invented as a design pattern is just summarizing algorithms into a vocabulary. Even the cornucopia of patterns that emerged from Fowler's PoEAA and Evan's Domain Driven Design are again just abstraction barriers applied.

Consider, for example, software engineering tenants such those found in the Domain Driven Design camp which are some of the most powerful tools introduced to software engineering. Layered architectures and Bounded Contexts compartmentalize subsets of Ubiquitous Language into abstraction layers. Each layer or bounded context has a particular language used throughout. For example, Persistence Ignorance is keeping the implementation of physically storing data out of the realm of those who speak about domain and into the realm of those who speak about databases. Knowing the power of abstraction its possible to take the fundamental concept and apply it to your own design techniques.

Quite a while ago I learned the exercise of anthropomorphising software components to help break down software design. I've just recently started to consider abstraction barriers as part of the excercise with great success. For instance, I'm always running into the nagging question of "Where does this particular piece of code belong?". Now, while playing the part of each object in the orchestration I pay attention to the language being used (that is the verbs and nouns of each component's vocabulary) along with the data in play and try to find the lines between abstraction layers. It becomes quite obvious when you've crossed a barrier as the language will change and finding where a piece of code belongs is a simple as identifying components that have the same vocabulary.

For example, on my current project we have a 3D control that works in terms of Geometries. We also have components that can be drawn in this control such as Pipes, Elbows and Tees. The 3D control works entirely with Geometries knowing nothing about Pipes, Elbows or Tees. Conversely Pipes, Elbows and Tees know nothing about Geometries. Here we have two abstraction layers, lets call them the 3D layer and the Domain layer. If I find myself with a piece of code that does something with Geometries its fairly obvious that it should be grouped with the 3D layer. Or more likely I'm tempted to throw some code that works specifically with Pipes on the 3D control. During an intense anthropomorphise excersise this would raise a red flag as the 3D layer doesn't include Pipes in it's vocabulary. I should relocate that code to a service in the Domain layer so the language is consistent. I could then inject the service into the 3D control so it can call on the service to achieve its goal.

As you can see abstraction is a powerful tool for software engineers, and definitely not a concept to be overlooked. So for you kids out there, don't be like me. Treat abstraction with the respect it deserves. For those out in the industry who have forgotten about it whether you read SCIP or not give abstraction a second thought.

If you're interested you can also watch the videos at http://groups.csail.mit.edu/mac/classes/6.001/abelson-sussman-lectures/. They're worth a viewing if at least for a chuckle at the stylish '80s stashes.

Using Feature Branches

One of the biggest changes we've made to our development process in the past month was to introduce feature branches. Historically our team ran with two branches: the trunk and a production maintenance branch. All new development occurred on the trunk while bug fixes were committed to the maintenance branch and merged down to the trunk. This was a somewhat successful approach in that it's simplicity was perfect for a team that had little experience with version control but it suffered one annoying problem: the trunk was largely unstable.

The first indication that we had a problem was that deployment became a timely event. We would wait until there was low risk that the trunk had bugs in it, close our eyes, pray to our respective gods and slam our fists on the big red deploy button. Sure enough there'd be a partially completed or untested feature in the package that would cause problems down the line.

This led to Code Freezes and the Scheduled Commits anti-patterns. Where around deploy time we'd have to postpone the risky commits in favor of the solid bug fixes. From a lean perspective this is a huge waste as there was code just waiting to be committed to the repository and a developer surfing Facebook waiting for Cruise Control to scream out "Woo hoo!".

So, we decided to try using feature branches for new development. A feature branch is simply a branch that has all of the changes from the trunk plus all of the changes required to implement a particular feature. When the feature has been tested and is ready to be released we merge the branch back to the trunk and delete it. The end result is that the trunk stays stable even while new development is occurring because the feature branches are the ones assuming the instability.

Remember: Branches are like puppies so don't create a branch unless you're prepared to take care of it.

The maintenance costs of a branch are that every day or two a developer has to merge all of the changes from the trunk up to the branch. With Subversion 1.5 this is simple task but you should be aware that it could result in a couple of problems. First, you will likely run into situations where a single file changes on both the trunk and the branch resulting in a conflict. These conflicts are no different than the conflicts you receive while doing an svn update and can be resolved as such. Another more severe problem is that changes can be lost if a file is removed on the trunk but changed on the branch. Unfortunately there is no easy way to detect this other than to scan your merge log for deleted files. I have not yet personally experienced this problem but I am hoping that if it does occur my test fixture will catch it. Finally, performing a large merge is much harder than small incremental merges, so merge often! Depending on your team size and the amount of change on the trunk you'll want to tweak your merge frequency. My team is currently merging every day or every other day depending on the state of the trunk. Plus we always make sure to merge the trunk to each feature branch after another feature branch has been merged down to the trunk.

Remember: Merging works with a range of revisions so when merging from the trunk to the branch you need to include ALL revisions since the last merge not just the HEAD revision.

Subversion mergeinfo tracking in SVN version 1.5 makes feature branches a breeze. I remember using feature branches under early SVN builds (and even CVS ) where we would have to track which versions were brought up to the trunk on an index card so we wouldn't duplicate efforts. Now the mergeinfo is tracked inside of SVN properties and SVN is smart enough to handle these tedious tasks for us. So just make sure your working directory is clean and from it run:

svn merge svn://svnserver/myrepository/myproject/trunk

Sometimes, however, you will want to know which revisions have been applied. In this case you can run the following command from your branch working directory to get a list of revisions merged up to the branch:

svn mergeinfo svn://svnserver/myrepository/myproject/trunk

Similarly you may want to know which trunk revisions are eligible to be merged onto the branch. In this case you can run:

svn mergeinfo svn://svnserver/myrepository/myproject/trunk --show-revs eligible

Remember: A small incremental merge is far easier than a massive merge, so merge often! Don't wait until the end of the feature to sync up your branch. You WILL feel pain.

The final task in feature branch maintenance is after the feature has been developed you will need to merge it down to the trunk. This is a step in our Kanban that occurs after Peer Testing but before UAT. In other words we are committing zero defects (as far as we know) to the trunk. Since our branch already contains all the changes from the trunk that occurred during the development of the feature the only changes that need to be merged down are the ones that belong to new feature. The beauty of keeping your branch in sync is that this happens natually in SVN 1.5 by running the merge command with the reintegrate flag from a clean trunk working directory:

svn merge --reintegrate svn://svnserver/myrepository/myproject/branch

Once the branch has been merged down to the trunk you will want to make sure that no more change occurs on it. This is accomplished by simply deleting the branch. It will remain in your repository so you can always browse back to find those changes but developers will not be able to check it out and commit against it.

Thats about all the mechanics there is to feature branches. But the question remains how do we know when we need to create a new feature branch? This is more of a fuzzy area. Some teams branch for every Story but the granularity of these branches is really dependant on what and how often you will be releasing. In general my team will create a feature branch based off a portion of a Stories in a Theme or Epic that should be released together. We call these chunks Minimally Markettable Features and we choose each one so it takes about an iteration to complete. So our feature branches deliver a useful feature to the business every 2 weeks.

The early results show that feature branches have greatly improved our ability to deliver stable releases and in turn make our client much happier. Our testing efforts are far more efficient since UAT now contains stable changes and there is no waste in developer cycles due to Code Freezes or Scheduled Commits. The downside is that feature branches require much more SVN savvy. To successfully implement feature branches your development team should be fairly experienced with version control systems. If your team is struggling to manage a trunk and a single maintenance branch then I would recommend mastering that before using feature branches.

If, however, you feel your team is ready for the next step and would like to implement feature branches I highly recommend reading the branching topics in the Subversion Book. Also get your hands dirty and play around with branches in a sandbox svn repository. You'll quickly get a feel for how they work and what they're all about. Lastly, drop me a line if you're have any questions.