Design Fundamentals: Abstraction

Design Fundamentals is my series on code design aimed at people who may have done some code in school, but haven’t done much code design, or who haven’t read much about design before.  This is the second article of the series.

Code design is all about making sense of a system. Any software worth mentioning quickly grows larger than your brain can easily track. A good design will ensure that each piece of the program can be safely worked on in isolation — that whatever you need to do, it’ll fit within your working memory.

Abstraction is a key component to keeping the amount of details you need to remember at once at a minimum. By abstracting a piece of code, you remove the details and only have to think about the abstract interface to the code, which should be smaller (or the abstraction was a bad idea).

So exactly how does abstraction work? I’m sure you’ve done some version of it before, maybe even without being aware of it. At a low level of abstraction, the code is filled with all the details. At a high level of abstraction, code uses concepts that hide details. So in abstracting code, we lift out the details to some other place — maybe a class or a function.

When to abstract

While you can debate any kind of rules for when to abstract (just like most things in code), I’ll attempt to give you some guidelines. Adapt them as they suit you and change them if they don’t make sense in your context.

For functions, what you can keep in your head is strongly related to what you can see on your screen. As soon as a function has you scrolling up and down, you’ve got a good sign that the function is too long and that you should abstract something away. For me, the maximum length of a function that is normally reasonable is about 50 lines — the exact number will vary with personal preference and the content of the function.

For both functions and classes, the relationship with other classes or functions also matters — specifically the fan-out (amount of other classes your class uses, for instance). You want to keep the fan-out low, since otherwise you risk forgetting the details of something while working on the code (or confusing the next coder who works with it). Steve McConnell has this to say about it in Code Complete (a recommended book if you’re interested in code and design):

Low-to-medium fan-out Low-to-medium fan-out means having a given class use a low-to-medium number of other classes. High fan-out (more than about seven) indicates that a class uses a large number of other classes and may therefore be overly complex. Researchers have found that the principle of low fan-out  us beneficial whether you’re considering the number of routines called from within a routine or from within a class.

The same thing applies with the number seven as with the above line count — use your judgement, but keep in mind when the fan-out is starting to increase that you might want to look at your design again.

Common abstraction mistakes

The number one mistake people do when it comes to abstraction is to confuse the need to abstract with the interface of classes that is exposed to others. Often, when abstracting a part of a function to a new function, it doesn’t need to go into the interface of the class you’re working with at all — it can simply be placed in an anonymous namespace at the top of the file you’re working on.

Another mistake that is fairly common is to treat abstraction as a simple cut and paste activity, where a number of lines of code are simply moved to another place. Stop to consider where to make the cut — what part of the code you’re working on is a logical unit that does something reasonable? If you end  up with a function named partOfMyOtherFunction() or something similar, you haven’t really achieved anything.

Keep thinking about your abstracted functions — do they share characteristics? Often you might find yourself creating helper functions that would fit better grouped together in a helper class. This class can also reside in the same file — there’s nothing which forces you to create a new header file and source file in order to abstract code.

Building abstract code

If possible, build your code in abstract layers from the start. Consider what parts you might need when implementing your functionality, and make sure implementation details are well partitioned off. Especially things like system differences and interfaces towards the operating system is good targets for an abstraction layer.

Encapsulate the system functionality in order to not have to bother with details everywhere in the code. This is especially useful when interfacing with badly designed APIs, to ensure that this doesn’t spread to all of your code. This lets the rest of your code think about the system functionality at a higher level, without the details.

Stay With the Flow

A lot is written in various places about programmer flow. It’s been called a few things: finding the flow, being in the zone, coding mode. Usually it’s mentioned when talking about programmer productivity in general, as in “get the programmers private offices, breaking their flow causes major productivity drops”, or sometimes some coder reflects “I was really in the zone yesterday when I wrote this”.

Not much is written about what the flow actually is though. Often it’s viewed as something near-magical, since even if you’re left undisturbed it can be incredibly hard to find the flow — but if you do, the results can be magical indeed.

But is it really magic? Let’s start by defining what it actually is:

Flow is the mental state of operation in which the person is fully immersed in what he or she is doing by a feeling of energized focus, full involvement, and success in the process of the activity.

To most people, focusing on something easy — sit down undisturbed and think very hard about that thing and only that thing. However, flow is something more — the feeling of energized focus and full involvement is a part of it, but the success factor is more important. You won’t feel flow unless you’re continually moving forwards.

Once we’ve understood that continual motion is a key to maintaining (and entering) flow, we can start looking at practical tips for getting into and staying with the flow. I’m not saying that these rules are going to solve all your problems, but they can get you quite a few steps on the way towards diving in with the flow.

Environments

Environments matter, but much less so than you’d think. Different people break out of flow differently and find varying difficulty in getting back into it, so find your own level here:

• Isolate yourself – it’s impossible to find the flow when continually talking to others or hearing conversations. Even if you’re sitting near other people, put your headphones on and put some music on.
• Get your equipment sorted — make sure you don’t run into any difficulties with the equipment, and that there’s nothing with it that annoys you. Small nagging annoyances can easily distract your full focus. Make sure your keyboard doesn’t glitch, make sure the editors and tools you use work smoothly, make sure you have good headphones that shut out the world around you, make sure there’s not going to be sunlight shining directly on your monitors in half an hour.
• Communicate your flow — tell people about it. When you isolate yourself it can feel hostile towards others, but trust me they’ll understand. Just tell them you’re going to be putting your headphones on because you need to focus on something.

When in doubt…

Flow breaks when you get stuck with a problem. Not having flow, it’s easy to get stuck on just about anything. These two factors are what stops many people from getting into flow, and since they’re interconnected it’s hard to break out of that loop — you’re stuck with something that’d be easy if you had flow because you don’t have it.

The good news is that this works the other way around as well. If you can manage to not get stuck and to always keep moving forwards, you’ll be in the flow before you know it.

• When in doubt, compile — often, while in the middle of writing something, you’ll run into some problem or other and get stuck. If you find yourself snagged on something for more than a few seconds, start the build. This may seem entirely unrelated to what you’re stuck on, but fact is you’re likely to have compile errors somewhere in the code you’re writing, so why not fix them now that your mind is busy figuring the problem out.This tip has two good properties. One is that you may gain insight on the problem you’re stuck on just by fixing the compile errors in the surrounding code. The second is that you keep moving, and thus you never get the same feeling of stopping what you’re doing, which can break flow.
• When in doubt, type — it’s easy to get stuck on a design problem and let that stop you from coding. However, there’s nearly always some typing to do — create a blank file for the class you’re thinking about. Fill in the file with all your template stuff that needs to be there (header guards for a c/c++ project, for instance).Now keep going — create a blank class. Type in a method signature (doesn’t matter if it’s not 100% correct) for a method you think you might need. Type in another. Create void implementations of the methods — maybe you’ll realize that you need something more, so add that in. Change things around. As you keep doing these small, seemingly mechanical tasks, you help your mind along the way by providing a framework for what you’re designing.

  Again, this has the dual point of moving forwards and helping your brain activity along. Sometimes, typing something into your editor and then immediately going “no, that’s not right” can save an hour of thinking (or even more, if you lost the flow and your mind wandered).

• When in doubt, find the loose end — sometimes you’re faced with a large amount of entangled code or design and you don’t know where to start. This tends to happen when starting new sections or subsystems, where the amount of work to do is large enough to cause you to not be able to grasp the entire thing up front. Maybe you don’t see the design for the system clearly, so you stop to ponder about it, losing flow.The best thing to do in this situation is to dive into the code. Find one strand and start untangling it. There’s bound to be some part of the code where there’s a small piece you know you’ll need. Start there. When you’re done with that, not only will the amount of remaining work be smaller, you may have found new insights into the surrounding code tangle, making it easier to keep untangling.

The overall theme is quite obvious: as long as you stay isolated from the distractions of the world and keep hammering at your code, even if it’s just doing mechanical stuff, you’ve got a large chance of finding the flow.

This makes it a lot less magical, and a much easier tool to bring out. Does that mean you should always do it? No, there are certainly points where the best thing to do is to stop and make sure you’ve really thought things through properly. That’s a choice you should make actively, not one left to magical chance.

Before you know it, you’ll be the magician.

Trying the Open Source Shortcut

Jeff Atwood relates the story of an anonymous open source developer, who has done code and design on several open source projects and wanted to use this as a good reference for job interviews.

One company seemed impressed with my enthusiasm for the job but it was part of their policy to provide coding tests. This seemed perfectly reasonable and I did it by using the first solution I thought about. When I got to the phone interview, the guy spent about five minutes telling me how inefficient my coding solution was and that they were not very impressed. Then I asked whether he had looked at the open source projects I mentioned. He said no – but it seems his impression was already set based on my performance in the coding test.

The anonymous programmer then goes on to ask if Open Source experience is overrated? I’m sure it isn’t, but just like Jeff writes, you’ve got to pick a good project – one that will impress me by its result. The thought that having done Open Source software development will somehow get you out of having to perform well on a coding test and on interviews is kind of peculiar.

In a recruitment situation, what you’re looking for is not only someone who’s skilled enough, but also passionate enough about getting that job to prove it. Joel Spolsky calls it pickiness. Now going back to what our anonymous coder wrote:

I did it by using the first solution I thought about

I’m not surprised about the result. Caring about the code you’re doing and the job you’re applying for shows, both on coding test results and on your cover letter.

Open Source development does look good on a resume to me, but I’ve also seem some people with impressive resumes completely fail at implementing a very simple coding test. I absolutely understand the recruiter in the quote above — if your coding test result is bad, I’m not going to take the time to give more than a cursory glance to your previous projects, especially if you can’t answer well about why you wrote it the way you did.

It’s also a somewhat weird expectation from his side. Reading code you don’t know is hard and takes lots of time so the chance I’m going to download your open source code and actually read through it is virtually zero. If you haven’t done anything that actually achieved something, why should I care?  Even if you have, why would that matter if you just proved to me that you’re not what I’m looking for? Trust me, reading the results of a coding test is hard enough work, and if you have a fair few candidates, you’re not going to want to do much more than that.

He continues:

One of the reasons I worked so hard on open source projects was to make job interviews easier. By providing prospective employers with large samples of publically available working code, I thought I would give them something more useful to think about than my performance on a particular coding test or whether the acronyms in the job skills matched my “years spent”.

I agree with Jeff that working on an open source development project is a good way to boost your resume — but it’s not a shortcut through having to perform well in all other areas. My tip is if you care about getting that job, put as much effort into the application and coding test as you would on any project code — open source or closed.

Design Fundamentals: Encapsulation

Encapsulation is one of the core design patterns of Object Oriented programming. The point of encapsulation is to split a large program into a number of smaller, independent parts to reduce complexity. In a sentence, encapsulation is hiding the implementation details of a module from its user.

The point of doing this is that it’s easier to use a module with a well defined interface and it’s easier to change the implementation if fewer things depend on it. If you expose the implementation of a module to its user, you can bet the user is going to end up in some way or other dependent on the implementation details. This means that the risk of breaking something increases every time you make a change to the module.

Why does this not become an apparent problem for a new programmer? There are two major contributing factors. One is that new programmers tend to write small to medium sized programs, and that dependency-based problems tend to become apparent only in large applications. In a small application, you’ll have a few users of your module, and when you break the implementation dependency, maybe one of them breaks. You fix the error and move on. Simply put: You don’t need design skills to write “Hello, World!” applications.

Experienced Software Engineers tend to write large to massive program systems. If you have a lot of things dependent on the implementation details you’re changing, chances are a lot of things will break, and maybe some of the errors won’t be apparent until much later. Even better, if all your components are dependent of implementation details of others, every change you make is virtually guaranteed to break something.

The second reason initiate coders fail to notice the need for encapsulation is that the errors that spring from this are delayed. There’s nothing immediately wrong with your code — it works. That’s why this is a question of design, not of code.

So how do you properly encapsulate code? Start by looking at your interface for the class. How does one interact with it? Consider what it means — does the interface tell you what it’s doing, or how it’s doing it? If you find it’s telling you anything at all about how it does things, consider what you could do to hide it.

Another useful way to think about a class interface in terms of encapsulation is “how could I break this object’s functionality“? If you find you could break it, something’s wrong with your encapsulation. A properly encapsulated object guarantees the consistency of its own state at all times. This way of thinking about a module leads to robust interfaces and code.

A good starting point for encapsulation is to make all your internal data private, and to expose only retrieval methods for those things the outside world has any business knowing about (languages with Properties like C# avoid this, but without them you’re better off doing this). Remember, one point of encapsulation is that you should be able to change the implementation without changing the interface, and if your internal variable are public, you can’t change how you store data.

There’s a tendency to expose other complex objects that your class owns by direct accessor functions as well. This is usually a mistake. In essence, you’re giving up control over these objects, which means you can no longer guarantee your internal state is consistent and you can’t switch to a different kind of object.

Consider an example of a logging facility that has an output stream. There might be a temptation to do something like this:

log.getOutStream() << "Testing, testing";

This is where you break encapsulation, becaue suddenly you have no idea what’s being done to the log stream. Maybe someone saved a reference to it somewhere? Is it safe to delete it? Did someone start a row and leave it unfinished (like I did above)? Dunno.

Also, you’re now stuck unable to change to logging over a network, onto a printer, into a GUI or whatever else could be useful to do (unless you want to derive your own iostream, which I wouldn’t recommend).

An option is to encapsulate the logging service fully, and only provide an interface to do things with (log text):

log.addTextRow("Testing, testing");

This way, whatever you do in addTextRow() is your own business, as long as logs get made.

As with most design issues, getting the design right when it comes to encapsulation is hard the first time around. You’ll almost never get it right the first time around. That doesn’t mean you shouldn’t try your best though — the reason you get it right the second time is that you notice what happened the first time you tried.

Many new coders tend to mix up project process with design process. Hearing about waterfall (and it’s negative sides), they say “I’m not doing waterfall”, which to them means not to do any design up front. This is a big mistake — think about your design all the time. When you sit down to write a new class, you should have a design idea in mind — trying to nail on design onto a kludge of code is a lot harder than to rework a bad design into a good one.

What does this have to do with encapsulation? It means you should make your variables private from the start, and immediately start thinking about your interface. That way, you’re not going to have to realize halfway through that you’ve missed encapsulating things. Nearly always, the key to getting the design right is to start thinking about the design.

When I recently sat down to implement a new feature of a side project I’m working on, the first thing I did was create 10 empty files, because I’d already thought the design through enough to know I’d need those classes. That doesn’t mean you should do all your design up front, and never touch it again (waterfall-style), it just means there’s nothing wrong with thinking about design before you start coding. In fact, I very much encourage it.

Design Fundamentals

I’ve been working on a new side project with a friend of mine. I’m thrilled to have found something that’s both an interesting challenge and something good to have on the side. It’s already teaching me new things that I would probably never have the opportunity to explore at work, for various reasons.

One of these things I’ve learned is that there are many things that I use in my coding are things that I take for granted but that are not natural or obvious to initiate coders. My friend is a coder who’s got some experience of coding through College, but I’ve realized that going from initiate coder to Software Engineer takes a lot of experience and that there are many things I’ve picked up along the way.

I got my share of that experience by creating a game engine in a startup, and doing a lot of other mildly complicated things. I had the fortunate position of having a colleague with whom I could discuss all design decisions to the bitter end, and the time to make a fair few mistakes and figure out how to turn them into successes.

What your path to getting that experience is, I can’t tell. Only you can walk that path, but at least I can set up some signs along the way to help. I’ll be starting a new series of posts called Design Fundamentals, where I’ll explore design concept that are fundamental to wrap your head around. Hopefully even experienced coders can gain something from these posts.

The posts will try to gather in one place only what a given design technique or pattern means, but also how to use it, why it’s useful and various tips and trips.

I’ll be using C++ as the language for these posts, as it’s my language of choice. Much of the information may benefit you if you’re coding in another language, and in fact the only recommendations I have for you when it comes to languages is pick one, and make it your native one. Become an expert in one language, don’t try to go wide. There’s a good point in learning to code in a few languages — but don’t make it your focus to become as good in all of them.

This post will serve as an index — all future posts in the series will be linked from here.

• Design Fundamentals: Encapsulation
• Design Fundamentals: Abstraction