Trying the Open Source Shortcut

Jeff Atwood relates the story of an anonymous open source developer, who has done code and design on several open source projects and wanted to use this as a good reference for job interviews.

One company seemed impressed with my enthusiasm for the job but it was part of their policy to provide coding tests. This seemed perfectly reasonable and I did it by using the first solution I thought about. When I got to the phone interview, the guy spent about five minutes telling me how inefficient my coding solution was and that they were not very impressed. Then I asked whether he had looked at the open source projects I mentioned. He said no – but it seems his impression was already set based on my performance in the coding test.

The anonymous programmer then goes on to ask if Open Source experience is overrated? I’m sure it isn’t, but just like Jeff writes, you’ve got to pick a good project – one that will impress me by its result. The thought that having done Open Source software development will somehow get you out of having to perform well on a coding test and on interviews is kind of peculiar.

In a recruitment situation, what you’re looking for is not only someone who’s skilled enough, but also passionate enough about getting that job to prove it. Joel Spolsky calls it pickiness. Now going back to what our anonymous coder wrote:

I did it by using the first solution I thought about

I’m not surprised about the result. Caring about the code you’re doing and the job you’re applying for shows, both on coding test results and on your cover letter.

Open Source development does look good on a resume to me, but I’ve also seem some people with impressive resumes completely fail at implementing a very simple coding test. I absolutely understand the recruiter in the quote above — if your coding test result is bad, I’m not going to take the time to give more than a cursory glance to your previous projects, especially if you can’t answer well about why you wrote it the way you did.

It’s also a somewhat weird expectation from his side. Reading code you don’t know is hard and takes lots of time so the chance I’m going to download your open source code and actually read through it is virtually zero. If you haven’t done anything that actually achieved something, why should I care?  Even if you have, why would that matter if you just proved to me that you’re not what I’m looking for? Trust me, reading the results of a coding test is hard enough work, and if you have a fair few candidates, you’re not going to want to do much more than that.

He continues:

One of the reasons I worked so hard on open source projects was to make job interviews easier. By providing prospective employers with large samples of publically available working code, I thought I would give them something more useful to think about than my performance on a particular coding test or whether the acronyms in the job skills matched my “years spent”.

I agree with Jeff that working on an open source development project is a good way to boost your resume — but it’s not a shortcut through having to perform well in all other areas. My tip is if you care about getting that job, put as much effort into the application and coding test as you would on any project code — open source or closed.

Design Fundamentals: Encapsulation

Encapsulation is one of the core design patterns of Object Oriented programming. The point of encapsulation is to split a large program into a number of smaller, independent parts to reduce complexity. In a sentence, encapsulation is hiding the implementation details of a module from its user.

The point of doing this is that it’s easier to use a module with a well defined interface and it’s easier to change the implementation if fewer things depend on it. If you expose the implementation of a module to its user, you can bet the user is going to end up in some way or other dependent on the implementation details. This means that the risk of breaking something increases every time you make a change to the module.

Why does this not become an apparent problem for a new programmer? There are two major contributing factors. One is that new programmers tend to write small to medium sized programs, and that dependency-based problems tend to become apparent only in large applications. In a small application, you’ll have a few users of your module, and when you break the implementation dependency, maybe one of them breaks. You fix the error and move on. Simply put: You don’t need design skills to write “Hello, World!” applications.

Experienced Software Engineers tend to write large to massive program systems. If you have a lot of things dependent on the implementation details you’re changing, chances are a lot of things will break, and maybe some of the errors won’t be apparent until much later. Even better, if all your components are dependent of implementation details of others, every change you make is virtually guaranteed to break something.

The second reason initiate coders fail to notice the need for encapsulation is that the errors that spring from this are delayed. There’s nothing immediately wrong with your code — it works. That’s why this is a question of design, not of code.

So how do you properly encapsulate code? Start by looking at your interface for the class. How does one interact with it? Consider what it means — does the interface tell you what it’s doing, or how it’s doing it? If you find it’s telling you anything at all about how it does things, consider what you could do to hide it.

Another useful way to think about a class interface in terms of encapsulation is “how could I break this object’s functionality“? If you find you could break it, something’s wrong with your encapsulation. A properly encapsulated object guarantees the consistency of its own state at all times. This way of thinking about a module leads to robust interfaces and code.

A good starting point for encapsulation is to make all your internal data private, and to expose only retrieval methods for those things the outside world has any business knowing about (languages with Properties like C# avoid this, but without them you’re better off doing this). Remember, one point of encapsulation is that you should be able to change the implementation without changing the interface, and if your internal variable are public, you can’t change how you store data.

There’s a tendency to expose other complex objects that your class owns by direct accessor functions as well. This is usually a mistake. In essence, you’re giving up control over these objects, which means you can no longer guarantee your internal state is consistent and you can’t switch to a different kind of object.

Consider an example of a logging facility that has an output stream. There might be a temptation to do something like this:

log.getOutStream() << "Testing, testing";

This is where you break encapsulation, becaue suddenly you have no idea what’s being done to the log stream. Maybe someone saved a reference to it somewhere? Is it safe to delete it? Did someone start a row and leave it unfinished (like I did above)? Dunno.

Also, you’re now stuck unable to change to logging over a network, onto a printer, into a GUI or whatever else could be useful to do (unless you want to derive your own iostream, which I wouldn’t recommend).

An option is to encapsulate the logging service fully, and only provide an interface to do things with (log text):

log.addTextRow("Testing, testing");

This way, whatever you do in addTextRow() is your own business, as long as logs get made.

As with most design issues, getting the design right when it comes to encapsulation is hard the first time around. You’ll almost never get it right the first time around. That doesn’t mean you shouldn’t try your best though — the reason you get it right the second time is that you notice what happened the first time you tried.

Many new coders tend to mix up project process with design process. Hearing about waterfall (and it’s negative sides), they say “I’m not doing waterfall”, which to them means not to do any design up front. This is a big mistake — think about your design all the time. When you sit down to write a new class, you should have a design idea in mind — trying to nail on design onto a kludge of code is a lot harder than to rework a bad design into a good one.

What does this have to do with encapsulation? It means you should make your variables private from the start, and immediately start thinking about your interface. That way, you’re not going to have to realize halfway through that you’ve missed encapsulating things. Nearly always, the key to getting the design right is to start thinking about the design.

When I recently sat down to implement a new feature of a side project I’m working on, the first thing I did was create 10 empty files, because I’d already thought the design through enough to know I’d need those classes. That doesn’t mean you should do all your design up front, and never touch it again (waterfall-style), it just means there’s nothing wrong with thinking about design before you start coding. In fact, I very much encourage it.

Design Fundamentals

I’ve been working on a new side project with a friend of mine. I’m thrilled to have found something that’s both an interesting challenge and something good to have on the side. It’s already teaching me new things that I would probably never have the opportunity to explore at work, for various reasons.

One of these things I’ve learned is that there are many things that I use in my coding are things that I take for granted but that are not natural or obvious to initiate coders. My friend is a coder who’s got some experience of coding through College, but I’ve realized that going from initiate coder to Software Engineer takes a lot of experience and that there are many things I’ve picked up along the way.

I got my share of that experience by creating a game engine in a startup, and doing a lot of other mildly complicated things. I had the fortunate position of having a colleague with whom I could discuss all design decisions to the bitter end, and the time to make a fair few mistakes and figure out how to turn them into successes.

What your path to getting that experience is, I can’t tell. Only you can walk that path, but at least I can set up some signs along the way to help. I’ll be starting a new series of posts called Design Fundamentals, where I’ll explore design concept that are fundamental to wrap your head around. Hopefully even experienced coders can gain something from these posts.

The posts will try to gather in one place only what a given design technique or pattern means, but also how to use it, why it’s useful and various tips and trips.

I’ll be using C++ as the language for these posts, as it’s my language of choice. Much of the information may benefit you if you’re coding in another language, and in fact the only recommendations I have for you when it comes to languages is pick one, and make it your native one. Become an expert in one language, don’t try to go wide. There’s a good point in learning to code in a few languages — but don’t make it your focus to become as good in all of them.

This post will serve as an index — all future posts in the series will be linked from here.

• Design Fundamentals: Encapsulation
• Design Fundamentals: Abstraction

Let’s Make a Game Engine!

I occasionally get asked about the game engine I wrote back at independent games dev Limebird Entertainment. Specifically, people want to know how you start such a massive project. 100,000 lines of code later, it’s easy to point at each and every subsystem you have, explain what they do and how, and what the important design decisions were. You could probably even explain the reasoning that led to it being created at some point.

But where do you start? When you’re sitting there with a blank slate, what’s on the first row? The problem here is you’re starting something massive that you probably know next to nothing about. Do you open up your editor and create a file? Which file? Or do you start designing your architecture on paper or a whiteboard?

It’s a reasonable question to ask, and also an utterly pointless question to ask. My answer is “anywhere”. The important part is not where you start, but that you start — and that you keep going.

Identify one piece of code you need, and start writing it. You’ll be facing a million decisions anyway — what build system do you put all your stuff into to get it all to build on all your target platforms? How do you set up unit tests for your code? What version control system should you use? How do you design system X? How does that fit with system Y? All of these questions are things you need to deal with, and the only way to deal with them is one decision at a time.

One of the important things to realize is that the normal way of working in a development team doesn’t really mean much in this situation. How do you prioritize one component against another when you’ll need them all to get anything at all? Do you even know all the components you’ll need? You may think you do, but you’ll find out you don’t. The important part is to get going and keep going until you can start iterating properly on your code base.

Still, some tips:

• As your first code file, pick something you’re reasonably familiar with to get going. Regardless of the scale of the project, there’s going to be some small part of it that resembles something you’ve done before — do that first, to get up and running. It helps to be in familiar territory when compiling your first class, because that one first class is going to be where you get your environment up and running.
• Don’t over-engineer. Stay well within the bounds of what you’re reasonably certain that you’ll need. At this point, you want to keep moving to new parts of the project, so you can quickly reach a point where it starts doing something you want — so don’t waste effort on things you might need down the line. This sounds like simple advice but is actually quite hard, since you have no idea what you’re doing at this point.
• Accept that whatever you do will be replaced. You’re new to the things you’re doing. The first things you write will be incredibly uninformed and will undoubtedly be rewritten a few times as you learn about the system you’re writing. The important part is not that you write the perfect system the first time around, the important part is that you write the system and learn from it. Once it’s running, you can go back and fix it. Sometimes, the only way to get a system just right is to have done it once before.
• Make it do something. You can write core systems and design your architecture all day, and even with unit tests and all kinds of fancy techniques to prove to yourself that it works, you dont’ actually feel it until you have something that runs. Even if what you have is just a simple rotating triangle, the thrill of having a rotating triangle in your own game engine is a lot better than seeing the TriangleRotationTest pass.

So where did I start? We had a very brief discussion on how our design would look like, and what code we needed on a very high level, and then set off coding. I remember coding up our first version of a 3×3 and 4×4 Matrix math code on my laptop at my grandparents cabin in the countryside that summer. It was somewhat buggy, and I didn’t really know what I was doing… but it got the ball rolling.

Programming Culture

I decided to learn Common Lisp.

After feeling like I’d somewhat gotten stuck in my programming development, I decided I needed to do something well outside my comfort zone in order to get off the plateau. So after a quick look around, I decided Lisp was probably a good target.

Then I decided not to learn Common Lisp.

Here’s why:

I’ve learnt a fair few languages in my days… some have been incredibly useful but scorned by others like Visual Basic, some are popular and have a great following like C++ or Perl. Some are small but surrounded by friendly enthousiasts, like Lua. Some are just generally strange but well documented, like tcl. All of these are different experiences to learn, because not only do the languages differ but their communities differ.

It’s a lot like learning to speak a new language and at the same time discovering the exciting culture that comes with it.

But some programming languages seem to have a more or less arrogant culture surrounding them. They tend to be small languages that claim to be revolutionary yet no one uses them. They’re small and they’ve been small for a long time. Lisp falls into this category. One article I read strongly asserted that after decades, the time for Lisp was just about to come. Soon, Lisp would be the mainstream choice. Pity the article was 20 years old, and the use of Lisp has hardly increased since.

It’s weird that this is the case, if the languages themselves are so revolutionary as their followers claim. I’ve seen it with both Lisp and Smalltalk, and after going through it a second time, I think can explain why: Their programming culture has confused teaching with preaching.

And learning new stuff is really important when it comes to programming… which means teaching is equally important.

Let’s back up a bit… I learned Lua. If you try it, your first move will probably be a google search and you’ll find books online and help pages that in a friendly tone go “here’s lua, have a try, plunge in”. But when you try to learn Smalltalk, the same method will get you pages on why Smalltalk’s design is great. Starting to browse forums about Smalltalk, most of the discussions I ended up reading were about how God Alan Kay certainly didn’t mean for object oriented programming to look like C++ (“lol, what a thought”).

The same thing happened to me when I tried to learn Lisp. Finding several books online was not a problem, Google kindly sent me to a site on learning lisp, and following the advice on that site I started reading through a book online, and in parallel started reading a couple of sample chapters of another book. The sample chapters gave me nothing — some code examples, some interesting stuff, but mainly the focus was on why Lisp is so much better than all the other languages.

When I was halfway into chapter two of the book, it had still not finished motivating why Lisp was such a wonderful language, that the absence of a syntax really is the only way to extend a language and build bottom-up rather than just top-down and how you worked in a completely new way with Lisp and that all other languages were nothing but shallow copies and could your language really do this? And at that point I gave up.

The only feeling I get from this furious defense of the language is a feeling of uncertainty. Why do you need to tell me it’s so superior when all I want is to be shown how it works? Surely actually showing me would be the best way to convince me, especially once I’ve already taken the interest and initiative to go looking for a place to learn?

I have no doubt that Lisp is a good language. I’m still interested in functional programming languages. But I’d rather take on one where the culture and community is open-minded and busy doing kick-ass things with their language rather than telling others how kick-ass it is (or worse, how everything else is crap).

Oh well, F# looks interesting. In the meantime, I learnt a different language and did some really cool stuff.