The broken web

So I spent the day looking at web design again. I don’t do much of it anymore, but Lethania does and so I tend to get pulled into it. I later ran into this ad, which I thought was kind of funny:

I would be much happier if everyone could start creating flash-free websites. Anyway, today’s ordeal made me think back to the fine old days when everything was a table. HTML wasn’t made to display layout. The nice things about HTML, hyperlinks, worked just fine in regular text documents with some simple formatting, like bold and italic, its creator figured.

The nice things turned out to be really nice though, sparking an incredible development where pages got increasingly sophisticated layouts. The original HTML text format included tables, a fact which was quickly used to hack together all kinds of pages. A table could be used as a grid to stuff things into certain places, and a table inside a table could be used to create more interesting layouts, not to mention a table inside a table inside a table.

Because tables were meant to be (you know…) tables, different browsers rendered them slightly differently. Fine, if all you want is a table of text. Not so fine, if what you wanted was a pixel-perfect design. The solution to this was to add a whole slew of properties to each table, row and cell.

So after hacking together a sophisticated web site layout using table, the result was predictably a complete mess. The tables holding the text in place was mixed in with the text itself, making sites a nightmare to update or maintain, to not even mention changing the design.

A solution was clearly needed, to separate the design from the contents of web pages. So, why not adopt a language that wasn’t meant as a design language either? Sure thing, Cascading Style Sheets (CSS), a language meant for styling rather than layout was adopted during a long and slow process full of bugs, browser incompatibilities and new hacks.

Today’s standard of “good” for the web means using HTML strict (which few do) and control the appearance using CSS. The result is slightly less convoluted than the tables approach, but takes about 20 times as much effort to understand or write correctly, involves just as much hacking and is so fraught with peril that most people simply avoid it, going back to tables or mixing in horribly ugly JavaScript. Or, as seems to be extremely popular with games companies, insist on making the entire website in Flash, even though there’s absolutely no need for it.

So the page we were looking at today needed 3 columns of the same height. Let’s do it the old way with tables, for nostalgia’s sake:

<table>
  <tr>
    <td>Column 1</td>
    <td>Column 2</td>
    <td>Column 3</td>
  </tr>
</table>

Okay. Not too bad. So how to do this with CSS? Generally, you’ll need 3 DIVs next to each other, all with float: left. This makes them each have an individual length though. There are several ways to get around this, but none of them are good. The first involves stretching images across the DIVs as background images, which means you’re now bound to the color of those images instead of a color value. Also, you can’t have borders.

The second way we tried involved a very complicated set of maneuvers using three extra panes and shuffling content out the left side of the screen and then back in. The description for how to do this was about 10 pages long. And oh, it turned out to not work with borders either. Sigh.

So finally we found one way that seemed to work with borders. Only it didn’t, since you didn’t get any bottom border that way. The good part is that you could hack around that by using an image.

So for the old horrible table version, you substitute a mess of nestled DIVs and several pages of CSS, and it doesn’t even work fully without adding a picture where you essentially paint the entire bottom of the border along the page (yes, you actually paint a snapshot of your entire webpage, 1 pixel high). Way to go.

<style type="text/css">
      #container { float: left; background: url(images/example-6.gif)
          bottom center no-repeat; padding-bottom: 1px; }
      #inner { float: left; overflow: hidden; }
      #inner div { float: left; background: #ccc; border: 1px solid #000;
          width: 200px; margin-right: 5px; margin-bottom: -1000px;
          padding-bottom: 1000px; }
      #inner .col2 { background: #eee; }
      #inner .col3 { margin-right: 0; }
      .clear { clear: both; padding-top: 10px; }
</style>
...
<div id="container">
  <div id="inner">
    <div>Column 1</div>
    <div class="col2">Column 2</div>
    <div class="col3">Column 3</div>
  </div>
</div>
<p class="clear">

That seems like a great way to code! Not to mention that part of your design is now locked up in the image.

The best part about all of this is that it’s presented as “without CSS hacks”. Well, if having to use 6 nestled divs, large-number positive and negative margins and all kinds of bullshit like that isn’t an ugly hack, then I don’t know what is. The fact that the art of web design has advanced to the point where the normal thing you have to do is one big hack isn’t encuraging. And then they add the hacks on top of that…

I never liked tables, but at least they worked. In one of those memorable IRC quotes (in Swedish), someone said:

<sycon> imagine 1000 ants, tables are like the cage that keeps them in place, in the right place
<sycon> css is like a child with spasms trying to poke them all into place with chopsticks.

After spending countless hours trying to do seemingly simple things, I think I agree. Something’s still broken with the web, and no one seems to care to fix it. Well, other than inventing more workarounds.

But hey, at least you can draw Homer Simpson with it. So, ok, let the flak begin. Look, a Parrot.

Wordpress image upload bug fixed

When upgrading my blogs to wordpress 2.9, I found that the image upload broke. After some fiddling around I managed to get a proper error message out of it, I managed to track down the error to the file wp-admin/includes/file.php, which used a ctype_ function that, for some reason, was disabled on my php version.

Funnily, the exact same bug has apparently been a problem on wordpress before, on version 2.5, in a different file.

Anyway, I fixed the problem. If anyone wants the fixed file, here it is:

Download file.zip

Download the archive, unzip the file.php inside and replace the one in your wp-admin/includes folder with it.

Web Form Verification for Dummies

The standard method for interaction with computer applications has gone from being the command line to being the native GUI, to being the web form. We were awesome at verifying input when it came from the command line — that was simple. Then we were kind of ok verifying input in the native GUI, although quality varied a lot more.

Now we suck at verifying user input from web forms. The current state of code that verifies user input has both managed to take us back to the kindness of the command line when it comes to freedom of input and manages to check all the wrong kinds of things. Why is it so hard to write these checks? I suspect because people don’t really think much about them, and I bet there are more interesting things out there than to write user input verification.

These problems aren’t some beginner coder errors either — they’re rampant on even the biggest sites out there like paypal.

The most common field to get me snared is the phone number field. In 99% of all cases, the site assumes that all phone numbers in the entire world are formatted like US phone numbers. Not, as one could imagine, because I’m claiming to live in America — I clearly just told the site that I live in a European country. So anyway, inputting my actual phone number causes an “invalid phone number” error. Not that there is any mention whatsoever on the site about what the correct format of a phone number is (there are, in fact, even several ways of writing a US phone number).

This sets off a wildly unamusing guessing game of how to “convert” my phone number into a format the site will accept. This practice often costs the sites money as I end up giving up and going somewhere else, frustrated and unable to make a simple online purchase that didn’t really require that phone number anyway, did it?

Passwords

Another highly amusing game is the one where web sites try to force users to choose “secure” passwords by enforcing the formats of passwords. “You must have at least 6 characters, with at least one letter and one number”. Sounds good, except in general these passwords are restricted to only contain letters and numbers. Hold on, isn’t it common wisdom to include at least one non-alphanumeric character in a secure password?

As such, out of my set of passwords, the only password which tends to pass most password verifications is my least secure one. The idea that you could fix a social problem through technology is somewhat funny anyway — “password1″ is not more secure than “password” in any way that really matters.

The same thing applies to the old trick of forcing your users to change passwords every month. This can have two potential outcomes — users append a counter to the end of their password, and increment it every time they are forced to switch, or they keep a post-it note taped to their monitor with their current password. Neither outcome is a net gain in terms of security.

Some sites even let the user set a password which is then considered invalid when the user tries to log in (ebay, for instance,  has done this) — causing a prompt for a new password and much annoyance.

Format wars

Parsing stuff is what computers are good at. So forcing me to input something in a strict format is always a loss. Either separate the fields and force me to select the individual parts of a date separately or actually use all that computing power at your disposal to do your user a favor. Telling me you have no idea what I mean by “2009-12-21″ because you expected “20091221″ is annoying the user for no good reason, even if you told me to not include dashes.

If you find yourself in a situation where you need to verify input  from the web, take an extra minute to consider how you could make things as convenient as possible for the user, which ones of your assumptions only hold true for the region you live in… and when you’re done, whatever you do make sure you tell the user exactly what the expected format is.

Black Box Black Box Testing

One of my friends is in College, and is currently feeling the full idiocy of a system that was only beginning to be rolled out as I left. Let me explain how it works.

Essentially, the system is meant to test the students’ solutions to homework problems. This is done by providing a solid definition of what the input and output of the application are supposed to be on the standard in/out channels, and setting up a whole bunch of test cases, including a memory limit and a CPU time limit. Students submit their source code to the system, which compiles it and runs all the test cases against the application in a black box test. So far, so good.

Seeing these guys at work, compared to me and my colleagues at work, makes a few things very apparent: even with a fairly solid grasp of algorithms and datastructure, their number one problem is code. Where professional programmers swim through code like sharks in the sea, the students appear to be more or less drowning. Theoretical learning aside, the education lacks practical programming, debugging, practical programming and some more practical programming.

It would seem that these programming exercises would be the perfect opportunity to get that kind of experience, if it wasn’t for the fact that the test system is itself a black box. You put in your code, and it tells you yes or no. It’s not quite a boolean pass/fail answer, but close enough: you will get told a result from the set: Didn’t compile, Passed, Failed, Crashed, Time Limit Exceeded. When I first heard of the system, it was motivated with the fact that sometimes in professional programming, that’s all you get.

I agree. Sometimes, you get gnarly bugs that give you less information than a world pro’s poker face. I’ve spent weeks tracking bugs like that sometimes, using all kinds of tools at my disposal to try to wring more  information out of the error, until finally the knot was untied. But — for all the bugs like that I’ve been through, none of them were eventually solved by guessing what was wrong and how to fix it.

Supposedly, the tool is meant to teach the students to debug their code… which it somehow does by disallowing all normal debugging tools. You can’t run a debugger on it, you can’t print traces, you’re not allowed to log to a file or socket, you’re not even allowed to know what input caused the error. The only tools you have at your disposal are your wit in coming up with your own test cases and code reviews.

Any attempts at normal debugging would be classified as cheating. If I was faced with a bug under those circumstances, I would do whatever I could to get more information out of it. Hey, I can crash it with different signals — that’s a few bits of information I could get back from it. All those kinds of tricks of the trade that real programmers use to, you know, solve problems… would be cheating.

This leads to a skewing of results… very simple bugs turn into monster problems, since you can’t identify and fix them. What they are learning is not how to debug their programs but how to painstakingly solve the very specific problem of pleasing the system. By artificially making easy things hard, the system has effectively found a way to avoid teaching the students essential skills in programming: simple debugging tools like tracing and breaking into a debugger. Instead, they learn programming by coincidence: poke something until you (hopefully, eventually) get a green light.

That’s not a lesson to learn.

The only way to go about this, faced by the obstacle made up of this system, is to learn a different skill: testing. More on that later.

More on studies: An Exceptionally Stupid Idea, Go Tinker, What’s a Good Final Year Project?

Don’t Be an Open Source Douchebag

I love open source software. It provides both a neat training ground for programmers, a good place to go scratch that itch. On the other side of things, it provides awesome software for people, including some software that would never come out of a big development house.

Still, there are some issues with free software that don’t really show up to the same degree with commercial software. One such thing is documentation. It’s painfully obvious that documentation is written by people who:

1. Already know the software in and out.
2. Don’t like writing documentation.
3. Know nothing about how people learn.

For instance, when I started a side project a few months back, I was looking for a build system. After settling on CMake, I set about trying to make sense of it. There’s the ever-present getting started example, of course. And then there’s the full reference of everything you could possibly want (almost).

But in between those, there’s nothing. Well, nothing except a book, which just goes to show you that there’s something missing — a professional writer could obviously make some money out of explaining things in a reasonable way.

The problem with this is that it doesn’t match how people learn. Getting started is a good step, but a relatively small one. Most of the time will be spent incrementally expanding the knowledge, moving from beginner to expert. Most time will thus be spend in some kind of zone in between the “getting started” and “reference of everything” levels.

Worse than that, some open source programmers have a tendency to view their full reference documentation as an appropriate resource for everyone. “It’s all in there,” right? But pointing a beginner at a 40-page document detailing all the options of some application when all they want is to run it properly isn’t very helpful. I’m sure you know what I’m talking about if you’ve ever used an open source command line tool.

That ends us up with the really dark side of free software culture. The true douchebags out there will not only be extremely smartass in their RTFM comments, they’ll also be incredibly sensitive and defensive about the software they’re working on.

I ran into a problem with cygwin’s SSHD implementation last week. In searching for the solution, I found this mail list answer:

  Wrong.  That is uninformed speculation and guesswork.  Stop
spreading misinformation.

  Cygwin SSHD has had the support for fully logging in as any
user since 1.7, as you have already been told and completely
ignored.  Go and read the manual.  The link was in the previous
email I sent in this thread.

  freesshd works exactly as Cygwin *used* to before it got
subauth support: when you log in with a key, rather than a
password, you just end up as an admin user.

Wow. This kind of answer is wrong on so many levels. First of all, while he makes it seem like the functionality has been there forever, cygwin 1.7 is still not even out of beta. The chance that an end user has it is about 0. So, with the current version (1.5),  supposedly cygwin sshd works just like freesshd. This is clearly false, because the original poster reports one working and the other not (which is, by the way, exactly the same results that I had).

So, a user reporting a problem about logging in gets pointed to a long documentation about security settings in a beta version, doesn’t understand a word from that document (no surprise there), and as a result gets told to “stop spreading misinformation”. Truth is, simply installed like any normal user installs applications, one works and the other doesn’t, something made quite clear by an answer from the original poster in a different place in the thread:

> Are you talking about password or public key authentication?
> If the latter, Have you tried the LSA authentication package
> in Cygwin 1.7?

I don't know. I'll try to deciper that. Sounds complicated. In
the meantime, friend is using freesshd.

The essence of what he’s saying (which has been completely missed by the cygwin developers) is that the effort required to get cygwin to work like one would reasonably expect of it is much higher than the effort required to just google for something that just works out of the box. The fact that you could potentially make it work is irrelevant, because he’s not getting any help actually making it work.

He might as well just have said, “I don’t care about making it work for you. It works for me.”

Software companies usually compensate for their complete lack of useful technical support with a good (or at least reasonably decent) amount of help documentation. Free software usually has neither.

I encourage any programmer to practice their technical skills on an open source project. But while you do so, take the opportunity to practice your people skills a bit as well, or why not your writing skills? Don’t be an open source douchebag — someone reporting your software’s flaws is not attacking you personally.