Endless litany of angry-customer or deluded-coder jokes aside (and believe me, I’ve heard them all), the battle between “it’s not working right” and “it wasn’t designed to be used that way” is an important one to keep having. Not that there’s any danger it will stop any time soon, you understand.

When I talk about tool transparency, my typical example is a pencil. You did not need to be instructed how to use a pencil - that is, you didn’t need to be told that you could make marks on things with its business end. Even a very small child gets that without being told. Where did the child get it? She either got it via osmosis - by watching her parents or siblings use one - or by trial and error - messing around with it until, hey, a useful feature of the pencil became revealed (usually on the walls).

But there are levels here. It’s possible to understand basic function and still have to learn subtleties, even with something like a pencil. In case you don’t think so, here are some higher-level things you had to learn about a pencil:

Safety. Don’t stick it in your ear. Don’t run with it; you’ll trip and put an eye out, kid.

Additional features. While scribble mode seems pretty obvious, eraser mode takes a bit more work to discover.

Fluency. Anyone can make marks with a pencil; not everyone can write legibly with one.

Maintenance. Once you learn to use the pencil you will inevitably have to learn to use the sharpener.

The pencil is an open-use tool. Pencils do not come with a warning card that says DO NOT USE THIS TO DRAW PICTURES OF SPARKLY UNICORNS or DO NOT USE THIS TO WRITE BAD FANFIC (although you could make an argument for the latter). For better or worse, no one tries to constrict or control what you do with a pencil.

And yet, most people intrinsically get the limits of a pencil; they don’t try to use a pencil to pry a strongbox open or stir a pot of soup with. You could attempt to use a pencil for both tasks, but your results would range from unsatisfactory to outright failure. But people seem to have no trouble knowing the pencil’s limits.

Now consider, say, something like Microsoft Excel.

The underlying concept of Excel - the ability to create, store, arrange, and calculate on columnar/grid data - is perfectly sound; and at its core Excel is a fine system with many uses. Yet I have never yet used Excel without stumbling upon one of its eccentricities, something I can’t do because I can’t figure out how to do it or because I am seemingly not allowed to do it, or something that simply doesn’t work the way I expect. Excel and I have never gotten along, and since I am what I would consider to be a very software-savvy user, highly experienced and quick on the uptake, I can only conclude that Excel is not designed for me, not designed for the way I think.

Reinforcing this: I’ve noticed other programmers have the same problems with Excel that I do, but the typical-office-users I’ve asked about it don’t. The difference is, they use Excel the way Excel tells them to use itself - they build their tasks around the interface - whereas people like me go in with a task and an expectation that they can make the interface do the task they want. We assume the tool can be used for the job; but Excel is built around the idea that the jobs will conform to certain decisions already made, beyond our control, about the design of the tool.

Viewed at the most simplistic level, Excel is no different from the pencil: It has a few things it does well, a few things it does adequately, and a lot of things it wasn’t meant to do at all. And yet it seems like we have taken a big step backwards from the pencil here in terms of user frustration. Is it mere increase in complexity, or are there other reasons why this has happened?

The answer is twofold:

1. They made it more frustrating in order to make it less frustrating.

2. They can only build one product.

Both of those are cryptic, I understand, but have patience.

Excel is a lot more complex than a pencil. The number of things you can just sit down and do with Excel by osmosis is very small; there’s simply too much going on. Sure, you can easily get the idea that there are all these little cells in a grid and you put information in them, and if you were only ever using Excel to store literal data in a grid, you might get away with just that. But sooner or later you would want to rearrange structure - move one column to a different location, or delete it entirely - or calculate values - fill in something in one part of the grid based on data in another part. And neither of these is intrinsically obvious.

Oh no, they aren’t. You may think they are, but your thinking so rests on assumptions that you have already learned.

Basic grid manipulation in Excel runs off the right-click menu and specific ways to select things. Both idioms - “clicking a cell selects just the cell, clicking a row number selects the whole row” and “right-click produces a local menu of actions specific to the selection” are things you had to learn. I train new users all the time who do not instinctively think to right-click things to see what happens. (And no, not all of them came from Apple-land.) Even an experimental user, like me, who routinely clicks on things to see what happens, would not necessarily immediately think to find options to insert or delete a row/column on the right-click menu … and as for moving or resizing a column, I still have not figured out the pattern that determines when Excel will give me a resize cursor, and usually have to try two or three times to get one.

As for cell formulae, suffice to say that as someone who uses Excel only about once or twice a month in short doses, I have never mastered them and will never bother to. Sure, I realize that A1 is the top left cell - the legend is built into the grid - but there’s a lot more to the notation than that and there’s no way to intuit most of it. Even trying to remember whether a cell range is specified with two dots or three is tricky if you don’t do it every day.

I’m not calling Excel out in particular. Believe me, I could use any other large piece of software as an example at random. It just happens that today I was trying to import a chunk of fixed-width data into Excel and after having to do the painful, fussy, easy-to-make-mistakes import routine a third time at about ten minutes a crack, marvelled that there was apparently no way to independently save an import specification, nor to edit it after the fact.

OK, but how often does anyone but a programmer or other wizard moving data between formats use that option? A telling feature here is that, when you import data in that way, Excel by default keeps its spreadsheet linked to that raw data source. In other words, if I have a file fixed_width_data.txt and I import it into Excel, Excel is aware of this connection, and if I change the contents of fixed_width_data.txt - assuming I always keep new versions of that file in the same location, name, and format - I can press a single Refresh button and have my spreadsheet magically update with the new contents.

Why is this telling? Because it says who the software was mostly designed for. Here are the hidden implications:

1. Your local wizard will set up the import spec for you. You don’t have to worry about that. Just make sure you save the raw file in [specific place/name your wizard will tell you] and all you have to do is press the Magic Button and not think too hard.

2. We didn’t bother making the import spec dialog easy to use because we knew you wouldn’t be using it. It’s not for you. It’s for the wizard, and he gets paid to put up with fussy, fiddly stuff like that. It need not concern you. In fact, we wish we could hide the button to get to it from you entirely, but in order to do that, we’d have had to put a special Wizards Only Mode in our software, and that takes time and money and manpower.

“It’s not for you” is italicized above because that’s the key sentence. This stuff is actually hard, it’s complex, and if you just threw everything at the user all at once with no decisions about what to show, what to conceal, levels of availability, etc, you’d give most users brainfreeze. Even wizards don’t like to stare at “kitchen sink” software where every single item is thrown out in front of you by default like some all-you-can-eat buffet. It gives us eyestrain.

(I dislike a lot of open-source software because of this. Open-source software is built by committee and everyone thinks their pet features are most important, and no one is in a position to make executive decisions about what to make prominent and what to obscure. Result: dreadful mess.)

So you have to make decisions about which features to highlight and which to downplay (or even eliminate) … and if you’re any good at all, the key thing you will base those decisions on is how the majority of the users want to use the product.

In the case of Excel they have - correctly - surmised that most of their users are not wizards, that most of their users are not explorers, not try-everything-to-see-what-it-does types; they’ve tried very hard to nest content to make the things people want the most the blatantly visible ones and hide things people don’t use as often further down, in layers of dialogs and options, so they don’t get in the way.

The problem comes when a user comes along who wants to do something that doesn’t fit this personality profile. Maybe she’s a wizard. Maybe she’s trying to broaden her horizons. Maybe she’s been asked to do something unusual, as a one-shot, that she’ll never need to do again for the rest of her life. But she does need to do it - and she suddenly finds that it’s buried so deep that doing it is far more frustrating than it should be. The software has been made more frustrating (for her) to make it less frustrating (for the majority user).

But software companies cannot afford to make different versions of their software for different types of users. They can’t afford to make their software completely user-customizable (I choose what I want to see on my menus, etc), although some try harder than others. Some things can’t be eliminated entirely; if your software allowed you to completely conceal the ability to exit it, for example, sooner or later someone would get themselves in trouble. Giving a user enough rope to hang is dangerous. And of course even if you have layers, you can’t hide access to the lower layers, which is why Excel can’t completely eliminate the import features from their Data tab, or not even the wizards could find them.

The point to all this is that sooner or later you are going to find yourself trying to use a piece of software in a way that is either irritating, non-obvious, barely adequate, or any combination of those, and you may find yourself running into the realization that “it wasn’t built to be used that way.” Well, no, it probably wasn’t, and you’ll have to accept that. You probably just landed on the wrong side of the majority rule, as described above. But you should not accept that this implies any culpability on your part.

Software developers (myself not excepted) hate changing things. First off, it’s bitter work, usually work on code we were glad to put behind us the first time. Second, we know that any time we change anything we piss someone off. For every feature change someone urgently wanted and cheers for, three people want to kill us because “it worked the way they liked it before and we broke it.” So we resist. We say, “No, it wasn’t built to do that,” in hopes you’ll give up and just make it work using whatever methods we did give you. But you shouldn’t give up without a fight.

Sometimes your cry will indeed be unreasonable. I’m not going to build something special into software that has 500 users just to please one idiosyncratic one. But weight of evidence counts. If ten users of those 500 complain to me about the same thing, then I start thinking about ways I can build it in without breaking any existing functionality. If 50 users of 500 complain about a thing, I move it to the top of the list. If 100 of 500 users complain about a thing, I start wondering if I designed the damned thing badly in the first place.

Let your voice be heard. Keep us honest. It may very well be that our majority user isn’t who we thought our majority user was. And, never forget: This industry, as a whole, does not do nearly as much usability testing - watching and learning from how people actually use the software in their actual lives - as it should. In fact, the general lack of usability testing is the software industry’s shameful secret. So let us know. It may well be that we just never gave the matter in question much thought and did it any way that seemed serviceable at the time.

If I told you how often that happens, you wouldn’t believe me.