There are some exciting new developments on the printer/paper front. If things go well, I’ll have something to announce next time! Please keep your fingers crossed.
Have you ever seen a pinball machine being scared?
Pinballs, already fascinating during their mechanical era, accelerated in complexity with the advent of microprocessors. Like many other things in the 1980s, pinballs too became computers, and this allowed them to reach new heights: music and digitized sound effects, small and then big animated screens, different play modes, remembering the state between players, and multiball.
But even aided with all this technology, there remained a fundamental challenge: most of the time, a pinball has no idea what’s going on. Its steel balls are not equipped with any positioning technology, and no cameras exist to alert the machine of their location. Solenoids, those primitive mechanical fingers activated by electricity hidden in bumpers and flippers, shoving the ball around, can’t predict where those balls are going to go.
The only connection to the real world pinball makes is when a ball touches or passes over a sensor every once in a while – with the most important of these sensors residing at the bottom of the play field, just past the flippers, telling the pinball the player has lost.
Like a general near an ancient battlefield sending couriers to report back with news, each pinball’s picture of reality is thus both incomplete and delayed. But since it’s a machine, the picture is also simplistic, realized in trivial binary code.
This is all okay for the purposes of a typical game, but a misfiring solenoid, a misaligned ramp, or a misadjusted spring are all it takes for the delayed, binary representation of reality to cease being enough – and nothing tests this more than when on an aging or poorly maintained pinball, inevitably, a ball does something it’s not supposed to do: it gets stuck.
As a player, you can sometimes spot the problem and shove the machine to attempt to dislodge the ball. But the pinball itself has no idea. The only thing that registers in its binary brain is “it has been a while since anything happened, and at a table slanted at an angle of 6½ degrees that shouldn’t ever be possible.”
And so, when a ball gets stuck, if you wait for a bit, you can see a pinball eventually getting impatient. Without any senses and really any understanding of the world, it tries to remedy this the only way it can: by progressively firing each of its solenoids in succession, hoping one of those spastic movements accidentally does the right thing, gets the ball rolling again, and restores the order to the universe:
⌘
Just like a pinball machine, a keyboard doesn’t understand reality, lacking a sense of sight or touch needed to be aware of its user’s fingers. Its only loose connection to the world is switches under the keys that tell it “the key has been moved down a certain amount,” from which the keyboard infers that “a human finger pressed that key and the computer should react to it.”
The limits of that inference are well known by anyone who’s ever owned a cat. But it gets worse. Most keyboards don’t know the actual status of every key, or even a delayed status of every key, but only something close to it. This is because keyboards usually interconnect all the keys via a two-dimensional matrix:
The way keyboard asks keys for their status is by periodically applying voltage to each wire column, and seeing if anything comes back – it only will if a key at a given intersection is currently held down:
Why is it done this way? Well…
A keyboard with 100 keys would normally need over a hundred wires. Laid out on a matrix, however, and you only need about 30.
Luckily, while computers are really dumb, they are really dumb really fast. This applies to modern inventions like machine learning, but it also helps with even a simple thing like a keyboard matrix. Go through each column fast enough – we’re talking microseconds – and you will be able to check in with reality at a speed that’s enough to accommodate even the world’s fastest typist, or the world’s most restless cat.
But there is a catch. It happens when someone presses more than one key. Some combinations are safe, like here:
But choose another combination, and “life finds a way” – the current from one column leaks back to another one:
This is known as “ghosting”: an appearance of a phantom key press from a key that wasn’t ever touched.
Any electric engineer can come up with a simple solution for ghosting. You only need to exchange each four-way intersection into a set of right-angle connections. This can be achieved by using diodes, which allow the current to flow in only one direction:
You can check how your keyboard reacts to many keys being held on a nice Microsoft ghosting demo site. I can already tell you it is not going to exhibit ghosting. But I’m also pretty sure your keyboard doesn’t have diodes, either. Only expensive mechanical keyboards do. Why?
Adding diodes is a bit more expensive than, well, not adding them. Most of the keyboards instead are equipped with a simpler solution that, like pinballs, recognizes that keyboards will never fully comprehend reality.
The diode-less keyboard’s software is made aware of all of the key combinations that are unsafe, and when it senses one… where pinball would start to panic, the keyboard just gives up, solving the problem by stopping to report any subsequent presses.
⌘
The pragmatic in me understands this. Pinball balls don’t get stuck often with good maintenance, and outside of gaming (and music!), keyboards don’t need flawless detection of many different keys being pressed.
Yet, it’s still disappointing, this indolence of machines, the fact that it’s all so simple, stupid, and unnervingly close to guesswork when it should be absolute certainty.
But in those limitations of machines there is also a fun opportunity. One of the most mind-blowing discoveries of my childhood was when my father showed me you can remove the glass shielding the pinball playfield, and play the game without the ball at all. Instead, you could use a screwdriver, a pen, or – if you are brave enough – even your fingers (brave because the angry solenoids kicking the ball around are strong and can actually hurt you!).
And with keyboards, it’s the reverse. If a keyboard knows nothing about fingers, that means you don’t need fingers at all.
This was actually once used professionally. Before computer keyboards came into their own, very early, room-sized computers were driven by repurposed typewriters, with solenoids put above keys, pretending to be mechanical digits:
We moved on from that, but the idea of mechanical fingers on typewriters is exciting enough that even recently, some artists and nerds have put it back to use:
And it wasn’t just typewriters. I’ve seen a few delightful experiments, for example this machine that plays the Chrome dinosaur game…
https://twitter.com/null4bl3/status/1247032404990210053
…or this one typing the famous 10 PRINT routine in a way where the lines between “automatic” and “manual” are blurred so much it’s no longer possible to say they exist at all:
You see that kind of messing with technology’s reality over and over again, in any place technology dares to enter. Record players don’t need to understand record scratching for it to work. Most cameras never agreed to double exposures, and typewriters weren’t born to create visual art. Messing with machines is fun and it’s more, too. One could argue that our collective creative experiments with ridiculing or disabling self driving cars are important tools of training for – excuse me getting very dark for a moment – humanity’s eventual survival.
⌘
But this portrayal isn’t totally fair. The creativity is there even without all these examples, just harder to see and appreciate. After all, someone else had to write the little piece of pinball software that made the machine try to get out of a situation that shouldn’t exist.
And… when I said “it’s cheaper” above, well. “Cheaper” also means “more affordable,” and “cheaper” often creates constraints that require more ingenuity than “expensive.”
Even simple matrices in real life show that space considerations and limits of the physical world can create challenges not present on cute animated charts. Here’s matrix for the keyboard of the TI-99/4A, already far away from a simple grid of squares:
And here’s one for ZX Spectrum+:
Even on relatively simple keyboards you can see literal strokes of genius – like here, where modifier keys like Ctrl and Alt have exclusive matrix lines so they can never cause ghosting:
On more complex or space-constrained keyboards, it’s even harder. There is both science and art to laying out traces to sneak around keys, go through over- and underpasses, and connect to the right switches at the right moment, in a pretty cramped space:
And here’s the matrix on the keyboard I’m typing it on – last year’s MacBook Pro – a genuine escape room where the traces can only go between the keys…
…the area around the spacebar looks like a traffic engineer’s career pinnacle…
…and, despite the lack of diodes (not available on mylar anyway), on this particular keyboard I can press more keys at the same time than on any MacBook I typed on before.
Sophisticated simplicity is different than dumb complexity. To me, this is wonderful. It’s in this area, this penumbra between design and engineering, I personally found interest first and career later. For as long as I remember, I have always been interested in human-machine interactions. That interest mostly existed in the “higher functions” of interfaces: the proper language of on-screen elements, the utility of typography, the emotions of colour schemes. Only much later came the realization that even on this ground floor, where senses and sensors collide, people and computers perceive reality in such different ways. It’s a miracle we can communicate at all.
In this space, I learned to love the drop-dead simplicity with which a pinball machine detects tilt:
In research for this book, I was delighted by Shift Lock being just that – a lock holding Shift in place, unaware its trivial skill serves a much bigger purpose:
I also loved how an old electric calculator will spin forever when asked to divide by zero:
And I was thrilled to discover yet another skeuomorph: the same solenoids that were used in pinballs were put in early computer keyboards, hitting the case from the inside, solely to make them louder and remind typists of noisy typewriters they grew up on (check out this video for a difference, it happens around 1:00):
Lastly, imagine my delight when I learned that the same idea of a matrix that’s used to sense a keyboard is used to draw on pinball’s displays:
⌘
The keyboard matrix feels simple today, but it could’ve been so much worse. Early typewriters struggled a lot with people overlapping their fingers, and their typebars got stuck often. A fascinating German teletype with a beautiful name Hellschreiber (don’t get too excited, Hell stands for the last name of its inventor) required the typist to press each keys at a precise rate of 2½ characters per second. You can watch it here – it’s frightening and mesmerizing.
Other teletypes not only forced you to decouple Return into CR and LF, but also punished you if you pressed them out of order, or too close to each other.
Later, early computer keyboards didn’t have n-key rollover (the modern name for properly supporting multiple keypresses) or even anti-ghosting measures. They also didn’t have a buffer that would help whenever the computer got too busy to process keystrokes – instead of storing your presses for near future, they would at best beep at you, and at worst physically lock the keyboard in a way that could actually hurt you. They achieved it with, you guessed it, solenoids.
It could’ve been so much worse. Learning all of this while researching the book was a surprisingly poignant and profound experience for me – a feeling of kinship with people I never met and I will never learn names of, but people who I felt would understand me and whom I would understand, people who decades or centuries ago worked on making this strange space that matters to me better, people who had to think like machines just so that machines could think like people.
I gave a talk to celebrate people like this once, and my book will too be a series of nods of appreciation to them. Because I’m pretty sure today that seeing my dad opening one pinball after another was a big part made me want to be an interaction designer, and to devote a hefty chunk of my life to trying to connect these incompatible universes.
Next time you find yourself playing with a solenoid, next time a keyboard locks up under your finger, or next time you ever see a pinball panic – this strange concert of paroxysms, this beautiful rhythmic progression, this programmed way to fix the universe that feels achingly human – maybe that’ll inspire you, too.
Marcin
This was newsletter №30 for Shift happens, an upcoming book about keyboards. Read previous issues · Check out all the secret documents