Sensory Interfaces (SI)

I’ve always hated the term Natural User Interface (NUI) to describe the collection of touch, voice, and gestural interfaces. I think we need a new, better term for them. There are reasons for this.

As Don Norman rightly points out, there is nothing natural about “natural” interfaces , other than that we’re using our bodies. (We also use our bodies to move mice and tap on keyboards.) There is nothing “natural” about manipulating objects under glass; nowhere in the natural world can I pinch to shrink. Nor can I make a gesture in space that touches nothing and have it affect something else nearby (aside from a human). Or use my voice to command real world objects to perform actions (other than dogs, perhaps). So-called “natural” interfaces are, like all interfaces, learned interfaces.

My second objection to NUI is that it’s traditionally a Microsoft term. I have nothing against Microsoft per se, but the proponents of the term NUI have been traditionally people from Microsoft, many of whom worked on Surface (the coffee-table-sized touchscreen device, not the current tablet). I have nothing against those folks, but I don’t want to be identified as one of them either.

My third objection is that it sounds dated, old-fashioned, and HCI-wonky. NUI doesn’t replace Graphical User Interfaces (GUIs), so why harken back to GUI at all? Granted, it builds on GUIs, but the current methods of interfacing are a different paradigm of interacting with a digital device.

And what makes them different, yet binds all three (touch, voice, gesture) together? Sensors. All of them rely on sensors to power them. Yes, physical buttons can use sensors as well, but they can also be purely mechanical; you can’t recreate a touchpad, Kinect, or Siri without making use of sensors. The new means of interaction all rely on sensors to make them possible.

This is why I’m proposing Sensory Interfaces (SI) as a way of labeling this new cluster of sensor-powered interactions. SI (“ess-eye” or even “Sí!”) makes sens(or). It doesn’t make any claims towards naturalness; it doesn’t come with any one company’s baggage; and it’s pleasing. Sensory is close to sensual, or pleasing to the senses. Although it’s not meant for consumers, it’s easy to say and remember. It also has some precedents: a brief Google search reveals a few thousand mentions in mostly academic papers, so there’s already some traction there.

Ever since I wrote Designing Gestural Interfaces (on touch and freeform gestures) in 2008, I’ve struggled to find a good term for the types of interfaces we’re designing now that use touch, gesture, and voice as input methods. And so too, I think, has the industry. I think this’ll work. Give it a try.

NUI, no. SI, sí!

2 thoughts on “Sensory Interfaces (SI)

  1. I think natural is intended to mean the natural world that we can navigate with innate knowledge about it’s behavior. I think where we get let down is when the simulation is only partially complete. And my point would be it’s the simulation that matters, not just the sensors.

    Sensors themselves are fairly dumb. If you looked at raw sensor data you would see lots of noisy signals but no meaning. On my helicopter we do something called sensor fusion – bring together multiple sources of complementary signals to create a sense of position, orientation, and inertia. In a sense, we have simulation of a helicopter running inside the helicopter.

    When developing “Natural” UI’s I’ve always taken the path of developing a simulation as the internal engine driving the interaction. It keeps the UI honest since the state of the machine is limited to a set of internal rules designed to model a user’s expectation. You can’t make many arbitrary things happen since it will violate the sim’s rules and the user will notice when things break. For example, when a “paper” page is rendered as if it’s a book but it only does a page flipping animation in some applications and not others. The trick is to have a simulation that satisfies the complexity of the interaction without making too many of these exceptions.

    Tron, the movie, was a funny implementation of user input capture – disassembling a user bit by bit. But a non-destructive version gets really interesting. By capturing the user’s actions to a high level we can bring the user closer to the simulation, and in turn build a more complex and satisfying sim. UI’s feel less like moving through wireframes, and more like magic again.


  2. Since sensors are just the enablers maybe Integrated Interfaces is what describes this more “natural pattern” of interaction.

Leave a Reply
