Augmented Reality Interfaces — Nobody Prepared Us for This

We have all lived through a revolution in how we perceive information. Remember how it went. The internet appeared, but for a long time we couldn’t believe it. By “we” I mean society, business, the professionals of a newly emerging industry. But some 10 years went by (what am I saying — in places it still hasn’t happened) before it began to dawn on us that the internet isn’t quite “brochures, only on a screen” and isn’t “television, only the links work.”

We discovered new qualities: total memory and availability. Interactivity. Remember how everyone ran around with that meme? You could stick “interactive” onto anything at all — and it got better. An interactive book. Interactive cinema. Interactive advertising. Anything you like — even an interactive vacuum cleaner.

And a while later it turned out that these new qualities change not only culture, but business too, and the structure of society in general. A whole industry appeared to serve these new qualities — search engines, social networks, websites, advertising technologies. Specialists who were learning to make sense of all this, gradually turning IT magic into comprehensible business practice. Departments inside companies that were learning to use the newly invented tools. Remember? How long did society need to master that new environment? 20 years? 30? I’m asking because you and I have to go through it again.

Here are the technologies whose combination will change everything:

• Virtual and augmented reality;
• 360° video;
• Wearable electronics;
• Machine learning and, in particular, speech and image recognition;
• Blockchain.

I’m sure you know what these words mean, so I won’t dwell on them especially. What matters for us is that all these technology trends are at the pre-final stage of the Gartner cycle — they’re about to become widely adopted. They passed the research stage long ago. It’s a bit late now to invest in them in a big way. But “ordinary business” still doesn’t know how to use them at all.

It doesn’t know how, because we humans don’t understand how the environment is going to change. Once again we think that virtual reality is “a new kind of internet, only in 3D.” We think blockchain is “a new kind of money, only you don’t necessarily need a bank.” That machine learning is “like programming, only it looks more like magic.” And even about 360° video we think it’s “like ordinary video, only you can swivel your head.” And about wearable electronics we think it’s fitness trackers. And all of that resembles what’s coming exactly as much as the internet is “a brochure, only with little buttons.”

User experience will change radically. Which means everything will have to be built all over again:

• The technologies for attracting users;
• We’ll have to learn to design interfaces again;
• Learn to understand which events to look at in the statistics;
• Somehow figure out which business in which situation actually needs all these new digital frills, where it’s just a fairground attraction, and where it does nothing but harm.

Today we have two reliable ways to say anything to the digital world. The mouse and the keyboard. Both of these instruments are very, very good — they’re fast, precise and very well controlled by the user. So much so that people quickly learn to “think with their fingers” and stop being distracted at all by these imperfect gaskets between human and computer.

It’s all so good that questions come up: “Why create any further input devices at all? Why do leapmotion, kinect and other interfaces where waving your hands is enough keep appearing? What new thing will they bring to the user experience?”

Let’s start with what won’t happen any time soon. All the new input devices will be slower than the keyboard, the mouse or voice input for a very long time yet. Well, that is, some 20 years. Slower means that to hand the computer a text message, we’ll still be using a keyboard or dictation.

Oh, by the way, about voice input. There’s a decent chance that voice interfaces will squeeze typing out quite a bit. Very soon recognition quality will become really good, and I’ll be able to dictate my next article instead of typing it. But nothing radical changes at this point — you’ll still need to concentrate your attention, it’ll still matter that you look at the screen to check which letters are appearing. But for a long time yet, people used to typing will just do it “the old-fashioned way.” The way your grandmother still finds it easier to write by hand.

Here’s what could potentially change:

• The habit of checking what you’ve written will disappear. Voice input doesn’t strictly require a screen at all — it’s enough for us that we hear ourselves;
• The line will blur between “I am entering text on purpose” and “I’m just chattering”: from being strictly determined by technology it will turn into an agreement between you and the machine. You are always being listened to, but sometimes it’s assumed that you said that as a hint, that a reaction is needed. Right now, to switch input modality people use crutches — the direct analogue of a voice button, a command. But it’s already clear that everything is moving toward guessing;
• The environment will nudge you into recording everything you say;
• The combination with wearable electronics will mean you won’t have to speak out loud at all. That will take a few more years (around 5–7), but after that it will be enough to “speak silently” — the text will be picked up from your vocal cords, not from acoustic vibrations.

The voice input example shows that all the most interesting things start when several technologies are laid on top of one another — voice input + machine learning give us the ability to guess what the person meant to say. And progress in data storage removes the question of “record it or not.”

Where these technologies cross, a mass of niches will appear for new marketing, new entertainment and new forms of sociality. What exactly will grow in such a niche isn’t very clear yet, but I’ll give a few examples:

• Targeting by the tone of a spoken conversation;
• A voice “third participant” in the conversation, who prompts topics and facts;
• A huge niche for voice intermediaries (like in “Cyrano de Bergerac” — when one person speaks the words on behalf of another). It will all start with translators from one language into another. Which is to say, it already has. But it’s far, far more interesting when they translate “from one Russian into another Russian.” For example, a straightener of spoken speech that strips out the interjections and the swearing;
• A separate (and sinister) niche has to do with censorship and with the security sphere in general. Here, I’m sure, you’ll put two and two together yourself.

I could go on, but that’s enough for illustration. Voice input isn’t just a replacement for the keyboard — it’s a radical change in how we interact with the environment. The line will blur between “the user gave a command on purpose” and “the machine thinks the user wants something.” This will completely change the interface environment and the approaches to marketing.

What will happen is the thing marketers always dreamed of and the thing dystopian writers were horrified by, and the thing that’s easiest to think about in terms of “spells and magic” — society will intrude directly into thinking. It won’t even require neural interfaces.

Now we can come back to the even newer input devices — gesture and neural interfaces. And they too are in no way “a new mouse.” You could write more than one series of articles about the niches appearing around them, but what matters is that this is a completely different experience, a different environment for business and different interfaces.

One of the consequences most important to me will be the final birth of UX design. Today, however much specialists in the industry say that user experience designers are about experience and not about pictures, business and society don’t really believe them. But the newly emerging environment will pose the task of directing experience straight out — without the mandatory use of such a familiar crutch as on-screen forms.

I deliberately chose the voice input example to show that the most familiar way of interacting — the voice — will change this radically even without revolutions in how we receive the result the digital environment has prepared for us. How the evolution of screens into virtual and augmented reality will change us is a separate, large topic. But for that one, my main thesis is the same.

We’ll all try to interact with it the old way and stumble.

Today we’re once again living in “interesting times.” Everything is about to change, and windows of opportunity are opening. The technologies already exist, and it’s time to think about how business and society will use them.

Right now is the time for experiments — and the first of them will be clumsy marketing attractions that every media outlet writes about and that consumers perceive as magic. Actually, why “will be”? We’ve already started, and that’s exactly how it is. About our first experiments our clients say precisely that: “magic” something. A magic poster. A magic app. But all of it is still play — very important play. Humanity is learning to fiddle with the new toys, is gaining experience. The sooner you put these toys into the hands of business, the higher the chance that it’s your business that notices, stakes out and names the new niche.