8 Key Questions About Metaverses: How to Live and Work There, What to Pay With and What For — and Why All of This at All?

What is a metaverse? What views are there on this new reality?

“The metaverse will erase the boundary between the acting actor and the environment”

I come to this topic from the digital production side — at JetStyle we’ve been doing virtual reality for about 10 years. My background is UX design, so I look at every story as a set of user scenarios.

I think virtual and augmented reality are the mom and dad of mixed reality, which right now is quite rare in actual practice. I look at the experience that exists in virtual reality in 2021 as a forerunner of the experience that will exist in the future.

What interests me first of all is the evolution of input/output devices and how those devices will affect human–machine interaction. My main thesis: in the 21st century the boundaries of interaction are blurring. In the case of virtual reality, we’re talking about the states online / not online, present / not present, interacting / not interacting, directive and non-directive control, and many others.

We’re used to one reality being what’s inside the glowing rectangle and another being outside its edges. And for a very large number of people VR will be the first experience where the interface has no rectangular border.

Two years ago I was in a full-body VR arena for the first time — that’s when you observe the synchronized involvement of your whole body, legs included — and I thought that the main control by which input will be performed in VR is going to be our body, all of it. And from that comes a problem, which is also a place to research: right now most users and engineers conceive of our “human — machine” interaction as directive.

Right now a person thinks of himself as the “master of the machine.” The tools we use for input today are directive: when we enter something from a keyboard, a mouse, a trackpad, or by voice, we’re confident that what we entered is a signal the system will react to. Meanwhile, the main input interface is more and more becoming the machine’s or the network’s prediction about our next action.

In VR the means of input is how we move, and moving your whole body consciously is significantly harder than moving one hand separately or deliberately saying something out loud. And from that point of view the boundary of the “self” will blur more and more. Most people hold the idea that the “self” is what’s inside the boundaries of the skin. That concept can be revised and criticized already today, because often it simply isn’t so, with no VR involved at all.

The border zone between the actor and the environment is becoming ever more blurred, and with each next step it will be harder and harder to say where the acting actor ends and the environment begins.

What will be worth something in metaverses? What will people be able to pay for, and what will they pay with?

“Money in the new economy is a voucher for creative labor”

I think today’s economy has all sorts of currencies that people pay each other with. By currency here I don’t mean money specifically. Money is only one of them.

In the economy of the late 20th century, money by and large meant one person’s right to dispose of another person’s time, because access to qualified time was a scarce good that we distributed with the help of such counting tokens. And the trend I’m describing now has been with us for a long time, and in a world of multiple reality it will only accelerate.

In a world of information overproduction and ever deeper automation of labor across a large number of industries, the boundary between consumer and creator will blur.

There used to be a clear dichotomy: book author — reader, director — viewer, game author — gamer, and so on. And now a gradient of roles is appearing between the two extremes: the streamer, the let’s-player, the one who makes mods, the one who walks you by the hand through multiplayer games, and so on. And those roles are placed at different positions along that spectrum, from “I create everything” to “I only watch.”

This will lead to one of the main scarcities being a scarcity of creative labor that’s in demand. And since money gives access to what’s scarce, in a certain way money in the new economy is a voucher for creative labor. Maybe that looks like a theoretical construct, but Rideró, for instance, is a market built on this concept. Today readers’ attention is scarce, and authors have to put in effort to get it.

Which user scenarios can expect the most noticeable changes?

“The operator and the environment will swap places, which means absolutely all scenarios will change”

My first thesis concerns the near term and has to do with the HR revolution. I work as a lead tracker in corporate accelerators, and today every second startup, if not more often, appears in the HR field — because whatever industry you look at, the defining driver and the process that limits everything is remote work. Plus there’s demand for a hybrid work experience.

To make it clearer why this turned out to be so important, here’s an example. When everyone on a call is online, the interaction is equal. But when a group of people joins offline from a single device, that group is discriminated against in terms of the density of the information it provides — because no interaction interface yet exists that would have equal bandwidth for online and offline. And VR has an attempted answer to this problem.

VR is the only artificial environment that is single-tasking by design. If you’re running a training session in virtual reality and a player’s avatar moves, that means the headset is on their head and they’re engaged in the process.

If you’re inside virtual reality, it’s hard to ignore it. There is only one reality at a given moment. If you want to stop interacting with it, you have to take the headset off your head — and that will be visible from the outside.

And here I have a conflict with Zuckerberg’s vision. He believes that the syncretic nature of the experience and the level of media pressure should be carried over from social networks into virtual reality as well — that is, that virtual reality apps should be made multitasking. In my view that’s a monstrous mistake, because today’s world has a big shortage of focused interaction, and virtual reality is exactly what provides that opportunity.

My second thesis has to do with the grim prospect of predictive slavery. I think the main type of interaction will become prediction about our next step. Every one of us already has that experience: the search bar, for instance, guesses what you’ll enter next, the music streaming service guesses what you should listen to next, the online cinema what to watch, and so on.

There are two directions of evolution in this process:

1. The prediction step — how far ahead we can make a prediction such that the person considers it accurate and uses the result of the prediction as if it were their own choice. And the prediction step will keep growing.
2. The kinds of experience the network will prompt a person with. Today we’re prompted with the next step in a search or the next object in a feed (it might be a product, a film, or a news item). We’re also prompted with routes when we use maps. Very soon we’ll be prompted with what exactly we’re seeing, how to react to it, and what to pay attention to and what not.

And here you have to understand that if we predict someone’s behavior, we predetermine it. In the end, the roles of leader and led change places between the human and the network.

Right now we think the operator is the human who issues commands to the environment through an interface. But in the predictive slavery scenario, the environment issues the commands that steer the human on the other side. The operator and the environment swap places.

Once a device appears that’s light, acceptable to wear on the street, with no sensory break and with a powerful processor that instantly reads the changing surroundings, that’s when the moment of steering our behavior in real time will arrive. Augmentation will be prompting us with something.

And at that moment absolutely all scenarios will change. Up to now, however much we’ve said that interface design is the design of scenarios, it still gets perceived as the design of pictures. Now, at last, perceiving it that way will become impossible. The metaphor of the linear scenario won’t hold either. In that sense one of the handiest metaphors for designing interfaces and scenarios will be open-world games, in which we steer scenarios through a field of stimuli rather than through a linear sequence of scripts. And once we get into an environment like that, it’ll be hard to name a scenario that doesn’t change at all.

Which competencies will the market be desperate to buy for big money?

“Modeling a field of stimuli, designing new experience, creating digital objects”

The first ultimately important skill is the ability to steer behavior by modeling a field of stimuli and to build dynamic models of behavior. This skill is close to game design.

The second skill is experience design itself. New interfaces make it possible to create an entirely new experience of perception. There will be demand for changing a person’s experience by offering them means and scenarios of interacting with the environment that are impossible in “ordinary” reality but that can be conceived and understood. This will be one of the scarcest specialties, and it’s the one that will set the discourse.

The third skill is creating digital objects. I think a gigantic market will emerge, for models separately and for skins separately. That’s because digital objects in multiple reality will be able to move from one reality into another, which means we have to be able to design model skeletons that come in handy in different contexts. And on the other hand, we have to be able to make styles applicable to a large number of skeletons — so that it doesn’t matter which skin you stretch onto which model.

How will we preserve intention in a predictive world?

“To leave a person their intention, you have to find someone who will pay to protect the ecology of attention”

From the standpoint of information exchange, today’s economy is very bad for mental health. A large number of people have an interest in predictive slavery; plenty of them will pay for it. And here the question arises: who will pay to protect the ecology of attention? To leave a person their autonomy, you have to find customers for preserving it. And that isn’t a separate individual — one person won’t be able to do much. And in that sense we need to create entities that have an interest in ecological interaction. What they might be, I can only invent, and for now that would be science fiction.

My second thesis is more applied. I’m sure that because of growing media pressure there will be demand for managing the bubbles of silence around yourself — that is, how often I want to hear signals, how single-tasking I want to be, how long the pauses between utterances are, how big the contrast between signal and noise is. In Russian there’s no term for this phenomenon yet; I call it the design of silence or the design of emptiness — and it may be an answer to the problem of growing media pressure.

It should be said that the platforms have already noticed this problem: both Android and iOS have already produced curtsies in the direction of self-limitation, and there will be more of that going forward.

Apart from technical limitations, what could lead to the new reality we’ve been discussing not happening?

“If geo-thinking wins and we create a second Second Life”

First, geo-thinking may win. After Neal Stephenson wrote Snow Crash, there was already the game Second Life, and it didn’t take off, because back then we failed the interface task. We took the properties of one environment and carried them over as is into another, without rising into the Platonic space, without doing the design of the principle, without laying it bare, without reinventing it — and it all turned out to be a dumb experience. If this repeats and the new reality tries to behave like a geo-metaphor, everyone will get bored of it fast. One day Facebook will stop spending money on it, and that’s that.

And second, here I agree with Kolya that if we don’t solve the problem of cognitive noise, then we’ll get a storage site for the toxic waste of the products of thinking. And it’ll be hard to survive there.

Why all of this at all? Where will real progress happen?

“What matters is that people themselves start looking for meaning in it for themselves”

When we look for an answer to the question of what the benefit is, we have to ask: benefit for whom?

The benefit for big corporations and the state is the growth of the consumption economy. Plus it increases how controllable we, the users, are as objects that others make money from and manage.

The benefit for ordinary folk is getting experience that doesn’t exist in real reality. For example, I have virtual reality experience, so I can say that drawing some volumetric stuff in Tilt Brush together with other people is an incredible rush. There’s already a community around Tilt Brush, streams by three-dimensional artists, and so on, but for now it’s a small, only just emerging and very timid practice.

The benefit for the individual is the chance to create associations and invent sandboxes where the experience we ourselves are interested in arises. But here I drift off into the subject of role-playing games, because I’m a role-playing game master and I have that in my experience.

The most important thing is that people themselves start looking for meaning in it for themselves. Then this environment will respond with something meant precisely for them, because it’s still new.

And how can this be useful to the ordinary user?

“Everyone will be able to practice creating their own worlds”

Traditional tools already exist, like in gamedev. Beyond that, there’s the practice you see in sandbox worlds like Roblox. And the specific thing that exists in virtual reality in particular is editors of three-dimensional worlds inside three-dimensional space. As an example I mentioned Tilt Brush. There aren’t many such tools yet, but they’re appearing, and you can already use them.

What I haven’t seen yet is tools for editing behavior scenarios inside virtual space. There’s something like that in Rec Room, but I’d still think harder about how to design interaction with a three-dimensional space from inside it. Though you can create something simple with no preparation: you put on the headset and go, as in Tilt Brush, Roblox, or Minecraft.