A Decentralized Virtual Reality Protocol

vr_protocol.md

A Decentralized Virtual Reality Protocol

by Leonard Ritter, Duangle GbR

The metaverse can be a utopia or it can be a dystopia. And we are very fortunate that the PC platform is the number one platform for serious computing today, and the internet underlying it are all completely open technologies. Anybody can build software. Anybody can distribute it. Anybody can do anything without getting anybody's permission. I feel we have seen a major retraction from that great state of affairs over the past decade, as closed platforms (...) have grown to the forefront of the industry. (...) And I wanted to point out that this was not inevitable. (...) As we are creating a new medium together it would be really tragic if the future metaverse that binds all humanity together into shared online environments were a closed platform controlled by a giant corporation. As always, they would use it to spam you with advertising. They would use it to gather information about your private life and sell it to the highest bidder, and they would act as the universal intermediary between all users, content creators and transactions, ensuring that everything has to be approved by them to some extent, and they would take a cut of everything. And we really should work at fighting to keep these things open. -- Tim Sweeney, November 2016

This is not a technical document. This is an attempt to suggest or to predict what an open metaverse might look like.

The Three Pillars Of DVRP

About twelve years ago I attempted to build the first open metaverse protocol. It was designed as an extension to the IRC protocol. Users in the system would not only send each other messages, but also transmit VR related information such as their position in space and links to resources that described their appearance and the space they inhabited.

I chose IRC (Internet Relay Chat) because it is the great remaining decentralized realtime communication network. IRC is maintained by volunteers working at universities and non-profit organizations. Despite Facebook, despite Twitter, despite Slack (which is greatly inspired by IRC, but lacks its decentral aspect), IRC is still alive and well. It has many desirable features you would also want in a metaverse:

• Channels, where multiple users can meet and talk together. A channel would be equivalent to a room or an alternate reality shared by multiple people.
• Private chat, a channel shared only by two people. Now you can tell people to get a room, and they can actually get one right away. ;-)
• Simultaneous occupation of many channels and chats over different networks. Ever notice how, with a VR headset on, you're in two places at the same time? In the metaverse, you could be in dozens of places simultaneously. (I'll get to how that might work later on)
• Channel rights management, permitting users to self-govern their channels: mark a channel as invite only, reciprocate administrative and communicative rights as well as mute, kick and ban intruding or malevolent guests.
• Username and channel services, represented as bots, which can be interfaced with through private chat to protect your username or channel when you're not online. Oh yes, that means we have lots and lots of...
• Service bots, typically run by users, providing all sorts of channel services (of which entertaining nonsense is a popular one), depending on what the channel is about.
• Open and extensible protocol, text based and relatively easy to implement on both server and client side in its simplest form. Like all other online protocols, it can be extended with experimental or proprietary message formats to gain new features. It can do so in a backwards compatible way. That means the metaverse could be enjoyed in text form only. This allows you to run it in an ascetic format when you're at work or on the phone.

The IRC network is the first part of the DVRP infrastructure.

Of course the IRC design is not without its flaws, particularly when seen through the lens of a potential multiverse protocol candidate. We shall look at these flaws as well to better understand what we are dealing with:

• Flooding. It is common practice of IRC servers to disconnect users which send an unrealistic number of messages to the server, as this can only possibly mean that they are trying to DDoS the server, disrupt channels or spam users. But the metaverse requires that users send high frequency updates pertaining to their position and appearance in space (very much in the same way that realtime multiplayer games do it), and this is at odds with flood detection. The solution to this problem is also the solution to the next one...

• Text based protocol over TCP. A text based protocol is comfortable to debug, but wasteful when non-text data (such as coordinates or orientations) needs to be sent. Sending it over TCP ensures that all messages arrive in the correct order, but the overhead increases latency for high frequency updates.

  My favorite solution here would be to do it the way multiplayer first person shooter games have done it, and use a UDP-based sidechannel to relay high frequency data in (possibly compressed) binary format at low latency. The "spatial sidechannel", as I call it, would not be subject to the same flood control as text messages. The spatial sidechannel is the second part of the DVRP infrastructure.

• P2P resource downloads only. A metaverse rich with vistas and sounds requires transfer and caching of Gigabytes of multimedia resources like textures, shaders, sound files and triangle meshes. If a user wants to share his world with others, he will require a means to broadcast multimedia to others. The IRC protocol however only supports CTCP requests for peer-to-peer file transfer, and that is insufficient, for performance reasons I don't want to elaborate too much on.

  Fortunately we already have two great open and decentral resource broadcast systems: the world wide web and the BitTorrent protocol. Upon joining a channel, a service can provide the user with a list of resources to download in order to make the space and its inhabitants manifest. These resources can either be stored on webservers, and/or, as we already know that many users are simultaneously online and interested in the same resources, as magnet links to permit users to rapidly share resources peer-to-peer. In my opinion this would be the moment of glory for the BitTorrent protocol, which, despite its conceptual brilliance, has an undeserved reputation for serving mostly semi-legal, potentially copyright infringing, needs.

  IRC servers have little to do with resource distribution. They only need to facilitate finding them, by distributing resource links. Downloading and sharing of resources happens purely client side. The resource distribution system is the third part of the DVRP infrastructure.

• Netsplits. An IRC network consists of many servers relaying messages between each other, of which each server, depending on how popular it is, hosts a large number of users. Occasionally a server needs to be restarted for maintenance, or succumbs to a DDoS attack, and is shut down, disconnecting the users it hosts from the network. IRC clients are programmed to try switching to a different server as soon as the outage is registered, but to other users in the network it looks as if swaths of users are suddenly disappearing. In the metaverse, it would look like the pilot of Leftovers, except that everyone returns after a few minutes of absence. If two servers only relay messages over an intermediary, and that intermediary shuts down, then both parties remain alive and well, but it looks to each party as if the other party had disappeared.

  Nowadays, netsplits are not as damaging, thanks to persistent services. Netsplits used to be an attack vector for users attempting to steal usernames or take over channels in the olden days, but these times have long since passed.

To Be Continued

This is as much as I could muster to write so far, but of course many questions remain open. These topics will follow soon:

1. What constitutes a sufficiently complete DVRP client.
2. How the metaverse can be navigated.
3. How avatars will work.
4. How to keep the protocol minimalistic, yet flexible enough to maximize the creative options of its users.
5. How to protect the personal space of users in VR. I consider this easy to solve, but I feel that due to recent events, it deserves special attention.

If you have questions, remarks or ideas regarding this document, please put them in the comments below.

A VOIP solution is needed too. (stating the obvious)

Let me chime in as someone who has run an IRC network for 20 years.
I love IRC and nothing has ever been able to replace it for me, which is why I still do it. BUT:

• IRC is not federated. It is little fiefdoms and big kingdoms and they don't interact at all
• IRC protocol is AWFUL on mobile. It doesn't work with push without intermediary clients like quassel
• IRC protocol, like all the old classic protocols, is awful to change. It takes huge forces to add to it. For example SASL authentication was something like 10 years from idea to practical.

Lastly, you make little mention of griefing. The most important thing we do as IRC network operators is moderate. Even on a network where we don't moderate 'ideas' much at all, we are constantly attacked by actors big and small. In a text environment that means floods of text (overwhelming the screen), or bandwidth attacks (overwelming the network), or attacks against clients. In the VR world those things can be much worse (cannot un-see images, in addition to the rest). One need only follow the news of Secondlife in its prime to see how often penis-attacks happen(ed) there etc.

Finally, what tools make the environment? In IRC we have channel modes and a topic, and that is about it as far as what a room 'looks' like. But in a VR world that is much much more important. Who creates the room? Who creates objects in the room? Can those objects move/run scripts? What about members & their costumes? Do you allow these things to just dump some script to the channel and everyone's clients run them?
-Rubin