Skip to content

Instantly share code, notes, and snippets.

@pfrazee
Last active June 6, 2018 22:12
Show Gist options
  • Save pfrazee/f789b37364956f4ee5c76a70ffa0b0da to your computer and use it in GitHub Desktop.
Save pfrazee/f789b37364956f4ee5c76a70ffa0b0da to your computer and use it in GitHub Desktop.

Peer-to-peer computing: Beaker 0.8 and beyond

There's a somewhat old-fashioned term in computing (old-fashioned being a relative concept) and the term is "live."

Back when Xerox PARC was making the first real GUI and building the first Object-Oriented programming language (Smalltalk) they were building a live environment. And "live" meant that the code was right there, available to the user, ready to edit. If you go back and look at the demos of Smalltalk, you see people jumping into that code and modding the environment on-the-fly, and that's what live meant! It meant you could mess with the code.

There's a complementary term, just as old-fashioned but much more relevant, and that's "dead." That's what cloud computing is: dead computing. It's compiled, packaged, shipped, and completely unchangeable -- a total black box to the user.

Live is vibrant. Live is community-owned. Live means that mods and plugins are going to emerge out of the userbase. Some of the most popular and well-known computer games started as mods: Counter-strike and Team Fortress for example. Halflife wasn't quite live, but they put out the source code and it was able to change. Imagine if that hadn't happened, and everybody had to build their games from scratch? We'd all lose out on the creativity of the young people who sat at home and messed with the code.

Now: Imagine what it would mean to have live social media, live forums, and live databases. Imagine what it would mean if the users could make and share mods. You'd have more than social media: you'd have social creation.

You can't have that with cloud computing, because the source code runs on the server. Users don't get to mess with that code. But in peer-to-peer computing, there is no server. It's just the users' computers and the source code. Hack away!

Computers may be the bicycle of the mind, but if the computing environment is dead, then you're basically the kid riding on the back, standing on the spokes and asking your friend to slow down. If you want to actually drive the bicycle, you need a live environment.

Zero dev-ops is the revolution

It is weirdly difficult to deploy an application on the Web. On Windows or MacOS, there's no problem: you install with a wizard and then you click the icon. You couldn't imagine a mainstream OS that doesn't make installing an app easy. But for the Web, it's like rocket surgery. The most consistent experience yet is using containers on a virtual Linux server; it's not even vaguely one-click.

Since users can't deploy their own version of our Web apps, we expect to make one deployment that scales to support everyone, which is of course the curse of success on the modern Web: if you're unlucky enough to make an app everybody loves, then you're on the hook to run it for them. When that happens, you best read up on vertical scaling and GDPR compliance, because you've now got a business, whether you wanted it or not. You thought you were just enjoying the magic of making software, but now you have to pay for servers. Shipping more cool features? Nope, that was for the MVP period. You're in scaling mode now!

Operations is a tax on the software industry, and you're paying it because the Web doesn't know how to take advantage of user devices. Wouldn't you much rather be able to launch apps as easily as you npm publish? Of course- but you can't. You have to run a server. In my opinion, the entire long-tail of indie developers is hosed by the need for operations.

Walled gardens, by the way, are the second-order effect of the Web's operations story. Since end-users have no way to deploy apps, they end up using somebody else's deployment, which is why they have no control over their data. Give users a way to deploy apps, and app-developers will suddenly find themselves able to leverage data from other apps, because that's what users will want them to do.

At the end of the day, there's two ways you can look at the downside of dev-ops. One, it's choking out the market, so consumers have fewer choices. Two, it's choking out free and open-source, because it stops users from accessing and modifying the code of their own deployment.

When peer-to-peer computing has matured, operations are going to become a niche specialty again. You're going to be publishing apps with one click so that users can deploy them with just as few. Fire-and-forget. And the second-order effect of zero devops will be an end to walled gardens. Users will have the code and their data all on their device, and they'll be free to fork and modify.

Beaker 0.8: making websites, made easy

The next release of Beaker is the conclusion of a year-long effort to figure out what Beaker should become. We started with a prototype and the p2p tech, and set out to discover what the p2p Web should be.

We explored some ideas of turning the Web browser into a networked OS, including a web command-line terminal, but ultimately realized that the value of the p2p Web isn't just giving more bits to fiddle: it's in making the hard things easier. It turns out, the hard thing about the Web is still devops and publishing.

We came to realize that this is where the p2p Web can shine: Making site-creation a push-button experience. This would be solving something that should've been solved back when the Web was born.

Once we understood that, it became clear that every change we made to Beaker should be in service of making site-creation easier. We focused on simpler flows, added more tools for authoring sites within the browser, solidified integration with external editors, and began adding goodies like starter templates and live-reloading.

With the feedback of a great community, we iterated through betas and finally landed on a vision for Beaker 0.8.0: becoming the frontend developer's favorite browser, and making it possible for anybody to build on the Web with the push of a button.

Making (and remaking)

TODO talk about copying sites

What about the APIs?

Everybody loves a static Website, but unless you can make well-featured apps, you might as well not bother with the technology. As I have personally learned over the years of working on p2p projects, the usability of the APIs is just as important as the usability of a UI. Devs are users too.

As we developed Beaker 0.8, we explored how far we could get with the DatArchive files API. This API gives you the ability to read, write, and watch files on Dat's p2p web archives (aka, sites). In some cases it can be perfect for your app, as in the case of this photos app or this rss reader. But, for applications that really need a database, a files API is difficult.

To mitigate this, we tried the WebDB experiment. WebDB is an indexer that runs on top of the DatArchive API. Basically, it tries to create a database on top of the files. We used this to build a p2p twitter clone, Fritter, and had a fair amount of success. However, it still seems that devs have a hard time learning how to work with WebDB, so we kept looking for opportunities.

Eventually, the protocol lead of Dat (and now our Chief of Research) put together an entirely new extension to the protocol, called HyperDB. This update will make it possible to create keyvalue stores (similar to LevelDB) that are shared on the p2p network, just like a Dat Archive. At the same time, HyperDB delivered a major performance boost to the protocol, which will soon be passed down to Dat Archives as well. We'll be adding HyperDB as the DatDB API, probably in the 0.9.x releases of Beaker, and we're excited to see how the usability of this interface helps the developer experience on Beaker.

One other major shortcoming in our APIs is that, currently, you can't open data channels to other users on a site. About two weeks it occurred to us that not having easy-to-open data channels is completely insane. The most obvious expectation of a p2p web browser is that you'd be able to easily communicate with other peers, and while that's currently true when it comes to publishing sites, it's not true for something as simple as chatting with other users. This is actually a major handicap, because it means the audience of a site has no way to discover each other. So we plan to solve this ASAP with an API for arranging data channels between users on a site.

There's much more to talk about -- especially Dat's coming support for collaboration on a site -- but we'll leave that for another time.

What kinds of p2p websites will we make?

I already mentioned our p2p twitter clone, Fritter. There have been CMS projects, text editors, svg editors, and even a few games. What else can we expect?

One of the areas that excites me most is the idea of "fast apps". A fast app is a website designed to solve a single problem without being connected to some larger ecosystem of software. Things like: like event invites, polls, photo albums, small forums, group chats, business pages, and so on. Users should be able to click the create button, select a fast app template for their site, and have the site ready to use. The caveat for these fast-apps is that, any time they require audience input (like RSVPs to an event) the author will need to keep their computer online in order to get those responses. That's just how p2p will work at first. Once we get the equivalent of p2p email, we'll be able to come up with more asynchronous applications.

You know a technology is really exciting when you can do something incredible with just a few lines of code. When Dat's collaboration features land, it's going to be possible for multiple users to work on a dataset or site directly. That'll have great use-cases for office software: documents, spreadsheets, calendars, filesharing, and anything else you do in your business. Based on what I've seen, I think it'll be possible to write a minimal p2p multi-user wiki, editor, or spreadsheet with less than 100 lines of code.

P2P computing and beyond

Peer-to-peer technologies are a step-change in fulfilling the Web's mission of accessibility and freedom. For the first time, it's going to be possible to publish Web sites and applications using only the browser. It's like converting the browser from a television into a high frequency radio tower; the broadcasting power is in users' hands.

I see easier site creation as the first step towards something really remarkable: a live networked computing environment, where users build their applications as a community from the ground up. The ability to easily create and copy sites is the core, but with new models for shareable modules and plugins, inter-process communication and even intent URLs, I think we'll see composition between users' work like we've never seen before. This is a the path to a more diverse, more user-defined, more democratic Web.

Beaker is an experimental browser and it may always be our crazy experiment, but I'm so glad to share it with you, and so thankful for the community that's carried us all this way. Come take a look, join our chatrooms and forums, and tell us what you think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment