Skip to content

Instantly share code, notes, and snippets.

@getify
Created June 5, 2024 18:08
Show Gist options
  • Save getify/2a3fce803ec879df2e5c044d252cd3af to your computer and use it in GitHub Desktop.
Save getify/2a3fce803ec879df2e5c044d252cd3af to your computer and use it in GitHub Desktop.
The Root of All (Web) Evils

The Root of All (Web) Evils

The consumer web as we know it is around 30 years old, but of course its origins stretch back many years before. In those original days of the web, the infrastructure of how content was distributed and accessed was fairly de-centralized. If I had something to share, I'd put it in a digital location I owned/controlled, and you were invited to get that information directly. Nobody could tell me what I should and shouldn't create and share, and nobody could prevent your access if you wanted to see it.

But over the decades, this infrastructure has consolidated into ever more complex, and ever more expensive, centralized cloud providers. The "server-client" model is so ubiquitous now that it's hard for nearly anyone to imagine a web that could work in any other way than with "Big Cloud". While the centralization of the web has offered numerous scalability (and efficiency) benefits for web developers and web adminstrators, it has come with a vast increase in cost. And no I'm not really talking about your AWS bill.

The real costs of centralization smack us in the face everyday, but we largely dismiss those concerns with sunk-cost reasoning.

Centralization has moved the ownership of information out of the hands of the users who create (and use!) it, into the hands of the cloud server providers. We rent this data from service provider overlords, who in turn rent this data from cloud providers. When you want to use the information you created yesterday, or even a moment ago, you're a beggar at the door, a stray rodent hoping for scraps to fall from the table.

If you don't have a fast and reliable internet connection at the moment, you can't have or use your information. If there's an infrastructure outage anywhere along the dozens of hops, you can't have or use your information. If the provider decides to raise the rent and you can't pay up, you can't have or use your information. If the provider goes out of business, you can't have or use your information. But you know who readily gets your information? Any hacker who sniffs out the juicy value of all that centralized user information and exploits the laziness or ineptitude of the provider's systems security. They seem to have access to your information whenever they want.

When our data is not ours by custody, then it's not truly ours at all. So one real cost of centralization is that there's no longer "user owned information", only vast stores of "user-based, provider-owned data".

As if that's not bad enough, we've never seen more information being created on our behalf (without us knowing!), in the form of tracking us, every keystroke and swipe, no matter where we go on the web. It seems that the only party in the whole equation that doesn't have unfettered access to the information is the very user that information is about. Every other player benefits, but the user is shafted.

Aside from loss of information ownership and privacy, centralization also harms the user's experience of the web. When we moved the web from a mostly static store of non-interactive information, to a dynamic, interactive application platform, we forced users to accept a "re-install on each access" model of application interaction. We sold this as the web being "evergreen", perpetually updated, always fresh. But then we also sold users on the idea that re-download/re-installation of the web experience, across the wire from server to client, could somehow also be instant-on, the way any other already-installed app on their device behaves.

Science tells us 100ms (1/10th of a second) is the maximum threshold of delay before a user can perceive a delay. That means the ultimate goal of the current web model is that we need to be able to deliver many tens of megabytes of JS, CSS, HTML, images, fonts, and... yes... data, down the wire from a server (even an "edge cdn" server) to a user's device, all in 100ms?

There are some users who are privileged enough to have internet access (and device capability) sufficient for that task of near-instance reinstall. But the vast majority of the world will never, under any circumstance, have such an experience. This creates a deeply unfair hierarchy, a structurally-biased class system that sifts the world's 8+ billion people into tiers of web experience fidelity. The web was supposed to be a leveling, empowering tool for everyone. But instead it's become yet another way to leverage power dynamics in favor of the privileged.

If you even dare to point out that a web experience is laggy or sluggish when simulated 3g connection speeds are turned on in browser developer tools, you'll be drowned out with laughter or derision. Most web developers (driven by clueless product owners) only build the web for the best case conditions, and care very little for the "long tail" experience.

To hide/minimize all the inevitable delays users experience when re-loading these huge web application payloads on almost every visit, an extremely impressive and complex stack of webdev sophistication has grown up, most recently capped by "React Server Components" and similarly related techniques. All the noise of partial pre-hydration, client and server routing, and more... that's all CHOSEN COMPLEXITY. It's not actually necessary. And the billion dollar "web performance optimization (and monitoring!)" industry has further layered on business models, employment opportunities, and myriad costs, atop the "modern web dev" process.

Is there any clear point, and is there any way to address these complaints?

The root evil of all of this is... the SERVER. Specifically, the web server. And even more specifically, in the conflation of "channel information exchange" and "server consumable app experience payloads" into the same bits of technology. This is the worst ideas the web has ever seen, and if left unchecked, is going to lead to the implosion of the web, collapsing under it's own weight.

But I believe there's something we can do about minimizing, perhaps even removing entirely, this evil. Not only can we, but we must do so, if we want the web to survive and thrive, and if we care about the web equally serving all of humanity.

There's a movement called "local first", which aims to completely re-orient our thinking about building software, especially web software. The main goals:

  • user ownership, autonomy, and portability of data
  • must be able to work without internet (not just "offline first" but "internet optional").
  • must continue to work even if servers that were once involved go away -- either by architectural design, or by open widely-adopted standards and independent implemetations

The primary idea is, user information should first and foremost be created by the user, locally on their own device. Optionally, that information may be transmitted/synced/backed-up elsewhere.

Not all apps even need to share information beyond the device that's used to create the information. These types of apps might be thought of as "local only", which some (myself included) believe is a meaningful subset of "local first". For example, a health tracking app, or a note-taking or todo/personal-productivity app, or a game. These are all apps that may very well create user information locally, and there's no desire or need for that information to ever leave their device.

Now, some may claim that users generally want their data available on all their devices. I call this your "local ring" (throwback to the old "web ring" terminology of the early web), where there's an ad hoc, private "network" that joins your own personal devices. From that perspective, a "local only" app is "local" to your ring of devices, rather to just one specific piece of hardware. So local-only apps can still offer very helpful syncing and backup between your various local ring of devices.

And even if your local-first (or local-only) app might want to share information non-locally... the default should be that only the minimal necessary data is sent remote, and that your device always retains the primary information you need/want.

Many in the local-first movement would also agree then, that your local copy of any information acts as the source of authority for that information. That is, if there's a conflict between the information you've retained locally, and any copy/subset of that data that's been sent elsewhere, then your local copy always wins.

Moreover, if local-first says that we must be able to have apps that work even if the data servers are inaccessible, doesn't that mean that a valid way of achieving local-first is to make servers completely optional? Like, what about "zero server"? The original experiences of the early web were much more what we'd call "peer to peer" than what we experience today, which is hub and spoke... everything goes from client up to server, before being sent back out to other clients. But why the middle-man? I believe local-first apps point to a returning of the web to its P2P roots, where much of the information exchange that happens between peers, happens DIRECTLY between those peers, instead of through a cloud server middle-man.

There are protocols being developed and spread where apps can communicate with each other through zero-server peer-to-peer relays (all with privacy protecting encrytpion), which for the first time in decades, offers us the real opportunity to unchain ourselves from the tyranny of the centralized cloud landlord system. I call this subset of local-first, "peer first". That means that we can design and build applications which default to assuming that information exchange is directed through peers (and relays), and only optionally falls back on centralized servers (or maybe not at all).

But we can go even further than this local-first, peer-first ideal as it relates to information exchange. If we can agree that this is a useful way to prioritize the ownership and control of our INFORMATION (and the exchange thereof), then shouldn't those same ideas apply to the executable payloads of the apps we use to experience and interact with that information?

We can move our thinking of the web from "re-install on every visit" to "install once, instant open", the same way traditional "non web" apps work. Apps can be built using web-stack technology (like HTML, CSS, JS, etc), but that doesn't mean that "web" requires these payloads to be delivered to a web browser client over-the-wire from some remote server.

"Web" can (and should!) mean the interconnectedness of information sharing, without having to also imply "centralized storage and delivery mechanisms for application experiences".

In fact, if you look over my shoulder and see an app installed and running on my device, and you'd like to also have that app, why should it take you round-tripping (with internet access) to some centralized app store (or web server) to get your install of that app? Shouldn't I be able to just directly share this app install with you, from my device to yours?

Some may wonder, how are we going to keep monetizing the access to, sharing, and management of information, if we democratize information and make it local-first and peer-first? And those same folks probably also fear that we'll lose the ability to monetize the sale of software if we push installation into P2P as well.

Nothing about local-first, or even peer-first, software design is inherently anti-thetical to business and monetization. Nothing here is suggesting that everything has to be free, and freely accessible. In fact, I believe the case can be argued, local-first, peer-first software and information exchange are inherently more valuable to users than any software that currently exists today. It's exactly what users would choose, and what they'd most happily pay for, if they even understood they had the options, and what the tradeoffs were to the old model.

What has to be given up is the notion that business and monetization require gate keeping.

Let me illustrate my point this way: I have written and published (and sold many copies of) over 12 books on JS and web programming topics. And all of that writing has ALSO been made freely available to consume, on my github account. The presence of free copies of my books has NOT killed the monetization of my book content. In fact, quite the contrary. I believe I've sold many times more books BECAUSE I didn't gate-keep the content (behind a paywall). I've sold hundreds of thousands of book copies, even in spite of having probably millions who've accessed my content freely. Hundreds of people have told me they later BOUGHT my books after having read and appreciated my freely available book content, as a thank you and as a way to support and encourage more of that (from me and others).

What I'm getting at here is, I have monetized my book content without gate keeping. I've made a plenty solid business of it. What I couldn't do was be consumed by greed to say that I had to capture every single consumption channel of my content. I had to be willing to accept that some (many, maybe even most) would access my content for free without paying me anything. But then there would also be those who preferred to pay me for the books. And any overlap between those groups is great, but even if there wasn't any overlap, that's fine too. The presence of each group actually promotes the growth of the other group, mutually beneficially.

Also, importantly, purchasers of my books (especially the print form) get something tangibly better than the free content readers got: a permanent, owned copy of my information. They paid for that extra value, because it mattered to them to have it. It improved their experience to own it. My published books are an "optimized" experience around the content, compared to just reading markdown rendered on github. And I am happy, and my customers are happy, that this optimized reading experience is what's payed for.

This is how I see monetization and business in the future world of the web. We'll make information and exchange and experience a widely accessible resource, and we'll "sell" the optimization and improvement of these experiences, without needing to gate keep to capture absolutely every single eyeball. I believe the non-greed of that approach will create bigger and more sustainable business models, for a wide variety of industries, compared to the business models that lock down and gate keep and charge for every single eyeball.

And notice that I've been talking all about de-centralizing the web, and even preserving monetization of this web, and I haven't once HAD to mention cryptocurrency. Does web3/crypto/blockchain have anything useful to contribute to this future web I'm laying out a vision for? Perhaps. But not necessarily.

I call the next evolution of the web, as I'm laying out, web2.5. And I'd say that it will pave a path toward a future where maybe web3 comes to fruition. But web2.5 is not inherently or necessarily web3. What I can say is, with a fair degree of certainty, that we will not simply jump from our current web over to web3, no matter how much the hype would suggest. We're on a long path of incremental (and evolutional) steps, and if the web of 2024 is ever going to become the web3 that some have pitched, I feel confident in saying it's going to have to first go through a web2.5 as I've envisioned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment