HTTP considered harmful

I’m convinced that, with the Right Tooling, it’s possible to develop distributed applications without having to deal with unreliable request/response semantics.

TCP, the foundation of modern electronic communications, is a pretty decent attempt at making the participants believe they’re in the same room. For a limited span of time, it facilitates reliable, error-free, in-order delivery of information—almost a Pareto improvement over the public switched telephone network, as far as computer systems are concerned. Seemingly all that was left was to extend the lifetime of a TCP connection to cover the lifespan of the application. And yet, somehow, the broader industry went entirely in the opposite direction: we reduced it down to a semi-reliable delivery of an individual message, and keep working around the same issues in essentially every distributed application with contingency measures like idempotent requests, retries, refreshes. There exist frameworks that make those issues less impactful but the measures are nevertheless required just the same.

Streams, increasingly adopted in data management systems and complex backend architectures, remain an exception. Despite running on the very same network, they expose its issues in an entirely different form: state and lag. I believe streams are a viable alternative for building correct and performant systems well beyond their current applications. If you build systems which cross the safe boundary of a reliable node, this is an attempt to inspire you to think differently about the challenges they face. Although the Right Tooling does not yet exist, I’m hoping that, with time, it will become a reality.

Let’s embark on a journey toward a declarative framework for building distributed systems whose semantics align with the nature of applications and not with the primitives of the Internet as it came together in the late 70s.

I’ll begin by outlining the systems and practices I’d like to challenge. Then, we’ll build the necessary intuitions:

We’ll discuss a couple layers of computer networks, classify them as connection-oriented and connectionless, and develop an understanding of gains and tradeoffs as we move between them.
We’ll review some theory behind streaming systems and incremental view maintenance.

Finally, we’ll apply those intuitions to come up with a novel way of thinking about distributed applications.

Motivation and scope

Contemporary distributed systems are missing a layer between HTTP and applications.

REST is a convention. RPC is an abstraction. What we need is an overhaul.

It’s hard to imagine modern applications without requests—HTTP requests, RPCs, or other connectionless constructs. (I’ll cover what I mean by “connectionless” later in this series.) When multiple components run on separate machines, this is typically how we get them to communicate.

It’s quite possible to reason about individual components in isolation without thinking about requests:

A modern web frontend application runs on a single machine at any given time. Its state is often mapped declaratively to the document object model, as is the case for React. When you remove the outside world from the equation, errors are typically deterministic and are a function of logic and data.
A DBMS often spans multiple machines, and yet it’s often a highly available system which encapsulates complex interactions and presents us with a relatively simple, declarative interface. Once you define a materialized view, you can rely on its contents being up-to-date or, in the case of delayed view semantics, having a known bound for its freshness.

However, as soon as we string those components together, problems arise:

Availability—some services might experience downtime or operate at capacity. Note that this isn’t intrinsicly so much an issue of distributed systems as it is an issue of interactive systems. An operating system can have availability issues, too. Networked systems just have a harder time dealing with it.
Versioning—separate release cycles and often weaker type systems cast the familiar problem of dependency management into the real-time domain.
Security—crossing the boundary of a process is scary enough. Going over the public Internet is the stuff of nightmares.

And, finally, the family of problems where, in my opinion, the most improvements are attainable over the status quo.

Network faults

Applications deal with connectivity issues in a variety of ways.

Loading indicator, AKA blocking

When an application attempts to read data from a remote service, some time passes before this data arrives. In the meantime, web applications usually show some sort of a loading indicator, and backend applications block a thread or a coroutine. If the request fails, there’s often some logic that attempts to retry it, but after a few failures, it gives up—and what this entails is for the application developer to decide.

This sometimes leaves the application in a strange, inconsistent state. Maybe the user can see their recent orders, but only some of the orders have their items listed? It’s worse in the backend, where fetching one feature flag might have succeeded while the other fell back to its default value, leading to an untested, possibly invalid configuration.

Some sophisticated applications periodically check if the service is reachable again (possibly aided by mechanisms such as the online event in web apps) and then attempt to request the resources again. In my experience, this isn’t a common practice and users are forced to refresh the website after they regain connectivity or the backend comes back up online. The issue of an untested timeout or error code crashing the downstream application is also familiar to backend engineers.

Offline mode, AKA caching

Some applications are able to display to the user cached data, sometimes with a disclaimer that they’re operating in “offline mode”. This makes for an interesting observation: an overwhelming majority of websites only load data at startup, and thus they effectively operate in offline mode soon after they’ve loaded—except it’s implicit.

To give this claim some substance, imagine you open a shopping cart in one tab, then add an item to that cart in another tab. The contents of the first tab will typically not change—it’s operating in a de facto offline mode. This isn’t a connectivity issue at the network level but instead at an application level. I believe that a few years from now, when users become more demanding of their applications, such behaviour will be perceived as a bug.

Idempotent requests

It is a truth universally acknowledged that any failed request must be in want of a retry.

Software engineers design their APIs to be, as much as practicable, idempotent. This often has nothing to do with the application itself and instead is a desperate measure to account for duplicate requests due to retries. This either requires extra state (idempotency key) or careful consideration of application semantics.