Jive for Artisanal Software

Disclaimer: Jive is not for every app but it drastically reduces complexity and accelerates development speed for certain types of apps. Judge for yourself.

Jive is an architecture for semi-native apps. Jive apps can run natively on mobile and desktop, and the web. Jive can be implemented in any programming language.

Let’s dig into the main assumptions:

1. Web browsers, specifically the DOM and CSS provide UI primitives that are ubiquitous, battle-tested, flexible, endlessly stylable.
2. Web apps are popular for their low friction and reach (just click a link) and cross-platform support.
3. The JavaScript runtime isn’t the reason people are choosing web apps. As an environment, being single-threaded and asynchronous limits performance and programmer productivity…
4. …as opposed to native apps which have access to more programming languages than the web, the file system, real databases, can be multi-threaded and are generally more performant but are also high friction to install and are a pain to write cross-platform.
5. If storage is an essential part of your app’s design, web browsers are still transient environments without a good durability story. For example, IndexedDB is deleted when users “clear cookies and site data” or in the case of Safari after 7 days of inactivity 🤷‍♂️
6. Virtual DOMs enable performant (enough) declarative UI programming with minimal paints. The massive success of React.js demostrates the power of this approach. Typically a vDOM sits in the browser but as projects like Replica show, it doesn’t have to!
7. Given low enough latency it would be possibly escape browser runtime limitations and instead leverage the power of native. With a server running on the same machine (i.e. localhost) a roundtrip takes ~0.23ms.

How to Jive 💃🏻🕺🏽

A Jive app combines all of the above into an architecture that treats the browser as a dumb view layer while moving all app logic and computation to a preferably local but optionally remote backend. In other words, they server-render but then they also server diff and send patches to the frontend. If you’re familiar with the virtual DOM concept–it’s that–but on the server.

On the server state updates result in DOM-patch commands that are sent to the browser runtime to be executed on the DOM. This puts your view code into Nativeland — land of multiple cores, languages, durability and file system access.

When installed on mobile or desktop it will look and feel similar to an electron app. When visited in the browser it can provide a similar experience only with slightly higher latency between events and DOM updates (more on latency in the Technicals section).

Reach of Web Apps

Jive retains the wide reach of web apps. As an on-ramp, new users can start in the browser. If they like it, being able to work offline and having lower latency (while also reducing server load) are good motivators to install the app locally. And if they ever need to share a link or login from another device they simply use the remote backend. The types of apps that jive well locally are tools and productivity apps — apps you use every day and don’t mind installing. That said, other than scaling, there is no reason that it wouldn’t work well for remote-only e-commerce sites.

Benefits of Native

Imagine the full power of (local) backends with their multiple cores, languages and databases but at the UI level! With Jive, a DB query can, if you choose so, live directly inside your UI. Your local DB (or even a cache) is durable and you don’t pay for network latency between queries. Compare that to SPAs that a) have to query through HTTP requests, b) asynchronously wait for responses, c) interpret the result (possibly using precious processing cycles of the singular JS thread) and then d) ensure that data stays in sync. In tools and productivity apps where all user data easily fits on disk (even on mobile) building SPA style is completely unnecessary, so why do we keep building them that way?

Artisanal Software

If it can be that much simpler, you might ask why didn’t it happen yet? One possible (and possibly wrong) answer is that Jive’s model is not compatible with the financial goals of ad-revenue giants like Facebook and Google. When the product is subsidized by ads that each contribute just a few cents, the profits lie in gargantuan scale (billions of users) at next to zero marginal cost. At that scale, Jive’s architecture where millions or even billions of remote vDOMs would need to be maintained, would be a cost nightmare. All the costly computation to update state and diff the DOM for each user would have to happen on the company’s servers instead of being executed in the user’s browser. It’s much cheaper to hire a few dozen more engineers to deal with the shortcomings of SPAs. But a fresh startup or small company, hiring a few dozen engineers is often prohibitive.

Another possible answer is that at the time when technologies like React.js came internet speeds were far slower. React came out in 2013 shortly after 4G started rolling out in the United States. A giant like Facebook seeks global market penetration, also targeting less technologically developed countries that lag behind in network speed and adoption of newer devices capable of leveraging 4G.

This might be why tech like React.js came straight from Facebook. The incentives are aligned — it improved developer productivity but only as far as it made sense for their business at the time.

Little of the above matters when you are making productivity tools that passionate users are happy to pay for, much less install. Marginal cost becomes a marginal concern, while reach, iteration speed, and developer productivity become paramount. Here, small teams, a focus on craftsmanship, and more concise codebases thrive and call for simpler architectures like Jive. This is what I call artisanal software.

Technicals

Latency

If you want to get a feel for real-world remote latency, have a look at this Trello clone written in Elixir’s Phoenix LiveView. It does not use a vDOM per-se but similarly to what is proposed in Jive it tracks reactive UI nodes on the backend and sends patches to clients over a websocket connection. It’s hella snappy!

Local Benchmark
I ran a latency benchmark using unoptimized code measuring a few hundred websocket roundtrips through performance.now(). The average latency out of over 10,000 requests was ~0.23ms. That's 4347 updates/second (1000/0.23 ~ 4347.82). This 0.23ms is especially negligible when processing and UI are not on the same thread (hi SPAs!).

Remote Benchmark
This will greatly depend on your internet connection. On my phone, a roundtrip took 19ms, on desktop over wifi 12ms. With the same math as above, that's 52.63 and 83.33 updates/second.

Disclaimer: actual updates/second speed will depend on the processing speed of your backend and the payload size of the response. However, for the latter, given that DOM diffs are the payload this should be ok.

Implementation

As far as I know Jive doesn’t exist yet. The backend part of the architecture is language-agnostic as long as you can run it where you want to run it. The frontend DOM patching code will have to be in JS but can be reused across multiple backend implementations.

3 Easy Pieces

On the technical side 3 things are required to Jive:

1. A virtual DOM implementation (in your language of choice)
2. vDOM <> DOM middleware: Websocket interface + thin client library to listen for events and patch the DOM
3. Native packager or compatibility with Electron for desktop and Cordova for mobile or alternatives

Jiving in Clojure

Side-note, Clojure is known for its J-puns. As such, Jive is a J-pun on Live.

Piece 2 — vDOM <> DOM middleware
This is overall pretty straightforward. Write some middleware to send events and commands between client/server. This might require some optimizations around high-frequency events like mouseover or dragover. We can steal a few things from Elixir's Phoenix LiveView.

Piece 3 — Native Packager
We can stand on the shoulders of giants: GraalVM compiles native binaries from many Clojure libraries that run virtually anywhere. And Gluon has shown that using GraalVM they were able to package apps for Linux, Mac OS X, Windows, iOS and Android. How is that for cross-platform?

Piece 1 — Clojure vDOM
I’m not aware of a vDOM implementation in Clojure that’s not tied to the browser API. That said there are examples of Clojure/Hiccup inspired virtual DOM’s like Karsten Schmidt’s excellent hdom library.

Get in Touch!

If you’d like to be involved in a Clojure effort, implementing Jive, please get in touch! It would be an amazing contribution to the Clojure community.

More Food for Thought

Local AND in-browser
Especially on desktop native apps, it can be annoying if you have a browser-heavy workflow. Many native apps don’t have tabs or allow multiple simultaneous instances. Additionally, there is the navigational overhead of switching between native apps and browsers and their tabs. A Jive app doesn’t have to be a standalone UI app. Since it’s web-based multiple browser tabs can connect to a local backend.

Interface integrity — No need for lossy serialization
I write most of my code in Clojure. The Clojure ecosystem is packed with rich data-structures some of which have custom interfaces. I commonly use sorted maps, array maps, lazy sequences, priority maps, ring buffers and even write my own when needed. Clojure datalog databases have a dynamic Entity type that lazily lets you traverse a entity and its relationships as if it’s a simple object/hash-map. But under the hood, Entity performs Database reads and caches the results. The richness of these powerful data types is lost when doing traditional JSON serialization between frontend and backend. Jive removes this limitation as most of your “UI code” lives in a native context.

Some client-side JavaScript
Some scenarios will require client-side code to run. For example, apps might want .preventDefault events. It's overall discouraged but there's nothing in Jive to stop you from using client-side code.

On mobile your mileage may vary
Jive is semi-native because it still relies on the DOM. DOM APIs have come a long way allowing access to the camera, file system etc. But some mobile apps have demanding UI transitions that simply don’t work as well when implemented using web technologies.

Final Words

It all comes down to simplicity. The web and frontend apps gave us incredible reach but at the cost of an environment (the browser) that was never designed for what we’re doing with it today. I’ve personally waited for the promises of indexedDB and WebWorkers with shared memory to come true for nearly a decade and they never did. But I imagine there is a way, a better one, where we can simply use the same tools that have run reliably on our machines for decades: advanced programming languages and databases with file system access that leverage multiple CPU cores. In a word, that’s Jive.

Inspiration

• Elixir’s Phoenix LiveView
• Clojure’s Ripley
• Blazor
• Hotwire Umbrella (esp. Turbo which used to be Turbolinks)
• React.js’ Virtual DOM

Questions? Feedback? Get in Touch!

There is no website or Github Org for Jive at the moment. But if you’d like to be involved in the project feel free to DM me on Twitter (@denik).

Thanks!

Chris Oakman for a snowballing one-liner about Clojure missing an equivalent for Elixir’s Phoenix LiveView that kicked off the pondering for all of the above.

Bardia Pourvakil for hardcore editing, Stepan Parunashvili and Joe Averbukh for input and feedback across the board. Peter van Hardenberg and @inkandswitch folk for pushing back and providing pointers to similar projects.

The Clojure community at large. Thank you!