Creating your own virtual machine for fun and profit

What do a programming language from the 70s, an application from the 80s, a game from the 90s, and a hobby project from today have in common? I realized the answer at Strange Loop 2023: they all benefit from being built on top of a kind of virtual machine.

This realization came at Devine Lu Linvega’s talk “An approach to computing and sustainability inspired from permaculture". It was a wide-ranging talk full of innovative thoughts on many fundamental aspects of software, so I wouldn’t be able to summarize the entire talk if I tried! Instead, I want to dig into a point they made about the idea of programming against a virtual machine. We’ll name and compare the four pieces of software I referred to above, and we’ll see how the idea of a virtual machine may be broader and more accessible than you realized. You might even find you can use a virtual machine to make the software you create more portable, sustainable, and flexible.

Another World

Devine talked about how the video game Another World (known as Out of This World in the United States) was built on top of a virtual machine consisting of only a few dozen opcodes. Note that this isn’t a system virtual machine that simulates a real computer, such as VirtualBox running Linux on top of a simulated x86. Instead, it’s a process virtual machine that implements a simple set of opcodes the developer designed to fit their use case. Using a virtual machine has made it relatively easy for the developers to port the game to many different platforms over the years because only the small virtual machine needs to be reimplemented; after that, the code that was written against the virtual machine runs unchanged.

Beyond the immediate commercial benefit of porting the game to different platforms, there is a sustainability benefit as well. A big focus of Devine’s talk was computing sustainability: writing software that can continue to be used as far into the future as possible. Devine gave the example of iOS games that can no longer be played because older iOS code no longer works and Unity games that can no longer be played because of changes to the Unity license agreement. If a game or piece of software is built against a virtual machine, the task of preservation is easier: when an underlying platform is changed or goes away, you can write a new virtual machine implementation to get the software running on platforms that remain.

Riverbed

Thinking over this point about virtual machines, I realized there is a parallel with a side project of mine that I was chatting with Devine about before the conference. The project is called Riverbed, and it consists of a web client and an iOS client that connect to a backend web service. In a sense, these clients are a bit like two virtual machine implementations.

Riverbed is an app for personal information management. You define what kinds of data you want to store; for example, if you want a reading list, you might configure it to store a book’s title, author, and the date you finished reading. You can also set up buttons that let you perform actions when clicked: for example, you might create a “Complete” button that saves the current date into the “date finished reading” field.

So, whereas most apps only allow users to edit your data, and some also allow you to edit your schema, Riverbed allows you to configure your actions as well. All of this configuration is stored on the server and sent to the clients, resulting in less logic hard-coded into the clients than for other apps.

This approach has already led to benefits for portability and sustainability. I wanted Riverbed to run both on the web and on native mobile, and I was able to target both with the first client I built using React Native Web—but ultimately I decided that I wanted distinct clients for each platform. For web, I transformed the React Native Web codebase into a plain React web application. For iOS, I reimplemented the client from scratch using Apple’s UIKit. This process wasn’t quite as simple as reimplementing a few dozen opcodes, but it was still fairly limited in scope: the server’s data told me which fields and buttons to show, and I just had to write the iOS code to put that on the screen.

This thin-client architecture prepares me for future change as well. Will Apple eventually deprecate UIKit so that I need to migrate to SwiftUI? Will I want to migrate to a web technology that is less dependent on Facebook, maybe even something server-rendered? Because these clients are thin, I have the flexibility to consider changes like this in the future.

Smalltalk, HyperCard, and Runtimes

Comparing virtual machines to thin clients gave me a new perspective on Smalltalk and HyperCard, two historical programming systems that also allow you to code at a level a bit removed from the platform you’re running on.

Devine mentioned Smalltalk in their presentation; it’s a programming language and environment responsible for major early influence on the direction of object-oriented programming and the graphical user interface. A Smalltalk implementation consists of two parts: an interpreter/virtual machine that executes low-level operations and, running on top of it, a Smalltalk “image” (as in, a snapshot of the memory of the virtual machine). Most of the classes and objects that constitute Smalltalk exist in the image; only the interpreter is machine-specific. As a result, when the Smalltalk team wanted to create a new version of the language that was portable across platforms, they were able to simply write a new interpreter in portable C code (using a fairly ingenious process), and then the existing code in the Smalltalk image ran more or less unchanged.

HyperCard operated at a different level of abstraction: it consists of stacks that contain cards, and you can put text fields and buttons on those cards and add scripting commands to them. HyperCard stack files contain the data about your cards and the scripts you’ve configured, and then the HyperCard application runs those stacks. Since Apple discontinued HyperCard, there have been a number of attempts to create successor applications that work with the same stack files as HyperCard. However, most of these projects have struggled to achieve full compatibility and keep up maintenance. One significant reason for this is the complexity of the application that runs the stacks. There is a lot to implement if you want to be able to run HyperCard stacks, including a fairly sophisticated English-like scripting language.

A continuum of runtimes

Previously, I had thought of Smalltalk and HyperCard as taking two opposite approaches to dynamic systems: code-focused vs. GUI-focused. But I now see Smalltalk and HyperCard on a continuum between virtual machines and thin clients, along with Another World and Riverbed. All of these systems separate out a low-level runtime (the virtual machine/thin client) from the code that is executed by that runtime. The difference is how much complexity lives in the runtime. Smalltalk and Another World’s runtimes are small virtual machines, meaning there is not much that needs to be ported to put them on a new platform, but also that programming happens at a lower level (at least until you build up higher-level abstractions within the image, in the case of Smalltalk). HyperCard and Riverbed’s runtimes are larger client applications, allowing the user to work with higher-level abstractions from the start, but this means it is more work to port them between platforms.

The idea of a virtual machine may seem interesting to you, but is it applicable to your applications? It might be! Ask yourself, would I benefit from writing a virtual machine? This may seem like an intimidating proposition, but you don’t need to have a PhD in computer science to write something like a virtual machine. I don’t have any experience creating programming languages; I just saw an opportunity to move a little bit of logic out of hard-coded platform-specific code and into data that is configurable by the user.

• Would your users benefit from having a bit more control over the logic that runs in your application?
• Would your developers have an easier time supporting multiple platforms if your application (like Another World) had non-user-modifiable code that is stored in a platform-agnostic representation?

These questions might be worth considering if you’re worried about the effects of lock-in to platforms, the cost of porting your application to new platforms, or the long-term sustainability of your software. If that’s you, then learning more about the concept of virtual machines may give you ideas that could help. When Devine’s talk video is made available online, we’ll link to it from this post; watching it would be a great first step to dig in further.

Join the conversation about this post on our N.E.A.T community

Not a N.E.A.T. community member yet? More info.