Composability: Designing a Visual Programming Language

Lattice is a high-performance visual scripting system targeting Unity ECS. Read more here.

I wanted to write a few posts on the design of Lattice as a language. Today, let's focus on “composability”. This is intuitively something we desire in programming languages. Some systems feel like they are effortlessly reconfigurable, recombinable, and then others.. just don’t. Some languages seem to actively reject our efforts at organization.

So what makes a system composable? I see two major properties:

Self-Similarity — Composable systems usually have a simple primitive that is self-similar. Think of a game grid, tetris pieces, repeating tessellations, etc. Self-similarity means most things can plug into other things. They may not do something useful, but they can be infinitely re-arranged.
Merging / Splitting — Two groups of primitives can be merged into a single primitive, and that 'merged primitive' acts just like any other primitive. Similarly, a primitive can be split into several 'pieces' and each of those pieces acts like any other primitive.

Take Legos, perhaps the most composable system in existence. So composable, in fact, that talking about them is almost reductive. But they follow these rules deeply:

Notice how not only can Legos be split into separate pieces, but they can be split in many different ways. Put another way, Legos have lots of seams: places where they can be split apart into component chunks.

Lego chunks can be split along many different seams.

A system with many seams is indicative of a very composable system: new behavior is built purely through composition, not by defining new primitives.

I think about composability a lot in my game design work, why do some game mechanics have so much emergent behavior? We can see splitting/merging at work there too. Factorio, with its endlessly recombinable and splittable factory tiles. Or a roguelike deck-builders with their infinitely splittable/mergeable decks. Or an action game with 20 different abilities giving you an incredible number of seams to rip apart builds and reconfigure them.

Blueprints are not composable.

Let's get back to Lattice, and visual programming language design. Unreal Engine’s Blueprints are a common design for visual programming, roughly mirroring code. But despite how easy it is to sketch something out, they seem to reject organization.Common wisdom from large teams in UE is “design in blueprints, then rewrite in C++”. It's shockingly tough to split a messy graph into reusable, smaller nodes. It's also pretty tough to merge several nodes into a single multi-purpose node. Why is that?

In my opinion, the problem is execution wires.

Execution wires don't have a trivial merge operation. Let's say I have two nodes I want to combine into a single node:

What do you do about the execution wires? It's illegal to split or merge an execution wire. Doing so would require a fundamentally parallel execution environment.It’s fun to muse on a language where splitting an execution wire means “run these operations in parallel”. It’s a bit of a can of worms, though, that PL folks have been attempting for decades. You must manually order them, instead. But which one should execute first? Even if it doesn’t actually matter, the semantics of the language require you to pick an option.

In this situation, Blueprints just gives up, exposing two execution wire inputs — the worst of all worlds. The issue here isn't Blueprints, itself. Execution wires are just a fundamentally non-composable primitive.Foot note:
Why does code work better?

It's interesting to ask why text code doesn't seem to have this problem. After all, C++ has 'execution wires', so to speak, between each line, but doesn't have these compositional issues. However, the execution wires in text code are 'implicit' -- it's based on the linear order of the text in the file. Every time you make an edit, you're answering the ordering question.

With a visual programming language lifted to a 2D plane, we can't rely on that luxury because there is no implicit linear order of nodes. Of course, we can implicitly order nodes in other ways.

A Better Option - Value Graphs

What happens if we design a language without execution wires? Actually, there's plenty of precedent for this already:

Houdini / Blender Geometry Nodes (Mesh Manipulation Tools)
VVVV, TouchDesigner, Cables (Live Effects Tools)
MaxMSP, Pure Data (Audio Synthesizers)
ShaderGraph, Animation Blueprints (Shaders, Animations)

All of these languages are value-based expression graphs under the hood. You're applying operations to input values, to produce certain output values, sometimes with some additional side effects.

If there's no execution wires, how do you control the ordering of execution? Usually, you don't have to. When a node precisely defines the data it requires, the order of execution is implicit. When does a node run? After all its inputs have finished.

Blueprints actually has this in the form of Pure nodes. No surprise, these nodes are much easier to combine and reuse throughout your graphs. Sadly, most BP nodes are not pure, so the functionality is limited in its usefulness.

These are fundamentally composable! It's perfectly valid to split an incoming value wire (it's just a copy of the same value on either end).

Something else to notice: in a value graph, any cut of the graph is a valid way of creating a sub-graph. The edges that that cross the 'cut' are precisely your inputs and outputs. Here we’ve selected a group of sub-nodes with 1 input crossing, and 1 output crossing:

Here’s another (more invasive) cut, with 1 input edge and 3 output edges:

Put another way, value graphs have lots of seams. We can cut them up any way we like, simplify nodes, apply general transformations, and the graph stays valid. There are no ‘edge cases’. The same cannot be said for execution wires.

So, Lattice borrows from this lineage of programming languages. That get us an incredibly composable computation model, however we need a few super-powers to give us a more expressive language as a whole. I'll chat more about that next time.

Until then.