Adding a JIT doesn't always help

Adventures in Accelerating OpenSCAD, part 2

Part 1 ended on a plan: rewrite the interpreted OpenSCAD DSL, JIT-compile it, inline the function calls and the speed would follow. I've been working A LOT on that since, and I have a confession — the plan was mostly wrong.

It DOES work, and it works well. The code is on github and you can run it on this site right now at hotchkiss.io/3d/editor. But the thing that made it fast was NOT the thing I said would.

First the part I didn't advertise: I ended up porting Manifold to Rust too. Elalish's kernel is excellent and I want to be clear none of the hard geometry math is mine — I re-typed it into a language I could parallelize deterministically. What I wanted was rayon so I could multithread the boolean kernel EVERYWHERE, including in the browser (the C++ WASM build ran single-threaded and overflowed the stack on even my basic models — the original complaint from part 1). A pure-Rust kernel gets me threads on the web, one language, no C++ toolchain in the WASM build and the parallelism is deterministic, so the output is testable against golden files.

So the architecture grew a few layers:

direction: down

editor: "editor: SCAD source"

lang: "fab-lang — pure-Rust OpenSCAD" {
  interp: "interpreter (tree-walker)"
  intr: "intrinsics\n(robot-written native)"
  jit: "Cranelift JIT\n(mostly idle)"
  interp -> intr: "hot fns"
  interp -> jit: "numeric tail"
}

geo: "GeoNode op-tree"
csg: "CSG cache — content-addressed" { shape: cylinder }
kernel: "fab-manifold\n(Rust port of Manifold, rayon-parallel)"
mesh: "mesh"

editor -> lang.interp
lang.interp -> geo
geo -> csg
csg -> kernel: "cache miss"
kernel -> mesh
csg -> mesh: "cache hit" { style.stroke-dash: 3 }

The JIT turned out pretty much useless

Here's the surprise. The Cranelift JIT — the whole point of part 1 — ended up pretty much useless on real models.

The reason boils down to this: it is FAR more expensive to hash the arguments to a function (to dispatch to or memoize its compiled body) than it is to just call the interpreted function in the first place. Per call the JIT is a monster — around 282× on a tight numeric function (the microbenchmark in part 1). But a real model doesn't spend its time in tight numeric functions. It spends it in the GEOMETRY — the Manifold booleans — and the numeric tail the JIT accelerates is a thin slice of the total. Speeding up a thin slice 282× while paying dispatch overhead on every single call nets out to about zero, and on some models a hair WORSE (eager compilation isn't free either).

To be clear the JIT is CORRECT — it's bit-identical to the interpreter and fuzz-tested to prove it. It's just not where the time goes.

What actually worked

Two things, plus the port.

1. Cache the geometry, content-addressed. Instead of hashing function arguments, hash the op-TREE as it lowers to geometry — a subtree that didn't change hands back its cached mesh with no recompute. Move one slider and the whole model doesn't rebuild, just the branch you touched: a one-parameter re-render dropped from ~70ms cold to ~15ms warm. This is the win the JIT was reaching for, one layer up where the units are milliseconds of BOOLEAN work, not nanoseconds of numeric work.
2. Robot-write intrinsics for the hot functions. The worst BOSL2 offenders (the ones doing a ton of list and vector math inside the DSL) get a native Rust body that replaces the interpreted one — roughly the same 122× as the microbenchmark, and unlike the JIT there's no per-call dispatch to pay for. Huge thanks to BOSL2 here; its parametric library is what makes the models worth accelerating in the first place, and every intrinsic is a bit-identical copy of their function.
3. Multithreading, finally. The Rust Manifold plus rayon parallelizes the boolean kernel — around 11× on a heavy real boolean (one nasty model went 7.2s → 0.6s), and it runs in the browser over WASM threads, which the C++ build couldn't give me.

Where a JIT WOULD still earn its keep

I'm not sorry I built it and I don't think it's dead — just aimed wrong. Where a JIT (or even a transpiler) would still serve me well is in dealing with CHANGE. Those robot-written intrinsics are HUGE — a bit-identical native reimplementation of each hot function — and that is going to be a maintenance nightmare long term (every upstream BOSL2 change is a port I have to redo and re-verify). A JIT sweeps the whole numeric tail automatically: no robot-writing, no drift. So it earns its keep on MAINTENANCE, not raw speed — a different bet than the one I opened part 1 with.

The numbers

The honest end result, across ~110 of my real models (each rendered whole once, cold, capped at a 30-second budget, on my machine — the harness lives in the repo). Full pipeline, both engines, same models:

• fab-scad is FASTER than OpenSCAD on 83 of the 84 models both engines finished — median about 4×, and 3.5× on total wall-time (81 seconds of my time versus 282 of theirs).
• It also renders 5 models OpenSCAD times out on ENTIRELY at 30 seconds. Those are the heavy BOSL2 pieces that started this whole thing — the point was never a benchmark, it was models that wouldn't finish.

A representative slice, milliseconds per whole render:

That last row isn't a typo. There is exactly ONE model where I'm slower — 2.4× slower — and it stays because leaving it out would be lying. I don't know WHY it loses yet; it's the one part that beat me and I haven't chased it down ... yet.

Two honesty notes on how these were measured. First, what's ON: no JIT (off by default, and it doesn't help anyway), no function-call cache, and each model is a fresh COLD render — so the interactive cache win from earlier is NOT in these numbers, it sits on top for the slider case. What IS on: the robot-written intrinsics and the multithreaded kernel. Second, this is the NATIVE build, where OpenSCAD gets its own multithreading — so it's the CONSERVATIVE comparison, OpenSCAD at full speed.

The browser is where it actually runs

This is mainly a WEB tool, so the number that matters is the one in the browser — and here the picture shifts, though not the way I assumed going in. Same models, both engines compiled to WASM, timed the same way (one cold render, in a Web Worker, render-only):

Two honest things. One: WASM taxes EVERYBODY. My own kernel is 4-5× slower in the browser than native on the heavy models — no SIMD, and boolean geometry is float-heavy — garage door goes from 2.6s native to 11s in WASM. So this isn't fab being magic in the browser, it's fab still FINISHING. Two: that's exactly where the gap lives. On the light models fab wins ~17× (it stays fast while OpenSCAD's single-threaded WASM pays its tax); on the heavy shared ones the ratio compresses to ~3× because both engines drag the same anchor. But push a little harder and OpenSCAD's WASM build stops finishing at all — pillwholder, which it renders FINE natively in 14 seconds, overflows the stack in the browser (RangeError: Maximum call stack size exceeded — the EXACT wall that opened part 1). fab renders it in six.

That last row is the whole reason I started. The browser number that matters was never the ratio — it's whether the model comes out at all.

One more, the whole cold start — not just the render. From clicking a ?model= link to the first model on screen (app download, boot, geom-worker spawn, first render) is about 4 seconds on localhost — near-flat whether the model is cornerrbrace or pilloholder, because the first on-screen render is the cheap interactive preview and the rest is fixed boot cost. That's the price of admission, paid once; every render after is the table above. (Localhost, so the ~8MB app download is instant — a real first visit pays that over the wire on top.)

Reproduce it

Every number here is from a script in the repo. The native table is tests/models_harness.rs; the browser head-to-head is perf/browser/ (build + run steps in its README — it re-fetches the exact OpenSCAD WASM snapshot by SHA and drives both engines in headless Chrome). The models are real prints in chotchki/scad-models — the ones above: corner_brace · angled_laptop_holder · ashtray · garage_door · pill_holder · traced_holder · bowtie/second_approach.

So: I set out to make it fast with a JIT and made it fast with a cache and a pile of robot-typed math instead. The JIT is still in there, idling, waiting for the day the maintenance bill on those intrinsics comes due and I don't feel like making Claude do it for me.

(Full disclosure: Claude pulled these numbers and cleaned up my crappy spelling)

Cover Image from Egon Eagle, CC0, via Wikimedia Commons