Benchmarking Mesh Booleans: First Results

We are working on building a public, reproducible benchmark for mesh boolean operations. The full suite is not ready yet, but the runner infrastructure is already usable and we have first interesting results from a non-trivial test case.

The case itself is relatively simple: subtract 9 tools (tori and smaller spheres) from a large sphere, one after another. All inputs are clean, watertight, regular tessellations. About 100K input triangles total, nothing exotic:

We tested 17 implementations across open-source libraries, commercial tools, and research prototypes. Of those, 12 produced correct or near-correct results. 5 either crashed, returned empty output, or got the geometry wrong by more than 1%. The timing spread among correct results is roughly 17,000x.

The runner programs that wrap each implementation are already open source: github.com/solidean/boolean-benchmark-runners

The raw input meshes and per-operation data for this case are at github.com/solidean/bench-blog-data, so you can reproduce the numbers yourself.

Results

Each runner executes the same 9-step subtraction chain and reports per-operation time. Correctness is checked against a reference area and volume (see Notes for details).

Solidean completed the full 9-step chain in about 27 ms with a correct result. The next fastest correct results came in around 85 ms and 109 ms (both from algorithms that claim robustness but not exact constructions). The "Rel." column shows runtime relative to Solidean.

Roughly half of the implementations matched the canonical answer (area and volume matching to 4 decimal places). Roughly a quarter was slightly off (some holes and flaps, visually "OK" for a preview). And the final quarter could not complete the chain at all or produced clearly wrong geometry.

Even restricting to implementations that produce correct results, the spread between fastest and slowest is roughly 17,000x.

What makes this hard

The inputs are about as friendly as it gets: regular tessellations, no self-intersections, consistent winding, no degeneracies. A single subtraction of two of these meshes is straightforward for most implementations. The difficulty really comes from iteration.

Each boolean's output mesh becomes the input to the next operation. Small numeric drift introduced by a triangulation choice in step 3 changes the geometry that step 4 has to cut. Errors compound across the chain. If a single cut triangle is misclassified in one step, it becomes a hole or wrong solid in the next. An implementation that is "almost right" on a single operation can drift into something visibly wrong by step 9. (They usually die pretty fast after introducing the first real topological error, we basically see no self-healing behavior at all.)

While the inputs are "easy", their arrangement is not. The primitives are concentric or near-concentric. Subtracting a torus from a sphere produces near-tangent edges and near-coplanar faces, exactly the configurations that stress numerical robustness. The running mesh also grows in complexity: after 9 subtractions it holds roughly 33K triangles, so later operations cut a complicated surface, not two clean primitives. We did not intend to make it a "hard" case, but symmetry has a way to make geometry processing complicated. And users arrange things symmetrically all the time.

Toward a public benchmark

This is one test case. The full benchmark will cover more input geometries, different operation types, and stress tests with higher iteration counts.

The runner infrastructure and build scripts are already open source at github.com/solidean/boolean-benchmark-runners. Each runner is a standalone program that reads a JSON task description, calls the library, and writes back timings and result meshes. Adding a new library means writing one small wrapper.

We will publish more cases as they are ready.

Notes

(This section is boring. It's there to be transparent and to make future comparisons clean.)

Input geometry

10 input meshes, all authored in Blender as standard primitives:

Total input geometry: approximately 100K triangles across the 10 meshes. All meshes are watertight, closed, oriented 2-manifold, with no self-intersections. Coordinate range is [-1.5, +1.5]^3. All meshes overlap in space (so the booleans do real work, not no-op subtractions of disjoint volumes).

Operation sequence

The operation chain is iterative:

• result_0 = A
• result_i = result_{i-1} - tool_i, for tool in {B, C, D, E, F, G, H, I, J}

9 boolean differences total. The running result feeds the next step.

Correctness

The reference ("canonical") result has area = 13.4224 and volume = 2.3513. Results matching to 4 decimal places are marked "correct". Results within ~1% of volume and area are marked "~correct" (often these are topological artifacts and small non-manifold regions).

Libraries and versions

(via our github.com/solidean/boolean-benchmark-runners)

Measurement

"Time" in the results table is the sum of per-operation time and import time (converting the input mesh into the library's internal representation).

Import time is important to consider because many algorithms build acceleration structures or otherwise do heavy conversions. While sometimes you reuse the same input many times, for this one we took the "everything is used once" assumption.

We did NOT measure any IO in this. Only "indexed triangle mesh to internal mesh structure" and the raw boolean operations. The exact code blocks that are timed can be seen in the runner code.

System:

• AMD Ryzen 9 5900X 12-Core @ 3.7 GHz
• 64 GB of DDR4 2666 MT/s RAM (yes, I need to upgrade...)
• Windows 11
• All algorithms were free to run in parallel as they want (many did)

Limitations

This is a single test case with specific library versions and default configurations. Results may differ for other geometries, operation types, iteration counts, or library settings. Some libraries may have configurations or modes we did not test. The versions listed above were the latest stable releases at the time of testing.