Wasmi 2.0 - Engineering of the Fastest Wasm Interpreters

In my last post about Wasmi 1.0 I promised a fundamental engine overhaul for the future Wasmi version. The future is now! 1Wasmi is an efficient and feature-rich WebAssembly (Wasm) interpreter. It is an excellent choice for IoT devices, plugin systems (Typst, Zellij, Josh), cloud hosts, smart contracts (Soroban, Ripple) and even for your lightweight game consoles (Firefly Zero).Before going into all the details, a huge thank you to the Stellar Development Foundation (SDF) that has been sponsoring the Wasmi project since October 2024. Without their sponsorship, the Wasmi project wouldn’t be where it is today. Also special thanks to Felix Kutzner for proofreading the article and suggesting many improvements.Wasmi 2.0 ReleaseToday, I am happy to announce that after eight months of focused work, Wasmi 2.0 is finally done and ready to use.This release focuses on execution performance: Wasmi 2.0 runs ~2.2x faster than Wasmi 1.0 in geometric mean across the wasmi-benchmarks suite on an Apple M2 Pro.Wasmi 2.0 also ships new knobs, such as the validate crate feature, that significantly reduce its binary artifact size. 2 Some user-requested features, such as stable fuel metering 3, support for WebAssembly’s deterministic profile and an improved Wasmi CLI tool, also made it into this release.RepositoryRelease NotesMigration Guide: 1.0 → 2.0CrateDocumentationYou can always make me happy with a star at GitHub!Where Wasmi 2.0 Landed2.2x faster than Wasmi 1.0 is great, but how does Wasmi 2.0 fare against its competition?For this, I have benchmarked Wasmi 2.0 against some of the fastest portable Wasm interpreters: 4Wasm3WAMR fast-interpreterWasmtime PulleyMakepad StitchWasmi 1.0Note: Wasmi 2.0 was inspired by all the interpreters above!The benchmarks were conducted using the wasmi-benchmarks project which should make it possible to easily reproduce them on your own machine.I ran the benchmarks on three different hardware setups to make interpreter preferences visible:Apple M2 ProAMD EPYC 7763Intel Xeon Platinum 8370CNote that this is just a peek of the total benchmarks and runtimes supported by the wasmi-benchmarks project but it provides a good overview.Geometric MeanThe following two plots show the geometric mean across all execute and all startup benchmarks of the wasmi-benchmarks suite from the above Wasm runtimes.Note: I had to use logarithmic scaling for startup because Wasmtime Pulley is quite an outlier. 5Despite the focus on execution performance in Wasmi 2.0, its startup performance is still outstanding and mostly on par with its previous version. 6Conclusion: BenchmarksIt is fair to say that Wasmi 2.0 clearly belongs to the category of the fastest portable Wasm interpreters.In a follow-up article I will present all the results and findings of the wasmi-benchmarks suite and put each of its many supported Wasm runtimes into the spotlight it deserves.What made Wasmi 2.0 so fast?Wasm3 and Stitch share a lot of similarities with Wasmi 2.0 under the hood. While this section details what ideas made it into Wasmi 2.0 it also discusses, where relevant, the similarities and deliberate differences from them.Note: This section assumes a basic understanding of Wasm and interpreters.New Modes of Instruction DispatchAs promised in the original Wasmi 1.0 blog post, Wasmi 2.0 now has four different modes of dispatching instructions:ModeDescriptionCrate FeaturesDirect-Threaded CodeThe fastest configuration that is used by both Wasm3 and Stitch. It embeds the function pointers directly into the internal IR of the interpreter and uses tail calls to jump from one instruction handler to the next.-Indirect-Threaded CodeVery similar to Direct-Threaded Code, but embeds op-codes into the internal IR and uses a jump table to map an op-code to its instruction handler’s function pointer upon dispatch. Roughly 10-15% slower than Direct-Threaded Code, but uses significantly less memory for its IR.indirect-dispatchSwitch-LoopThis is the technique used in Wasmi 1.0. It is the naive way to build interpreters using a loop and a switch (or match). Unfortunately, it leaves a lot of performance on the table, especially on Apple Silicon.portable-dispatch + indirect-dispatchCall-LoopThis calls the next instruction handler within a loop without tail calls. Unfortunately, it is very slow and not memory efficient, therefore I cannot recommend using it. It exists only because portable-dispatch and indirect-dispatch are independent crate features, so this combination simply falls out of the configuration matrix.portable-dispatchWasmi users should useDirect-Threaded Code: if they want to maximize interpreter performance.Indirect-Threaded Code: for a good balance between interpreter performance and memory usage.Switch-Loop: for running on platforms that do not support tail calls.Wasmi 2.0 ships the auto-dispatch crate feature that automatically uses threaded-code-based configurations where possible. 7How Do Instruction Dispatch Modes Perform?Note: CoreMark results for Direct-Threaded Code do not perfectly match the ones from above since it was a different run and we used the wasm-coremark-rs project instead.Despite these extreme differences in performance, all of these instruction dispatching modes share the same interpreter execution logic and architecture under the hood.If you are interested in how the instruction dispatch selection in Wasmi works in detail, you can find the code here: Wasmi Dispatch SelectionExecution Handler SignatureBefore execution, Wasmi translates the Wasm bytecode to Wasmi IR.Each Wasmi IR instruction has its own instruction handler (or execution handler) which defines how the instruction is executed.In Wasmi 2.0, all instruction handlers share the same signature:fn( store: &mut PrunedStore, // A reference to the `Store<T>` that is associated to the execution. ip: Ip, // The instruction pointer. sp: Sp, // The stack pointer. mem0: Mem0Ptr, // The pointer to the data of the default linear memory: `(memory 0)` mem0_len: Mem0Len, // The number of bytes of the default linear memory. instance: Inst, // A pointer to the Wasm instance that is used by the currently executed function. ireg: Ireg, // Accumulator register for integer and reference values. freg32: Freg32, // Accumulator register for `f32` values. freg64: Freg64, // Accumulator register for `f64` values. ) -> Done; // State used to signal traps or successful halts. The store argument is basically a Store<T> that was pruned by its T type. This is important since instruction handlers are not allowed to be generic. The store is used for fuel metering, host calls, memory.grow and table.grow operations.The ip argument is the instruction pointer which tells the executor where in the stream of encoded instructions it is and which instruction it has to decode and execute.The sp argument is the position of the currently executed function within the value stack.The mem0 and mem0_len arguments are used for optimized access to the default memory (memory 0). This is very common in Wasm even when using the Wasm multi-memory proposal.The instance argument is used to load Wasm instance related objects such as globals, functions, tables, memories, data and element segments. We will go into greater details later in the post.The ireg, freg32 and freg64 arguments are so-called accumulator registers which are used to efficiently store intermediate results between instructions. We will go into greater details later in the post.The Done result is just a bit pattern that tells the executor why execution halted. More detailed information is communicated via the store for later retrieval.Problem: Calling Conventions7 out of the 9 arguments in Wasmi’s instruction handlers require passing their values in general-purpose registers (GPRs), namely store, ip, sp, mem0, mem0_len, instance and ireg.However, common calling conventions such as sysv64 only provide up to 6 GPRs for integer arguments. A 7th integer argument would trash performance because it would have to be spilled to the stack on every dispatch. Both Stitch and Wasm3 circumvent this issue by using only 6 and 4 GPRs respectively.The simple solution is to turn one of Wasmi’s GPR arguments into a floating-point value where necessary. The instance argument was chosen since it is used only for relatively expensive operations anyway.Benchmarks show that the integer-to-float register domain move isn’t a big deal.Note: the currently unstable preserve_none ABI might be able to improve this situation in the future once it becomes stable and available on more platforms.Accumulator RegistersHow Wasmi 1.0 WorkedWasmi 1.0 pervasively uses stack offsets (stack slots) for operands and results of IR instructions.A simplified i64.add instruction handler computing res = lhs + rhs is shown below, where res and lhs are stack slots, and rhs is an immediate i64 value:fn i64_add(ip: Ip, sp: Sp, ..) -> Done { let res: Slot = decode_slot(ip); let lhs: Slot = decode_slot(ip); let rhs: i64 = decode_i64(ip); let lhs: i64 = lhs.load(sp); let sum: i64 = lhs + rhs; res.store(sp, sum); ip.offset(encode_size::<i64_add>); next!(ip, sp, ..) } Decode the result Slot from ip.Decode the left-hand side lhs operand Slot from ip.Decode the right-hand side rhs: i64 operand from ip.Load the value from lhs. (sp[lhs])Compute the sum sp[lhs] + rhs.Store the sum into sp[result].Offset ip to point to the next instruction handler.Execute the next instruction handler. 8How Wasmi 2.0 WorksWasmi 2.0 introduced the three new accumulator registers: ireg, freg32 and freg64. This allows Wasmi 2.0 to load and store instruction operands and results from and to actual hardware registers.A simplified i64.add example that computes res = lhs + rhs where res and lhs refer to the ireg accumulator and rhs is a i64 immediate value would look like this:fn i64_add(ip: Ip, sp: Sp, ireg: i64, ..) -> Done {