fix(quickjs): individual repl sessions use individual wasm module causing inefficient memory usage (#524)
### Summary
This PR optimizes memory usage by allowing us to use a single WASM
module. To do this we needed to fix two failure modes that occur when
multiple sessions share a single WASM module. Both are caused by the
async module loader creating asyncify suspensions (unwind/rewind cycles
on a module-global stack buffer) during import resolution.
__Failure 1__: Concurrent eval crash. The asyncify variant allows only
one async operation per module at a time. If two sessions eval
simultaneously and both hit an import, the second eval tries to suspend
asyncify while the first is already suspended. This crashes with
QuickJSAsyncifySuspended: Already suspended.
Fix: AsyncEvalQueue queues evalCodeAsync calls so only one runs at a
time.
__Failure 2__: Silent disposal corruption. When a session imports a
multi-file skill (e.g. an entry point that imports greet.js and
shout.js), each import causes an asyncify unwind/rewind cycle. The eval
succeeds. But when the session's runtime is disposed, the disposal does
not fully reset the module's asyncify stack buffer and the unwind/rewind
cycles left residual state (saved stack frames, rewind pointers) that
emscripten manages at the module level, not the runtime level. The next
session creates a fresh runtime on the same module, but the corrupted
asyncify state causes module loader callbacks to silently never fire.
Fix: synchronous module loader. Skill files are preloaded into a Map
before eval starts. The loader callback does a plain map.get(). No
await, no asyncify suspension, no residual state on disposal.
Note: The eval queue alone does not fix failure 2. The corruption
happens at runtime disposal, which runs outside the queue. Both fixes
are required.
#### What This PR Adds
- Replaces per-session WASM module instantiation with a process-global
singleton shared by all sessions. Each session still gets its own
runtime and context (full isolation); only the underlying WASM module is
shared. Reduces WASM memory from ~16 MB per session to ~16 MB total.
- Adds AsyncEvalQueue to serialize evalCodeAsync calls across sessions,
enforcing the asyncify one-concurrent-async-call-per-module constraint.
- Makes the module loader synchronous by preloading skill files into an
in-memory cache before eval (preloadReferencedSkills). Why is this
needed? An async module loader causes asyncify suspensions on each
import. Disposing a runtime after multi-file skill imports corrupts the
shared module's asyncify state and silently breaks the loader for all
subsequent sessions. The sync loader eliminates asyncify suspensions
from imports entirely.
#### Synchronous Module Loading
As stated above, synchronous module loading is a necessary change due to
async module loading causing asyncify suspensions on each import which
corrupts the singleton WASM module's asyncify state which silently
breaks the loader for all subsequent sessions. The following
demonstrates the before and after regarding this change:
__Before__:
User code: `import { run } from "@/skills/demo"`
1. eval() is called
2. evalCodeAsync() runs the code
3. QuickJS hits the import, calls the module loader callback
4. The callback is async: await ensureSkillLoaded("demo")
→ fetches /skills/demo/index.js from the backend (network I/O)
→ this await causes the first asyncify suspension in the WASM module
5. index.js has: import { greet } from "./greet.js"
6. QuickJS calls the loader again for greet.js
→ await ensureSkillLoaded() → network fetch
→ The second asyncify suspension in the WASM module is hit
7. index.js has: import { shout } from "./shout.js"
8. QuickJS calls the loader again for shout.js
→ await ensureSkillLoaded() → network fetch
→ third asyncify suspension in the WASM module is hit
9. eval completes, runtime is disposed
10. runtime.dispose() is called to clean up the session
11. The runtime disposal does NOT fully reset the module's asyncify
stack buffer. The 3 unwind/rewind cycles left residual state (saved
stack frames, rewind pointers, etc.) that the dispose path doesn't know
to clean up, because asyncify bookkeeping is managed by emscripten at
the module level, not by QuickJS at the runtime level.
12. Next session creates a fresh runtime on the same module, but the
module's asyncify machinery is corrupted. When QuickJS tries to call the
new session's module loader, the stale asyncify state causes the
callback to silently not fire. No crash, no error. Imports just resolve
to nothing.
__After__:
User code: `import { run } from "@/skills/demo"`
1. eval() is called
2. preloadReferencedSkills() runs BEFORE evalCodeAsync():
→ scans source code for "@/skills/*" specifiers
→ finds "demo"
→ fetches all 3 files from the backend into a Map:
"@/skills/demo" → source code
"@/skills/demo/greet.js" → source code
"@/skills/demo/shout.js" → source code
→ (this is async but happens OUTSIDE the WASM module so no asyncify)
3. evalCodeAsync() runs the code
4. QuickJS hits the import, calls the module loader callback
5. The callback is sync: map.get("@/skills/demo") → returns source
immediately
→ NO asyncify suspension
6. QuickJS resolves greet.js → map.get("@/skills/demo/greet.js") →
immediate
→ NO asyncify suspension
7. QuickJS resolves shout.js → map.get("@/skills/demo/shout.js") →
immediate
→ NO asyncify suspension
8. eval completes, runtime is disposed
9. Zero asyncify suspensions occurred → no residual state → module is
clean
10. Next session works perfectly
### Tests
- 4 new AsyncEvalQueue unit tests: serialization (no interleaving of
concurrent ops), return values, error isolation (rejected op doesn't
break queue), insertion order preservation
- All 160 existing tests pass (session, skills, middleware, transform,
utils, eval queue)
### Memory Benchmarking
#### Per-session module (before)
| Sessions | RSS Delta (MB) | Per-Session RSS (MB) | WASM Memory (MB) |
Module Init (ms) | Setup (ms) |
|----------|----------------|----------------------|------------------|------------------|------------|
| 1 | 0.53 | 0.53 | 16 | 11.4 | 15.9 |
| 5 | 2.13 | 0.43 | 80 | 30.2 | 37.9 |
| 10 | 5.28 | 0.53 | 160 | 31.1 | 42.8 |
| 15 | 5.53 | 0.37 | 240 | 57.4 | 72.1 |
| 20 | 6.80 | 0.34 | 320 | 73.6 | 91.1 |
---
#### Singleton (after)
| Sessions | RSS Delta (MB) | Per-Session RSS (MB) | WASM Memory (MB) |
Module Init (ms) | Setup (ms) |
|----------|----------------|----------------------|------------------|------------------|------------|
| 1 | 0.61 | 0.61 | 16 | 6.7 | 9.1 |
| 5 | 0.59 | 0.12 | 16 | 12.5 | 28.5 |
| 10 | 0.91 | 0.09 | 16 | 12.2 | 37.0 |
| 15 | 1.81 | 0.12 | 16 | 12.0 | 45.1 |
| 20 | 1.70 | 0.09 | 16 | 10.4 | 50.4 |
---
#### Comparison at 20 sessions (realistic workload)
| Metric | Before | After | Improvement |
|------------------------|---------|--------|-----------------|
| WASM virtual memory | 320 MB | 16 MB | 20x reduction |
| RSS delta | 6.80 MB | 1.70 MB| 4x reduction |
| Module instantiation | 73.6 ms | 10.4 ms| 7x reduction |
| Setup time | 91.1 ms | 50.4 ms| 1.8x reduction | C
Colin Francis committed
2cbd5245a43fb1ba97fa532c1942a8903e090cfa
Parent: 6da7007
Committed by GitHub <noreply@github.com>
on 5/6/2026, 10:43:23 PM