Files

T

evan 376d302e1d Reader opens in WASM: bump QEMU store-TB workaround; document panic

Bumps third_party/qemu to the wasm32 store-TB TCI workaround so the
browser reader no longer panics in uzlib font decompression. Adds
docs/wasm-inflate-panic.md with the full investigation (native control,
non-deterministic reject, JIT promotion) and next steps. Updates ISSUES:
the panic is worked around; the reader now inverts into muddled
grayscale after the initial BW page (new top reader defect). tests
navigate the 4th Open press into the reader.

2026-07-21 09:16:40 -04:00

6.5 KiB

Raw Blame History

WASM-only reader panic in uzlib inflate

Living record of the investigation into the browser-only crash when opening a book. Native QEMU (same submodule) does not crash, so this is a wasm32 TCG backend bug. Update as things are ruled in/out.

Symptom

Opening a book in the browser panics; native opens the same book fine.

Invalid read at addr 0x3FCB7999, size 1, region '(null)' size 0x0 owner '(none)' ram=0, reason: rejected
Guru Meditation Error: Core 0 panic'ed (Load access fault). Exception was unhandled.
MEPC : 0x4201e210  RA : 0x4201e18c  SP : 0x3fcb3810

Established facts

Fault site (via same-version local ELF addr2line, not byte-identical): uzlib_uncompress (tinflate.c:613) ← tinf_inflate_block_data (tinflate.c:476) ← InflateReader::read ← FontDecompressor::decompressGroup ← prewarmCache. "Indexing" = the reader inflating compressed glyph groups from flash into a malloc'd temp buffer.
Input is static flash data, independent of SD. The compressed font bank is in the firmware image; the SD file only selects which glyph groups get prewarmed (the characters present in the text).
Native renders the reader ("chapter one") with no fault — proven with _scratch/native-reader-repro.py + QMP screendumps. Confirms the firmware, the font data, and the decompression logic are all correct.
Fault address is non-deterministic: 0x3FCB7999 and 0x3FCB7997 across runs. A static machine-map boundary would fault at a fixed address. Drift ⇒ a computed pointer is slightly wrong, i.e. runtime/codegen, not the memory map.
Rejected region is bogus: region '(null)' size 0x0 owner '(none)' ram=0. The address lands in mapped DRAM (0x3fc80000+0x60000) but the wasm softmmu TLB routed it to the IO slow path with a zero-size section. Hallmark of a bad address/TLB result from a JIT-compiled TB.
Menus work. Home/file-browser render crisp text through the same font system and the same TLB fast path, so the TLB codegen is not broadly broken — the bug is pattern/data specific in a hot, JIT-promoted TB.
User observation: firmware-written crash logs open fine; prepopulated files do not. Consistent with data-dependence — a crash log's character set prewarms different glyph groups than lorem-ipsum, avoiding the group whose compressed byte pattern trips the miscompiled path.

Architecture notes (why JIT is implicated)

tcg/wasm32.c: TBs run on the TCI interpreter until executed INSTANTIATE_NUM times, then get compiled to native WASM (instantiate_wasm) and run as added table functions. The inflate hot loop gets promoted.
WASM_TCI_ONLY_ENTRY forces a TB to stay on TCI forever; wasm_tci_only_tb (per-TB, reset each translation) requests that.
Dynamic promotion must stay on for usable performance (all-TCI starves the guest watchdog), so a permanent fix must correct codegen, not disable JIT.

Hypotheses

#	Hypothesis	Status
H1	A store-containing TB is miscompiled (the cleanup removed `wasm_tci_only_tb = true` from `tcg_out_qemu_st`); the compiled TB corrupts a pointer a later load dereferences	confirmed (E1)
H2	Compiled load codegen bug (`tcg_wasm_out_qemu_ld` / `_direct`)	untested
H3	TLB fast-path codegen edge case (`tcg_wasm_out_tlb_load`) hit only by certain access patterns	untested
H4	asyncify/fiber unwind corrupts vCPU state mid-inflate	unlikely (inflate makes no async import calls)
H5	mimalloc / linear-memory / threads issue	unlikely (native heap layout differs, no crash)

Experiments

E1 — force store-containing TBs to TCI (tests H1) ✅ CONFIRMED

Re-added wasm_tci_only_tb = true; to tcg_out_qemu_st in tcg/wasm32/tcg-target.c.inc.

Result: panic gone. make browser-test reaches the reader and renders a correct page (black lorem-ipsum on white, footer 80% example 1/3 33%). No reject, no Guru Meditation, 1 boot, 0 watchdog panics. New refresh frames (dtm2_white=0.891) appear past the old 0.960 stall.
Conclusion: the compiled qemu_st codegen is the bug. Forcing store TBs to TCI is a working workaround, not the fix — every store-containing TB then runs interpreted (broad perf cost; prior attempts reported watchdog starvation under sustained load, though this finite-burst run was clean).
The reader first paints a correct BW page (black text on white), then a follow-up grayscale refresh pass inverts it into muddled white-on-black text. That second issue is real and still open (see ISSUES.md "Reader renders inverted grayscale"); it is independent of this panic fix. The browser-test wait-for-stable happened to screenshot the initial white page.

Caveat on E1's specificity

Forcing store-containing TBs to TCI drops the whole TB (its loads, stores, and arithmetic) to the interpreter. So the bug is in a TB that also has a store, not provably the store op. tcg_wasm_out_qemu_st_direct and tcg_wasm_out_qemu_ld_direct are byte-for-byte symmetric and look correct in isolation, which points away from the direct emit and toward the store path's surrounding block/rewind/helper scaffolding in tcg_wasm_out_qemu_st, or an interaction with a co-located op.

Next options

Ship E1 as a guarded workaround — reader works now; measure real navigation/page-turn perf before judging it unacceptable.
Localize precisely — bisect by making the TCI trigger conditional (store width; presence of the miss/helper path; TB size) to shrink the suspect set, then read the generated wasm for that TB.

Diagnostics currently in the tree (uncommitted)

system/memory.c: reject log prints region size/owner/ram to identify the faulting region.
hw/display/xteink_x3_eink.c: [eink] refresh popcount trace (per-plane white ratio) — also used for the separate grayscale-rendering issue.

Reproduction

WASM: make browser-test (the test does 4 confirm presses: Browse Files → Test → highlight → Open). Panic appears in _scratch/browser-raw-sd.log.
Native (control, no panic): nix develop -c python3 _scratch/native-reader-repro.py, then inspect _scratch/native-step4-reader.ppm.

Ruled out

Not SD / filesystem: fixed separately (raw FAT32, SSI-SD token, CMD13); the file reads correctly and "Indexing" begins.
Not firmware / font data: native decompresses and renders identical input.
Not the machine memory map: fault address drifts; native map is the same.

6.5 KiB Raw Blame History