evan/xteink-web-emulator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
f972fbe78bf6be34fe7e118dbd557ed3044a38ea
xteink-web-emulator/HANDOFF.md
T
evan f972fbe78b initial commit
2026-07-19 19:23:51 -04:00

15 KiB
Raw Blame History

Handoff — xteink-web-emulator

Everything below is hard-won context from the session that built the native X3 emulator. If it isn't in this repo or this doc, it's gone. Read this before touching QEMU internals. Pairs with ROADMAP.md (the plan) — this is the "how it actually works and what will bite you" file.

0. TL;DR of current state

  • The xteink-x3 QEMU machine boots unmodified canonical firmware to the home screen, mounts a FAT32 SD image, and takes button input. Verified with QMP screendump.
  • All Phase-1 device models live in one patch: qemu/patches/0001-xteink-x3-machine.patch, applied onto espressif/qemu tag esp-develop-9.2.2-20260417 by scripts/build-qemu.sh.
  • Next: X4 sibling machine → compile to WASM (qemu-wasm) → static browser UI.

1. Environment / hardware realities (READ FIRST)

  • The dev box is ~15 yr old, 4 GB RAM. A full QEMU build is ~1660 objects and takes a long time. Always -j2 (more OOMs the linker) and use very long timeouts (I used 7200 s) for any ninja/pio/nix build. Do not treat a long-running build as hung.
  • First full QEMU build ≈ many minutes; incremental after a patch edit is seconds (only the touched .c + relink). Iterate incrementally; never rm -rf build.
  • Nix sandbox capability leak: running FHS/bwrap-wrapped tools (PlatformIO, and the prebuilt release QEMU) fails with bwrap: Unexpected capabilities but not setuid, old file caps config?. Workaround — clear ambient caps: 
    SP=$(nix build nixpkgs#util-linux.bin --no-link --print-out-paths)/bin/setpriv
"$SP" --inh-caps=-all --ambient-caps=-all nix develop <flake> --command <cmd>
    Our source-built QEMU is a normal nix-linked ELF and does NOT need this (that's a bonus reason we build from source, not just to patch).

2. Building espressif-qemu from source (the fiddly part)

scripts/build-qemu.sh encodes all of this, but here's why:

  • Reuse nixpkgs' QEMU build env: nix develop nixpkgs#qemu --command … (or the repo devShell, which has inputsFrom = [ pkgs.qemu ]). Getting glib/ pixman/meson/ninja right by hand is painful; borrow qemu's.
  • slirp: the fork unconditionally compiles net/slirp.c even with --disable-slirp (build-system bug), so you must enable it. nixpkgs provides libslirp; --enable-slirp + libslirp on PKG_CONFIG_PATH works, or it falls back to the bundled subproject.
  • libgcrypt: the ESP32-C3 crypto devices (esp32c3_aes.c, rsa, ds, xts) are gated on gcrypt.found(). QEMU prefers gnutls and skips gcrypt unless you pass --enable-gcrypt. Without it the build succeeds but the machine fails at runtime: unknown type 'misc.esp32c3.aes'.
  • Meson caches pkg_config_path: exporting PKG_CONFIG_PATH after the first configure is ignored on reconfigure. Use meson configure build -Dpkg_config_path=… or pass it to ./configure.
  • Config that works: --target-list=riscv32-softmmu --enable-gcrypt --enable-slirp --disable-werror --disable-docs --disable-tools.
  • /tmp/qsrc is where I built it this session and is EPHEMERAL (lost on reboot). The patch is the durable artifact; make qemu reclones+rebuilds.

3. How espressif-qemu's esp32c3 machine is structured

  • File: hw/riscv/esp32c3.c. It hand-builds the SoC (no device tree).
  • Catch-all MMIO: esp32c3_io_ops (esp32c3_io_read/write) covers the whole peripheral window from ESP32C3_IO_START_ADDR (== DR_REG_UART_BASE == 0x60000000) for 0xd1000 bytes. Any peripheral without a dedicated device falls here and reads 0 / ignores writes. That's why unmodeled peripherals silently misbehave instead of faulting.
  • Turn on #define ESP32C3_IO_WARNING 1 (top of the file) to log every unsupported access as [ESP32-C3] Unsupported read/write to $ADDR. This is the single most useful debugging lever. It's noisy (megabytes/sec), so run ~12 s and bucket by address: 
    grep 'Unsupported read' log | sed -E 's/.*\$//' | sort | uniq -c | sort -nr | head
    The hottest address is almost always the next thing to model. Leave it 0 in committed code.
  • Devices that already exist in the fork: UART, GPIO (barebones — we fleshed it out), SPI1 (flash only), GDMA, AES/SHA/RSA/HMAC/DS/XTS, RTC_CNTL, systimer, timg, efuse, intmatrix, cache, JTAG, TWAI, esp_rgb (a QEMU-specific virtual framebuffer, NOT real X3 hardware — we disable it on the xteink machine).
  • Devices that did NOT exist and we added: SAR ADC, SPI2, the UC8253 panel, and real GPIO register behavior. There is no I²C controller model — this matters for X3/X4 detection (see §7).

4. Peripheral register facts (discovered, verified against firmware)

Base addresses from ESP-IDF soc/esp32c3 headers; all confirmed live.

SAR ADC — DR_REG_APB_SARADC_BASE = 0x60040000

  • The original boot blocker: firmware polls INT_RAW (+0x44) bit 31 (ADC1_ONESHOT_DONE) after starting a conversion; the unmodeled reg returned 0 forever → busy-loop with interrupts off → interrupt-WDT panic (Guru Meditation … Interrupt wdt timeout). The panic dump's S0/FP = 0x60040000 literally pointed at the ADC base.
  • Model (hw/adc/esp32c3_adc.c): INT_RAW always returns done bits 31|30; ONETIME_SAMPLE (+0x20) bits [28:25] select the channel; 1_DATA_STATUS (+0x2c) / 2_DATA_STATUS (+0x30) return the selected channel's 12-bit value. Per-channel values are QOM props adci[*], default 0xfff.
  • adc_oneshot_ll_get_raw_result reads adc1_data & 0xfff.

SPI2 (display + SD) — DR_REG_SPI2_BASE = 0x60024000

  • Completely different register map from SPI1 (hw/ssi/esp32c3_spi.c is flash-specific — do NOT reuse it). GP-SPI2 layout: CMD 0x00 (USR = bit24, UPDATE = bit23), MS_DLEN 0x1c (bits[17:0] = bitlen-1), data buffer W0..W15 at 0x98..0xd4.
  • Model (hw/ssi/esp32c3_spi2.c): on CMD write with USR set, clock (MS_DLEN/8)+1 bytes out of W0.. onto an SSIBus via ssi_transfer, writing RX back into the same words. DMA path is intentionally unimplemented — the firmware's CPU-buffer path is enough; add DMA only if a future firmware needs it (it'll show up as SPI2 traffic that the CPU-buffer model can't explain).

GPIO — DR_REG_GPIO_BASE = 0x60004000

  • The stock model only returned the strap register. We added real OUT 0x04 / OUT_W1TS 0x08 / OUT_W1TC 0x0c / ENABLE 0x20 (+W1TS/W1TC) / IN 0x3c, with 32 named input + output qemu_irq lines (ESP32_GPIO_INPUT/ESP32_GPIO_OUTPUT).
  • CRITICAL SUBTLETY: an output line must read HIGH when the pin is configured as input (output_enable bit clear), not follow the (zero) OUT latch. Real boards have pull-ups; the X3 display CS is pulled high while SD is being probed. Without this, CS floated LOW during SD init and the panel greedily consumed the SD command bytes (FF … 40 00 … 95) as garbage commands. This one bit cost real debugging time.

SD-over-SPI — QEMU's ssi-sd + sd-card-spi

  • These exist upstream; we attach them to SPI2 at CS12. But the 9.2 fork's ssi-sd.c has a bug: CMD58 (read OCR) hardcodes the R1 byte to 0x01 (idle) even after ACMD41 reported ready → SdFat rejects the card → firmware shows "SD card error". Fix: track idle state, clear it when ACMD41 returns 0, return it for CMD58. (Upstream qemu master rewrote this path entirely; we did the minimal equivalent + a vmstate version bump to 8.)

5. The e-ink panel model (hw/display/xteink_x3_eink.c)

  • X3 uses a UC8253 controller, 792×528, 99 bytes/row, 52 272 bytes/plane.
  • Protocol (from chip/eink-x3.chip.c, the old Wokwi model — keep it as the reference): DC low = command, DC high = data. 0x10=DTM1 (old plane), 0x13=DTM2 (new plane), 0x12=refresh (render + pulse BUSY), 0x04/0x02= power on/off (pulse BUSY). Bit 1 = white, 0 = black. Rows arrive bottom-to-top (wire row i → framebuffer row H-1-i).
  • BUSY is a GPIO output from the panel back into GPIO6 (active low; we pulse it low for 5 ms of virtual time via a QEMUTimer, then high).
  • Rendered to a QEMU graphical console (not a custom file callback) so the WASM front-end reuses QEMU's display surface. Capture headless with QMP screendump → PPM.
  • Orientation: the controller framebuffer (792×528 landscape) is rotated 90° clockwise to the physical portrait panel (528×792). We bake that into the render loop (screen_x = H-1-y, screen_y = x) so the console is already physical-orientation. Verified by eye: text reads upright at 528×792.
  • SSI gotcha: a TYPE_SSI_PERIPHERAL subclass must set ssi->realize even if empty — the base class calls it unconditionally and a null pointer segfaults at machine init. Also set cs_polarity = SSI_CS_LOW and user_creatable = false.

6. Machine wiring (xteink_x3_machine_init)

xteink-x3 is a subclass of the esp32c3 machine that, after the base init:

  • creates the panel on spi2.bus, wires GPIO out 21→CS, 4→DC, 5→RST, and panel BUSY→GPIO in 6;
  • creates ssi-sd at CS index 1 on spi2.bus, wires GPIO out 12→its CS, and a sd-card-spi backed by drive_get(IF_SD, 0, 0) (QEMU block layer — the SD "interface" you asked for; local -drive now, browser blockdev later);
  • defaults both CS lines high at init (qemu_set_irq(cs, 1)) so nothing is selected before the firmware drives GPIO.

Pin facts (X3, from freeink-sdk BoardConfig.h XTEINK_X3): display SCLK8/MOSI10/CS21/DC4/RST5/BUSY6; SD shares SCLK8/MOSI10, MISO7, CS12; button ADC ladders on GPIO1 & GPIO2; power button GPIO3; UC8253 @ 16 MHz.

7. X3 vs X4 detection — the thing that wasted time

  • Both X3 and X4 ship in ONE ESP32-C3 binary. Which panel/battery profile is used is decided at runtime by an I²C fingerprint on SDA20/SCL0: presence of BQ27220 gauge (0x55), DS3231 RTC (0x68), QMI8658 IMU (0x6B). ≥2 of 3 on two passes ⇒ X3; 0 ⇒ X4. Result is cached in NVS (cphw/dev_det) and can be overridden (cphw/dev_ovr: 1=X4, 2=X3).
  • QEMU models no I²C, so detection currently returns X4 by default. The crosspoint-reader-lua firmware never calls setDisplayX3() at all → it is X4-only in emulation and drives SSD1677 commands (0x18/0x0c/0x44/0x45…), which our UC8253 panel ignores (→ blank screen). Don't chase that as a bug; it's the wrong firmware for an X3 machine.
  • To test the X3 display this session I built the canonical firmware with a one-line temporary hack: detectDeviceTypeWithFingerprint() in /tmp/crosspoint-reader-main/lib/hal/HalGPIO.cpp returns X3 at the top. THIS EDIT IS IN /tmp AND WILL BE LOST ON REBOOT. The merged test image is /tmp/crosspoint-main-x3-flash.bin (also ephemeral).
  • Production plan (no custom firmware): the xteink-x3 machine should expose the three X3 I²C fingerprint chips (needs an esp32c3 I²C controller model + trivial BQ27220/DS3231/QMI8658 stubs that return plausible WHO_AM_I/values), OR the emulator seeds the NVS dev_ovr key. Either makes stock firmware auto-select X3. The xteink-x4 machine simply omits the fingerprint. This is how launch-time X3/X4 selection stays firmware-agnostic.

8. Reproducibility model

  • Durable = the patch (qemu/patches/0001-xteink-x3-machine.patch), verified to git apply cleanly onto a pristine checkout of the pinned tag. When you edit /tmp/qsrc, regenerate it and re-verify: 
    cd <qemu-src> && git add -N <new files> && git diff --binary > <repo>/qemu/patches/0001-xteink-x3-machine.patch
git apply --reverse --check <repo>/qemu/patches/0001-xteink-x3-machine.patch   # must succeed
    (Reverse-check on the dirty tree proves patch == pristine→current diff, which is equivalent to forward-applying to pristine.)
  • Firmware/SD/flash images and /tmp/qsrc are all disposable; make qemu + make firmware sdimage rebuild them.

9. Debugging workflow cheat-sheet

  • Headless run + capture (canonical loop this session): 
    qemu-system-riscv32 -machine xteink-x3 -L <src>/pc-bios -display none -serial null \
  -qmp unix:/tmp/q.sock,server=on,wait=off \
  -drive file=flash.bin,if=mtd,format=raw -drive file=sd.img,if=sd,format=raw &
sleep 200   # this CPU is slow — give boot real time
scripts/qmp-screendump.py /tmp/q.sock out.ppm
  • Inspect the screen: convert with nix shell nixpkgs#imagemagick -c magick out.ppm out.png; the black-pixel count is a cheap "did the screen change" signal (home ≈ 15914, browse ≈ 11962, boot-white = 962).
  • Inject a button (proper edge = set then clear; hold-from-boot gives no press edge): scripts/qmp-qom.py /tmp/q.sock set /machine/adc "adci[1]" 2760 then … 4095 to release. Ladder-1 midpoints: Back 3512 / Confirm 2694 / Left 1493 / Right ~5; ladder-2: Up 2242 / Down ~5. >3900 = no button.
  • SD protocol trace: flip //#define DEBUG_SSI_SD 1 in hw/sd/ssi-sd.c.
  • Panel trace: add an info_report in xteink_x3_eink_transfer (bounded with a static counter — it's hot).
  • Symbolize a panic PC: llvm-addr2line -f -C -e firmware.elf 0xADDR (ignore DWARF "premature terminator" warnings).
  • Flash images must be 2/4/8/16 MB. The partition table spans 16 MB (app is <4 MB), so always pad to 16 MB (truncate -s 16M); a mismatched size errors with "only 2, 4, 8, and 16MB images are supported".

10. Next steps (in priority order)

  1. X3 fingerprint or NVS seeding (§7) so stock firmware auto-selects X3 with no source edit. Smallest path to a genuinely firmware-agnostic X3 machine. Needs a minimal esp32c3 I²C controller + 3 tiny I²C target stubs, OR NVS pre-seed. This is the last thing standing between "works with a hacked build" and "works with any downloaded binary."
  2. xteink-x4 machine: SSD1677 panel (different command set — 0x18 set-RAM, 0x44/0x45 window, 0x4e/0x4f cursor, 0x24 write, 0x22/0x20 update; BUSY active-HIGH), X4 battery via ADC (not I²C gauge), 792×528? confirm X4 geometry from BoardConfig.h XTEINK_X4. Sibling machine + a second panel device; reuse ADC/SPI2/GPIO/SD unchanged.
  3. Compile to WASM: combine this patched fork with ktock/qemu-wasm's WASM TCG/TCI backend + Emscripten. Expect toolchain friction; the device models themselves are portable C and shouldn't need changes. De-risk by first building an unpatched esp32c3 target under qemu-wasm, then apply our patch.
  4. Static browser front-end: <canvas> fed from the display surface; DOM buttons → the ADC/GPIO QOM injection (same calls as qmp-qom.py); file input → emulated flash; preloaded/uploaded SD image → block backend; X3/X4 chosen before boot. Must run fully from static assets (airgapped).

11. Key paths

What Where
Emulator repo /home/evanreichard/Development/git/personal/xteink-web-emulator
The patch qemu/patches/0001-xteink-x3-machine.patch
QEMU source (ephemeral) _scratch/qemu-src (via make qemu); was /tmp/qsrc this session
Canonical firmware (X3 hack in /tmp, ephemeral) /tmp/crosspoint-reader-main (lib/hal/HalGPIO.cpp forced X3)
Lua firmware (X4-only in emu) /home/evanreichard/Development/git/personal/crosspoint-reader-lua
Upstream firmware refs crosspoint-reader/crosspoint-reader (canonical, cleaner) + its freeink-sdk submodule
espressif/qemu tag esp-develop-9.2.2-20260417
qemu-wasm github.com/ktock/qemu-wasm
Reference in New Issue View Git Blame Copy Permalink