diff --git a/ISSUES.md b/ISSUES.md index 30116b8..4e24dd1 100644 --- a/ISSUES.md +++ b/ISSUES.md @@ -1,28 +1,12 @@ # Known Issues -## SD reads can corrupt guest memory +## Resolved: directory-backed SD reads corrupted WASM state -Opening `/Test/example.txt` can panic consistently: +Opening `/Test/example.txt` through QEMU `vvfat` could corrupt coroutine/guest state. The browser now builds a standards-compliant 64 MiB FAT32 image from OPFS and mounts it as a raw block device, bypassing the failing host-file mapping path. The official CrossPoint 1.4.1 firmware can traverse `Test/example.txt` without a WASM trap or guest panic. -```text -Invalid read at addr 0x253D7474, size 4, region '(null)' -Guru Meditation Error: Core 0 panic'ed (Load access fault) -``` +## Resolved: SD card remained in `receivingdata` -The fault occurs in `spiTransferByteNL()` after the `SdSpiCard` SPI pointer at `0x3fc94b40` becomes `0x253D7474`, which resembles file data. The crash is deterministic with the directory-backed `vvfat` card. A raw FAT32 diagnostic image fails to mount, so it has not yet provided a clean backend comparison. - -The crash dump also shows `SdSpiCard::readData()` reaching its CRC reads while its destination pointer and end pointer differ. Keeping conditional translation blocks on TCI did not fix the corruption. The next diagnostic build keeps guest-store TBs on TCI and watches writes to `0x3fc94b40`–`0x3fc94b57`. - -## SD card can remain in `receivingdata` - -The SD model can become stuck after an incomplete `CMD24` write: - -```text -SPI: CMD17 in a wrong state: receivingdata (spec v3.01) -SPI: CMD24 in a wrong state: receivingdata (spec v3.01) -``` - -Subsequent reads and writes then fail. This may be related to the guest-memory corruption, but remains tracked separately until the first incomplete transfer is identified. +`ssi-sd` consumed a write-data token while leaving response mode, then parsed payload bytes as commands. The adapter now enters token mode immediately after the final R1 byte. SPI CMD13 also uses the upstream-correct `sd_r1` internal response so status polling no longer loops on `Unexpected response to cmd 13`. ## Reader page can render completely black diff --git a/Makefile b/Makefile index 813244e..1369e44 100644 --- a/Makefile +++ b/Makefile @@ -11,7 +11,7 @@ QEMU_WASM_SRC ?= third_party/qemu WASM_OUT ?= dist/wasm VARIANT ?= x3 -.PHONY: firmware sdimage qemu qemu-wasm web web-test run run-upstream chip clean +.PHONY: firmware sdimage qemu qemu-wasm web web-test test browser-test run run-upstream chip clean firmware: esptool --chip esp32c3 merge-bin -o flash.bin \ @@ -39,6 +39,12 @@ web: qemu-wasm web-test: node scripts/test-web.mjs +test: + $(MAKE) -C tests test + +browser-test: + $(MAKE) -C tests browser + # Select X4 with `make run VARIANT=x4`. Add `-display none` for headless. run: qemu @sd=sd.img; tmpdir=; \ diff --git a/README.md b/README.md index 57c99b2..74600c2 100644 --- a/README.md +++ b/README.md @@ -60,12 +60,13 @@ map to the device's ADC button ladders and active-low Power GPIO. `make web` serves the required COOP/COEP headers for Emscripten pthreads; a generic static server without those headers will not work. -Real firmware boots in WASM and drives the X3 panel (528×792). The browser SD -uses QEMU's directory-backed FAT provider: the guest sees a writable, ephemeral -32 GiB FAT32 card while memory use scales with actual files instead of logical -capacity. It starts with `Test/example.txt`; inspect it in the Files tab. The -Debug tab reports committed WASM heap, visible MEMFS data, SD file bytes, and -browser JS heap where supported. X4 remains a white display stub. +Real firmware boots in WASM and drives the X3 panel (528×792). Browser SD files +are persisted in OPFS and edited in the Files tab. At boot, the browser builds a +64 MiB raw FAT32 image with nested-directory and VFAT long-name support; QEMU +mounts that image directly, avoiding `vvfat`. Choose **Apply & reboot** to expose +edits to the guest. Guest writes modify only the ephemeral image. The Debug tab +reports committed WASM heap, visible MEMFS data, and browser JS heap where +supported. X4 remains a white display stub. ## Contents @@ -81,8 +82,12 @@ browser JS heap where supported. X4 remains a white display stub. - `flake.nix` — the espressif-qemu *release* binary as the fast generic baseline (`nix run .#run-upstream -- flash.bin`), plus a devShell with QEMU's full build environment. `Makefile` — `qemu`, `firmware`, `sdimage`, `run`, `chip`. -- `scripts/mksd.sh` — build the FAT32 SD backing image (local-disk impl of the - SD block interface). +- `web/fat32.js` and `web/sdstore.js` — dependency-free FAT32 image generation + and persistent OPFS file storage. +- `scripts/mksd.sh` — build a FAT32 SD backing image for native QEMU. +- `tests/` — `make test` runs JS/FAT32 checks; `make browser-test` boots the + official CrossPoint 1.4.1 firmware and navigates to `Test/example.txt` in + headless Firefox. - `chip/eink-x3.chip.c` — the reverse-engineered X3 e-ink protocol, first written as a Wokwi chip and now the reference for the QEMU display device. `make -C chip test` runs its host self-check. diff --git a/ROADMAP.md b/ROADMAP.md index 0205048..8fdfdc8 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -58,7 +58,7 @@ reuses unchanged — no bespoke abstraction layer: | Peripheral | Native seam | Browser binding | |------------|-------------|---------------------------| | e-ink panel | X3 UC8253 (528×792) or blank X4 SSD1677 stub (480×800) → QEMU graphical console | `` from the same display surface | -| SD card | `ssi-sd` + `sd-card-spi` backed by QEMU block layer (`-drive if=sd`) | QEMU `vvfat` over a JS-created MEMFS directory; future persistence can use OPFS | +| SD card | `ssi-sd` + `sd-card-spi` backed by QEMU block layer (`-drive if=sd`) | OPFS files → browser-built 64 MiB FAT32 image → raw MEMFS block device | | Buttons + battery | ADC QOM props `adci[1]`/`adci[2]`; GPIO QOM `input-level[3]` for Power | DOM buttons → `qom-set` equivalent | | Firmware image | flash via `-drive if=mtd` | uploaded `.bin` written to emulated flash | @@ -85,9 +85,7 @@ reuses unchanged — no bespoke abstraction layer: ## Remaining phases -1. **Browser SD editing/persistence** — add file controls and optionally OPFS; - the current 32 GiB logical card is writable and ephemeral. -2. **Full X4 panel** — implement SSD1677 RAM/window/update commands when visible +1. **Full X4 panel** — implement SSD1677 RAM/window/update commands when visible X4 rendering is needed; the selection, dimensions, wiring, and stub exist. ## Repo layout diff --git a/scripts/test-web.mjs b/scripts/test-web.mjs index 19ebe75..8e95297 100644 --- a/scripts/test-web.mjs +++ b/scripts/test-web.mjs @@ -1,5 +1,6 @@ import assert from 'node:assert/strict'; import { BUTTONS, DEFAULT_SD_FILES, EXAMPLE_SD_TEXT, formatBytes, setButton, mergeFirmware } from '../web/app.js'; +import { buildFat32Image, FAT32_IMAGE_SIZE } from '../web/fat32.js'; // Merge - app image is placed at 0x10000 in a 0xFF-erased 16 MB flash with bootloader and partitions. const bootloader = new Uint8Array([1, 2, 3]); @@ -19,6 +20,36 @@ assert.equal(EXAMPLE_SD_TEXT.split('Lorem ipsum').length - 1, 10); assert.equal(formatBytes(512), '512 B'); assert.equal(formatBytes(64 * 1024 * 1024), '64.0 MiB'); +const fatPayload = new TextEncoder().encode('A'.repeat(700)); +const sdImage = buildFat32Image([ + { path: 'Test/example.txt', bytes: fatPayload }, + { path: 'Books/A long filename.txt', bytes: new TextEncoder().encode('book') }, +]); +const sdView = new DataView(sdImage.buffer); +assert.equal(sdImage.length, FAT32_IMAGE_SIZE); +assert.equal(sdView.getUint16(11, true), 512); +assert.equal(sdImage[13], 1); +assert.equal(sdView.getUint32(32, true), FAT32_IMAGE_SIZE / 512); +assert.equal(sdView.getUint32(44, true), 2); +assert.deepEqual([...sdImage.slice(510, 512)], [0x55, 0xaa]); +const fatSectors = sdView.getUint32(36, true); +const fatOffset = 32 * 512; +const dataOffset = (32 + 2 * fatSectors) * 512; +const root = sdImage.subarray(dataOffset, dataOffset + 512); +const testEntryOffset = Array.from({ length: 16 }, (_, i) => i * 32) + .find(offset => root[offset + 11] !== 0x0f && new TextDecoder().decode(root.subarray(offset, offset + 8)).trim() === 'TEST'); +assert.notEqual(testEntryOffset, undefined); +const testCluster = new DataView(root.buffer, root.byteOffset + testEntryOffset, 32).getUint16(26, true); +const testDirectory = sdImage.subarray(dataOffset + (testCluster - 2) * 512, dataOffset + (testCluster - 1) * 512); +const fileOffset = Array.from({ length: 16 }, (_, i) => i * 32) + .find(offset => new TextDecoder().decode(testDirectory.subarray(offset, offset + 8)).trim() === 'EXAMPLE'); +const fileEntry = new DataView(testDirectory.buffer, testDirectory.byteOffset + fileOffset, 32); +const fileCluster = fileEntry.getUint16(26, true); +assert.equal(fileEntry.getUint32(28, true), fatPayload.length); +assert.equal(sdView.getUint32(fatOffset + fileCluster * 4, true), fileCluster + 1); +assert.equal(sdView.getUint32(fatOffset + (fileCluster + 1) * 4, true), 0x0fffffff); +assert.deepEqual(sdImage.slice(dataOffset + (fileCluster - 2) * 512, dataOffset + (fileCluster - 2) * 512 + fatPayload.length), fatPayload); + const calls = []; const module = { _xteink_wasm_set_adc: (...args) => calls.push(['adc', ...args]), diff --git a/tests/Makefile b/tests/Makefile new file mode 100644 index 0000000..88ff097 --- /dev/null +++ b/tests/Makefile @@ -0,0 +1,17 @@ +.PHONY: test unit fat32 browser + +test: unit fat32 + +unit: + cd .. && node scripts/test-web.mjs + +fat32: + node generate-fat32-fixture.mjs + mdir -i ../_scratch/test-fat32.img :: >/dev/null + mdir -i ../_scratch/test-fat32.img ::/Test >/dev/null + mdir -i ../_scratch/test-fat32.img ::/Books >/dev/null + test "$$(mtype -i ../_scratch/test-fat32.img ::/Test/example.txt)" = "hello from OPFS" + test "$$(mtype -i ../_scratch/test-fat32.img '::/Books/A long book filename.txt')" = "chapter one" + +browser: + ./run-browser-raw-sd.sh diff --git a/tests/__pycache__/browser_raw_sd.cpython-314.pyc b/tests/__pycache__/browser_raw_sd.cpython-314.pyc new file mode 100644 index 0000000..0eb13f0 Binary files /dev/null and b/tests/__pycache__/browser_raw_sd.cpython-314.pyc differ diff --git a/tests/browser_raw_sd.py b/tests/browser_raw_sd.py new file mode 100644 index 0000000..669d351 --- /dev/null +++ b/tests/browser_raw_sd.py @@ -0,0 +1,77 @@ +from pathlib import Path +import time +from selenium import webdriver +from selenium.webdriver.common.action_chains import ActionChains +from selenium.webdriver.common.by import By +from selenium.webdriver.firefox.options import Options +from selenium.webdriver.support.ui import WebDriverWait + +ROOT = Path(__file__).resolve().parent.parent + +options = Options() +options.add_argument('-headless') +driver = webdriver.Firefox(options=options) +driver.set_page_load_timeout(60) +try: + driver.get('http://127.0.0.1:8180/web/?firmware=/_scratch/crosspoint-reader-1.4.1-release-firmware.bin&variant=x3') + confirm = driver.find_element(By.CSS_SELECTOR, '[data-button="confirm"]') + WebDriverWait(driver, 8 * 60).until(lambda _: confirm.get_attribute('disabled') is None) + + indicator = driver.find_element(By.ID, 'frame-generation') + + def generation(): + return int(indicator.get_attribute('value') or '-1') + + def wait_frame(previous, timeout=180): + WebDriverWait(driver, timeout).until(lambda _: generation() > previous) + + def wait_stable(quiet=5, timeout=180): + deadline = time.time() + timeout + last = generation() + stable_since = time.time() + while time.time() < deadline: + time.sleep(0.25) + current = generation() + if current != last: + last = current + stable_since = time.time() + elif time.time() - stable_since >= quiet: + return current + raise RuntimeError('display did not settle') + + power = driver.find_element(By.CSS_SELECTOR, '[data-button="power"]') + before = generation() + ActionChains(driver).click_and_hold(power).perform() + wait_frame(before) + ActionChains(driver).release(power).perform() + wait_stable() + + def press(button): + driver.execute_script('arguments[0].scrollIntoView({block: "center"})', button) + before = wait_stable() + ActionChains(driver).click_and_hold(button).perform() + time.sleep(0.3) + ActionChains(driver).release(button).perform() + wait_frame(before) + wait_stable() + + press(confirm) # Browse Files + press(confirm) # Test + press(confirm) # example.txt + time.sleep(60) + + status = driver.find_element(By.ID, 'status').text + log = driver.find_element(By.ID, 'log').text + (ROOT / '_scratch/browser-raw-sd.log').write_text(log) + driver.save_screenshot(str(ROOT / '_scratch/browser-raw-sd.png')) + + bad = ( + 'RuntimeError', 'Guru Meditation', "panic'ed", 'Invalid SD card size', + 'FS error', 'SPI:', 'SD:', 'Unexpected response to cmd', + ) + matched = [marker for marker in bad if marker in log or marker in status] + if matched: + raise RuntimeError(f'emulator failure detected: {matched}') + print(f'browser raw-SD test passed at frame generation {generation()}') +finally: + driver.quit() diff --git a/tests/generate-fat32-fixture.mjs b/tests/generate-fat32-fixture.mjs new file mode 100644 index 0000000..4751124 --- /dev/null +++ b/tests/generate-fat32-fixture.mjs @@ -0,0 +1,10 @@ +import { mkdir, writeFile } from 'node:fs/promises'; +import { buildFat32Image } from '../web/fat32.js'; + +const encoder = new TextEncoder(); +await mkdir('../_scratch', { recursive: true }); +await writeFile('../_scratch/test-fat32.img', buildFat32Image([ + { path: 'Test/example.txt', bytes: encoder.encode('hello from OPFS\n') }, + { path: 'Books/A long book filename.txt', bytes: encoder.encode('chapter one\n') }, + { path: 'empty.bin', bytes: new Uint8Array() }, +])); diff --git a/tests/run-browser-raw-sd.sh b/tests/run-browser-raw-sd.sh new file mode 100755 index 0000000..afb4257 --- /dev/null +++ b/tests/run-browser-raw-sd.sh @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT=$(cd "$(dirname "$0")/.." && pwd) +cd "$ROOT" +mkdir -p _scratch + +FIRMWARE=_scratch/crosspoint-reader-1.4.1-release-firmware.bin +EXPECTED=c3735b487378650f7fb4be5b286db1a9796768ff02bc8831dc4ba0a3771659a1 +if [[ ! -f $FIRMWARE ]] || [[ $(sha256sum "$FIRMWARE" | cut -d' ' -f1) != "$EXPECTED" ]]; then + curl -fL https://github.com/crosspoint-reader/crosspoint-reader/releases/download/1.4.1/firmware.bin -o "$FIRMWARE" +fi + +python3 scripts/serve-web.py --host 127.0.0.1 --port 8180 >_scratch/browser-raw-sd-server.log 2>&1 & +server=$! +trap 'kill "$server" 2>/dev/null || true' EXIT +sleep 2 + +nix shell --impure --expr \ + 'with import {}; buildEnv { name = "xteink-selenium"; paths = [ (python313.withPackages (p: [p.selenium])) geckodriver firefox ]; }' \ + -c python3 tests/browser_raw_sd.py diff --git a/third_party/qemu b/third_party/qemu index 13b9231..3623a36 160000 --- a/third_party/qemu +++ b/third_party/qemu @@ -1 +1 @@ -Subproject commit 13b9231f6eea9a88fc83ef7395ae0909ba6f67b1 +Subproject commit 3623a36693d4e29e116f2fb8461fee115180b1dd diff --git a/web/app.js b/web/app.js index bd9517e..cfc2491 100644 --- a/web/app.js +++ b/web/app.js @@ -1,3 +1,9 @@ +import { + listSdFiles, readSdFile, writeSdFile, deleteSdFile, readAllSdFiles, + seedDefaultsIfEmpty, saveBootState, loadBootState, +} from './sdstore.js'; +import { buildFat32Image } from './fat32.js'; + const FLASH_SIZE = 16 * 1024 * 1024; const BOOTLOADER_OFFSET = 0x0; const PARTITIONS_OFFSET = 0x8000; @@ -42,16 +48,6 @@ const LOREM_IPSUM = 'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Se export const EXAMPLE_SD_TEXT = Array(10).fill(LOREM_IPSUM).join('\n\n'); export const DEFAULT_SD_FILES = new Map([['Test/example.txt', EXAMPLE_SD_TEXT]]); -function mountEphemeralSdCard(fs) { - for (const [path, contents] of DEFAULT_SD_FILES) { - const parts = path.split('/'); - const filename = parts.pop(); - const directory = `/sdcard/${parts.join('/')}`; - fs.mkdirTree(directory); - fs.writeFile(`${directory}/${filename}`, contents); - } -} - // A full 16 MB image is used as-is; anything else is treated as a CrossPoint app image and merged with the bundled bootloader and partition table. async function prepareFirmware(bytes, assetRoot) { if (bytes.length === FLASH_SIZE) { @@ -113,6 +109,9 @@ function startDisplayLoop(module, canvas, setStatus, onReady) { const nextGeneration = module._xteink_wasm_frame_generation(); if (nextGeneration !== generation && drawFrame(module, canvas)) { generation = nextGeneration; + const indicator = document.querySelector('#frame-generation'); + indicator.value = String(generation); + document.dispatchEvent(new CustomEvent('xteink-frame', { detail: { generation } })); setStatus('Running.'); if (!ready) { ready = true; @@ -188,46 +187,46 @@ export function formatBytes(bytes) { return `${value.toFixed(value >= 10 ? 1 : 2)} ${units[unit]}`; } -function sdEntries(module = activeModule) { - if (!module) { - return [ - { path: 'Test/', directory: true }, - ...[...DEFAULT_SD_FILES].map(([path, contents]) => ({ - path, - size: new TextEncoder().encode(contents).length, - read: () => contents, - })), - ]; - } +const TEXT_EXTENSIONS = new Set(['txt', 'md', 'json', 'cfg', 'csv', 'log', 'html', 'xml', 'ini', 'yml', 'yaml']); +let selectedSdPath = null; - const entries = []; - const visit = (directory, relative = '') => { - for (const name of module.FS.readdir(directory).filter(name => name !== '.' && name !== '..').sort()) { - const absolute = `${directory}/${name}`; - const path = relative ? `${relative}/${name}` : name; - const stat = module.FS.stat(absolute); - if (module.FS.isDir(stat.mode)) { - entries.push({ path: `${path}/`, directory: true }); - visit(absolute, path); - } else { - entries.push({ - path, - size: stat.size, - read: () => module.FS.readFile(absolute, { encoding: 'utf8' }), - }); - } - } - }; - visit('/sdcard'); - return entries; +function isTextPath(path) { + return TEXT_EXTENSIONS.has(path.split('.').pop().toLowerCase()); } -function renderFiles() { +async function selectSdFile(path) { + selectedSdPath = path; + const editor = document.querySelector('#file-editor'); + const save = document.querySelector('#file-save'); + const del = document.querySelector('#file-delete'); + del.hidden = false; + try { + const bytes = await readSdFile(path); + if (isTextPath(path)) { + editor.value = new TextDecoder().decode(bytes); + editor.readOnly = false; + save.hidden = false; + } else { + editor.value = `(binary file — ${formatBytes(bytes.length)}; not editable here)`; + editor.readOnly = true; + save.hidden = true; + } + } catch (error) { + editor.value = `Could not read ${path}: ${error.message || error}`; + editor.readOnly = true; + save.hidden = true; + } + for (const button of document.querySelectorAll('#file-list button')) { + button.classList.toggle('selected', button.dataset.path === path); + } +} + +async function renderFiles() { const list = document.querySelector('#file-list'); - const preview = document.querySelector('#file-preview'); - const entries = sdEntries(); + const editor = document.querySelector('#file-editor'); + const entries = await listSdFiles(); list.replaceChildren(); - let firstFile = null; + let firstPath = null; for (const entry of entries) { if (entry.directory) { @@ -239,26 +238,27 @@ function renderFiles() { } const button = document.createElement('button'); button.type = 'button'; + button.dataset.path = entry.path; const path = document.createElement('span'); path.textContent = entry.path; const size = document.createElement('small'); size.textContent = formatBytes(entry.size); button.append(path, size); - button.addEventListener('click', () => { - try { - preview.textContent = entry.read(); - } catch (error) { - preview.textContent = `Could not read ${entry.path}: ${error.message || error}`; - } - }); + button.addEventListener('click', () => selectSdFile(entry.path)); list.append(button); - firstFile ||= button; + firstPath ||= entry.path; } - if (firstFile) { - firstFile.click(); + const stillPresent = entries.some(entry => entry.path === selectedSdPath); + const target = stillPresent ? selectedSdPath : firstPath; + if (target) { + await selectSdFile(target); } else { - preview.textContent = 'The SD card is empty.'; + selectedSdPath = null; + editor.value = 'The SD card is empty.'; + editor.readOnly = true; + document.querySelector('#file-save').hidden = true; + document.querySelector('#file-delete').hidden = true; } } @@ -283,13 +283,12 @@ function renderDebug() { if (!activeModule) { return; } - const visible = ['/flash.bin', '/bios', '/sdcard', '/var/tmp'] + const visible = ['/flash.bin', '/bios', '/sd.img', '/var/tmp'] .map(path => pathBytes(activeModule.FS, path)) .reduce((total, item) => ({ bytes: total.bytes + item.bytes, files: total.files + item.files }), { bytes: 0, files: 0 }); - const sd = pathBytes(activeModule.FS, '/sdcard'); document.querySelector('#debug-wasm-heap').textContent = formatBytes(activeModule.HEAPU8.byteLength); document.querySelector('#debug-memfs').textContent = `${formatBytes(visible.bytes)} in ${visible.files} files`; - document.querySelector('#debug-sd-files').textContent = `${formatBytes(sd.bytes)} in ${sd.files} files`; + document.querySelector('#debug-sd-files').textContent = '64.0 MiB raw image (OPFS source)'; } function initializeInspector() { @@ -307,11 +306,47 @@ function initializeInspector() { } }); } - renderFiles(); + wireFileControls(); + seedDefaultsIfEmpty(DEFAULT_SD_FILES).then(renderFiles); renderDebug(); setInterval(renderDebug, 1000); } +function wireFileControls() { + document.querySelector('#file-add').addEventListener('change', async event => { + for (const file of event.target.files) { + await writeSdFile(file.name, new Uint8Array(await file.arrayBuffer())); + } + event.target.value = ''; + await renderFiles(); + }); + document.querySelector('#file-new').addEventListener('click', async () => { + const name = prompt('New file path (e.g. Notes/todo.txt):'); + if (!name) { + return; + } + await writeSdFile(name, ''); + selectedSdPath = name; + await renderFiles(); + }); + document.querySelector('#file-save').addEventListener('click', async () => { + if (!selectedSdPath) { + return; + } + await writeSdFile(selectedSdPath, document.querySelector('#file-editor').value); + await renderFiles(); + }); + document.querySelector('#file-delete').addEventListener('click', async () => { + if (!selectedSdPath || !confirm(`Delete ${selectedSdPath}?`)) { + return; + } + await deleteSdFile(selectedSdPath); + selectedSdPath = null; + await renderFiles(); + }); + document.querySelector('#file-reboot').addEventListener('click', () => location.reload()); +} + function logLine(message) { logBuffer.push(message); if (logBuffer.length > LOG_MAX_LINES) { @@ -341,7 +376,11 @@ async function boot(file, variant, setStatus) { prepareFirmware(new Uint8Array(await file.arrayBuffer()), import.meta.url), fetchBin(new URL('esp32c3-rom.bin', artifactRoot)), ]); - const sdMessage = 'Using a 32 GiB ephemeral directory-backed SD card; writes will be discarded when the emulator restarts.'; + await seedDefaultsIfEmpty(DEFAULT_SD_FILES); + const sdFiles = await readAllSdFiles(); + setStatus(`Building SD card image from ${sdFiles.length} browser file(s)…`); + const sdImage = buildFat32Image(sdFiles); + const sdMessage = `Using a 64 MiB FAT32 image built from ${sdFiles.length} browser file(s); guest writes are discarded on reboot.`; console.log(sdMessage); logLine(sdMessage); const { default: createQemu } = await import(qemuUrl); @@ -360,7 +399,7 @@ async function boot(file, variant, setStatus) { '-nic', 'none', '-d', qemuLogFlags, '-drive', 'file=/flash.bin,if=mtd,format=raw', - '-drive', 'file=fat:32:rw:/sdcard,if=sd,format=raw', + '-drive', 'file=/sd.img,if=sd,format=raw', ], locateFile: path => new URL(path, artifactRoot).href, mainScriptUrlOrBlob: qemuUrl.href, @@ -373,7 +412,7 @@ async function boot(file, variant, setStatus) { options.FS.mkdirTree('/var/tmp'); options.FS.writeFile('/bios/esp32c3-rom.bin', rom); options.FS.writeFile('/flash.bin', firmware); - mountEphemeralSdCard(options.FS); + options.FS.writeFile('/sd.img', sdImage); }]; setStatus('Starting emulator…'); @@ -428,13 +467,14 @@ function initialize() { } }; - form.addEventListener('submit', event => { + form.addEventListener('submit', async event => { event.preventDefault(); const file = firmwareInput.files[0]; if (!file) { setStatus('Choose a firmware image first.', true); return; } + await saveBootState(file, variantInput.value); start(file); }); @@ -447,7 +487,21 @@ function initialize() { } return response.blob(); }) - .then(blob => start(new File([blob], 'firmware.bin'), params.get('variant') || 'x3')) + .then(async blob => { + const variant = params.get('variant') || 'x3'; + const firmware = new File([blob], 'firmware.bin'); + await saveBootState(firmware, variant); + await start(firmware, variant); + }) + .catch(error => setStatus(error.message || String(error), true)); + } else { + loadBootState() + .then(saved => { + if (saved) { + variantInput.value = saved.variant; + start(saved.firmware, saved.variant); + } + }) .catch(error => setStatus(error.message || String(error), true)); } } diff --git a/web/fat32.js b/web/fat32.js new file mode 100644 index 0000000..a74f5a5 --- /dev/null +++ b/web/fat32.js @@ -0,0 +1,293 @@ +export const FAT32_IMAGE_SIZE = 64 * 1024 * 1024; + +const SECTOR_SIZE = 512; +const RESERVED_SECTORS = 32; +const FAT_COUNT = 2; +const ROOT_CLUSTER = 2; +const FAT_EOC = 0x0fffffff; + +function setAscii(bytes, offset, value, length) { + for (let i = 0; i < length; i++) { + bytes[offset + i] = i < value.length ? value.charCodeAt(i) : 0x20; + } +} + +function fatGeometry(imageSize) { + const totalSectors = imageSize / SECTOR_SIZE; + let fatSectors = 1; + while (true) { + const clusters = totalSectors - RESERVED_SECTORS - FAT_COUNT * fatSectors; + const required = Math.ceil((clusters + 2) * 4 / SECTOR_SIZE); + if (required <= fatSectors) { + return { totalSectors, fatSectors, clusters }; + } + fatSectors = required; + } +} + +function makeNode(name, directory, parent = null) { + return { name, directory, parent, children: new Map(), bytes: null, shortName: null, firstCluster: 0, clusters: [] }; +} + +function normalizeFiles(files) { + const root = makeNode('', true); + for (const { path, bytes } of files) { + const parts = path.replaceAll('\\', '/').split('/').filter(Boolean); + if (!parts.length || parts.some(part => part === '.' || part === '..' || part.includes('\0'))) { + throw new Error(`Invalid SD path: ${path}`); + } + let directory = root; + for (const part of parts.slice(0, -1)) { + const existing = directory.children.get(part); + if (existing && !existing.directory) { + throw new Error(`SD path conflicts with a file: ${path}`); + } + if (!existing) { + directory.children.set(part, makeNode(part, true, directory)); + } + directory = directory.children.get(part); + } + const name = parts.at(-1); + if (directory.children.has(name)) { + throw new Error(`Duplicate SD path: ${path}`); + } + const file = makeNode(name, false, directory); + file.bytes = bytes instanceof Uint8Array ? bytes : new Uint8Array(bytes); + directory.children.set(name, file); + } + return root; +} + +function sanitizeShortPart(value) { + return value.toUpperCase().replace(/[^A-Z0-9$%'-_@~`!(){}^#&]/g, '_'); +} + +function assignShortNames(directory) { + const used = new Set(); + for (const child of directory.children.values()) { + const dot = child.directory ? -1 : child.name.lastIndexOf('.'); + const rawBase = dot > 0 ? child.name.slice(0, dot) : child.name; + const rawExt = dot > 0 ? child.name.slice(dot + 1) : ''; + const base = sanitizeShortPart(rawBase); + const ext = sanitizeShortPart(rawExt).slice(0, 3); + const direct = base.length >= 1 && base.length <= 8 && rawExt.length <= 3 && `${base}${ext ? `.${ext}` : ''}` === child.name.toUpperCase(); + let short; + if (direct) { + short = `${base.padEnd(8)}${ext.padEnd(3)}`; + } else { + for (let n = 1; ; n++) { + const suffix = `~${n}`; + short = `${base.slice(0, 8 - suffix.length).padEnd(8 - suffix.length, '_')}${suffix}${ext.padEnd(3)}`; + if (!used.has(short)) { + break; + } + } + } + if (used.has(short)) { + throw new Error(`Duplicate FAT short name for ${child.name}`); + } + used.add(short); + child.shortName = short; + if (child.directory) { + assignShortNames(child); + } + } +} + +function lfnChecksum(shortName) { + let sum = 0; + for (const char of shortName) { + sum = ((sum & 1) << 7) + (sum >> 1) + char.charCodeAt(0); + sum &= 0xff; + } + return sum; +} + +function lfnEntries(node) { + const shortDisplay = `${node.shortName.slice(0, 8).trimEnd()}${node.shortName.slice(8).trimEnd() ? `.${node.shortName.slice(8).trimEnd()}` : ''}`; + if (node.name === shortDisplay) { + return []; + } + const units = Array.from(node.name).flatMap(char => { + const code = char.codePointAt(0); + return code <= 0xffff ? [code] : [0xd800 + ((code - 0x10000) >> 10), 0xdc00 + ((code - 0x10000) & 0x3ff)]; + }); + if (units.length > 255) { + throw new Error(`FAT filename is too long: ${node.name}`); + } + units.push(0); + while (units.length % 13) { + units.push(0xffff); + } + const count = units.length / 13; + const checksum = lfnChecksum(node.shortName); + const entries = []; + const positions = [1, 3, 5, 7, 9, 14, 16, 18, 20, 22, 24, 28, 30]; + for (let sequence = count; sequence >= 1; sequence--) { + const entry = new Uint8Array(32).fill(0xff); + entry[0] = sequence | (sequence === count ? 0x40 : 0); + entry[11] = 0x0f; + entry[12] = 0; + entry[13] = checksum; + entry[26] = 0; + entry[27] = 0; + const chunk = units.slice((sequence - 1) * 13, sequence * 13); + const view = new DataView(entry.buffer); + positions.forEach((position, index) => view.setUint16(position, chunk[index], true)); + entries.push(entry); + } + return entries; +} + +function shortEntry(node) { + const entry = new Uint8Array(32); + setAscii(entry, 0, node.shortName, 11); + entry[11] = node.directory ? 0x10 : 0x20; + const view = new DataView(entry.buffer); + const fatDate = ((2024 - 1980) << 9) | (1 << 5) | 1; + view.setUint16(14, 0, true); + view.setUint16(16, fatDate, true); + view.setUint16(18, fatDate, true); + view.setUint16(22, 0, true); + view.setUint16(24, fatDate, true); + view.setUint16(20, node.firstCluster >>> 16, true); + view.setUint16(26, node.firstCluster & 0xffff, true); + view.setUint32(28, node.directory ? 0 : node.bytes.length, true); + return entry; +} + +function directoryEntryCount(directory) { + let count = directory.parent ? 2 : 0; + for (const child of directory.children.values()) { + count += lfnEntries(child).length + 1; + } + return count + 1; +} + +function allNodes(root) { + const nodes = []; + const visit = node => { + nodes.push(node); + for (const child of node.children.values()) { + visit(child); + } + }; + visit(root); + return nodes; +} + +function allocateClusters(root, availableClusters) { + let next = ROOT_CLUSTER; + const allocate = (node, count) => { + if (next + count - ROOT_CLUSTER > availableClusters) { + throw new Error('SD card contents exceed the 64 MiB image capacity.'); + } + node.clusters = Array.from({ length: count }, (_, i) => next + i); + node.firstCluster = count ? next : 0; + next += count; + }; + for (const node of allNodes(root).filter(node => node.directory)) { + allocate(node, Math.max(1, Math.ceil(directoryEntryCount(node) * 32 / SECTOR_SIZE))); + } + for (const node of allNodes(root).filter(node => !node.directory)) { + allocate(node, Math.ceil(node.bytes.length / SECTOR_SIZE)); + } + return next; +} + +function writeBootSector(image, geometry) { + const boot = image.subarray(0, SECTOR_SIZE); + const view = new DataView(image.buffer); + boot.set([0xeb, 0x58, 0x90]); + setAscii(boot, 3, 'XTEINK ', 8); + view.setUint16(11, SECTOR_SIZE, true); + boot[13] = 1; + view.setUint16(14, RESERVED_SECTORS, true); + boot[16] = FAT_COUNT; + view.setUint16(17, 0, true); + view.setUint16(19, 0, true); + boot[21] = 0xf8; + view.setUint16(22, 0, true); + view.setUint16(24, 63, true); + view.setUint16(26, 255, true); + view.setUint32(28, 0, true); + view.setUint32(32, geometry.totalSectors, true); + view.setUint32(36, geometry.fatSectors, true); + view.setUint16(40, 0, true); + view.setUint16(42, 0, true); + view.setUint32(44, ROOT_CLUSTER, true); + view.setUint16(48, 1, true); + view.setUint16(50, 6, true); + boot[64] = 0x80; + boot[66] = 0x29; + view.setUint32(67, 0x58544549, true); + setAscii(boot, 71, 'XTEINK', 11); + setAscii(boot, 82, 'FAT32', 8); + boot[510] = 0x55; + boot[511] = 0xaa; + image.set(boot, 6 * SECTOR_SIZE); + + const fsInfo = new DataView(image.buffer, SECTOR_SIZE, SECTOR_SIZE); + fsInfo.setUint32(0, 0x41615252, true); + fsInfo.setUint32(484, 0x61417272, true); + fsInfo.setUint32(488, 0xffffffff, true); + fsInfo.setUint32(492, ROOT_CLUSTER + 1, true); + fsInfo.setUint32(508, 0xaa550000, true); + image.copyWithin(7 * SECTOR_SIZE, SECTOR_SIZE, 2 * SECTOR_SIZE); +} + +function dotEntry(name, cluster) { + const entry = new Uint8Array(32); + setAscii(entry, 0, name, 11); + entry[11] = 0x10; + const view = new DataView(entry.buffer); + view.setUint16(20, cluster >>> 16, true); + view.setUint16(26, cluster & 0xffff, true); + return entry; +} + +export function buildFat32Image(files, imageSize = FAT32_IMAGE_SIZE) { + if (imageSize < 64 * 1024 * 1024 || imageSize % SECTOR_SIZE || (imageSize & (imageSize - 1))) { + throw new Error('FAT32 SD image size must be a power of two and at least 64 MiB.'); + } + const geometry = fatGeometry(imageSize); + const root = normalizeFiles(files); + assignShortNames(root); + const nextCluster = allocateClusters(root, geometry.clusters); + const image = new Uint8Array(imageSize); + writeBootSector(image, geometry); + + const fatOffset = RESERVED_SECTORS * SECTOR_SIZE; + const fatView = new DataView(image.buffer, fatOffset, geometry.fatSectors * SECTOR_SIZE); + fatView.setUint32(0, 0x0ffffff8, true); + fatView.setUint32(4, FAT_EOC, true); + for (const node of allNodes(root)) { + node.clusters.forEach((cluster, index) => fatView.setUint32(cluster * 4, node.clusters[index + 1] ?? FAT_EOC, true)); + } + image.copyWithin(fatOffset + geometry.fatSectors * SECTOR_SIZE, fatOffset, fatOffset + geometry.fatSectors * SECTOR_SIZE); + + const dataOffset = (RESERVED_SECTORS + FAT_COUNT * geometry.fatSectors) * SECTOR_SIZE; + const clusterOffset = cluster => dataOffset + (cluster - ROOT_CLUSTER) * SECTOR_SIZE; + for (const directory of allNodes(root).filter(node => node.directory)) { + const entries = []; + if (directory.parent) { + entries.push(dotEntry('. ', directory.firstCluster)); + entries.push(dotEntry('.. ', directory.parent.parent ? directory.parent.firstCluster : ROOT_CLUSTER)); + } + for (const child of directory.children.values()) { + entries.push(...lfnEntries(child), shortEntry(child)); + } + entries.push(new Uint8Array(32)); + const bytes = new Uint8Array(directory.clusters.length * SECTOR_SIZE); + entries.forEach((entry, index) => bytes.set(entry, index * 32)); + directory.clusters.forEach((cluster, index) => image.set(bytes.subarray(index * SECTOR_SIZE, (index + 1) * SECTOR_SIZE), clusterOffset(cluster))); + } + for (const file of allNodes(root).filter(node => !node.directory)) { + file.clusters.forEach((cluster, index) => image.set(file.bytes.subarray(index * SECTOR_SIZE, (index + 1) * SECTOR_SIZE), clusterOffset(cluster))); + } + + const freeClusters = geometry.clusters - (nextCluster - ROOT_CLUSTER); + new DataView(image.buffer).setUint32(SECTOR_SIZE + 488, freeClusters, true); + image.copyWithin(7 * SECTOR_SIZE, SECTOR_SIZE, 2 * SECTOR_SIZE); + return image; +} diff --git a/web/index.html b/web/index.html index cc0439b..ace17f7 100644 --- a/web/index.html +++ b/web/index.html @@ -38,6 +38,7 @@
+
@@ -65,8 +66,17 @@