pi-lsp/AGENTS.md

# pi-lsp — LSP Extension for pi Coding Agent

## Overview

A [pi coding agent](https://github.com/mariozechner/pi-coding-agent) extension that provides LSP tools (`lsp_hover`, `lsp_definition`, etc.) to the LLM, plus automatic diagnostics after edits. It runs a **long-lived background daemon** so language servers stay warm across tool calls instead of cold-starting on every request.

## Non-Obvious Things (Read First)

### Two-Process Architecture

The extension has **two separate Node.js processes** communicating over a Unix socket:

| Layer | File(s) | Runs In | Responsibility |
|-------|---------|---------|----------------|
| **Extension** | `index.ts` | pi's process | Registers tools/commands, calls into daemon, formats responses |
| **Daemon** | `src/daemon.ts` + `src/client.ts` | Detached background process | Owns LspClient instances, spawns/manages language server processes |

The extension is **stateless** — it opens a fresh socket connection per request. The daemon is **stateful** — it caches one LSP server per `(server.id, rootDir)` and evicts on idle timeout.

### Daemon Protocol

Communication is **newline-delimited JSON (NDJSON)** over a Unix socket at `$XDG_RUNTIME_DIR/pi-lsp-$UID.sock`. Each line is one independent request/response pair with an `id` field for matching. See `src/daemonProtocol.ts` for the type definitions (`DaemonRequest`, `DaemonResponse`).

Current ops: `request`, `diagnostics`, `status`, `shutdown`, `destroy_server`. `request` and `diagnostics` include a `launch` context with the caller env. The env is used only when spawning a new server for `(server.id, rootDir)`; existing running servers keep their original process env until idle eviction or manual destroy/restart.

### Server Lifecycle

1. First LSP tool call for a file triggers `getOrCreateEntry()` in the daemon
2. `pickServer()` matches the file extension against `server.ts` registry
3. `findRoot()` walks upward looking for root markers (e.g., `go.mod`, `tsconfig.json`)
4. A new `LspClient` is spawned with the caller/session environment from the daemon request, initialized via LSP `initialize`/`initialized`, and waited on (`waitForReady()`)
5. The file is synced via `didOpen` or `didChange` (based on mtime comparison)
6. On idle timeout (default 5 min), the entry is evicted and the server process killed

### File Sync Strategy

The daemon tracks opened files per-entry in a `Map<uri, mtimeMs>`. On each request:
- **First access** → `didOpen` with full file contents
- **mtime changed** → `didChange` with full text replacement
- **mtime unchanged** → skip (server already has it)

A per-entry `serializer` promise chain prevents concurrent syncs from racing.

### Workspace File Watching

Each `ClientEntry` lazily owns a `WorkspaceWatcher` (`src/watcher.ts`,
chokidar + picomatch) that translates filesystem events into
`workspace/didChangeWatchedFiles` notifications. This keeps the server's
workspace index fresh when files are created/changed/deleted **outside** of
LSP tool calls (build scripts, codegen, `git checkout`, the agent's own
file writes).

Non-obvious bits:

- **Patterns come from the server.** We honor `client/registerCapability`
  for `workspace/didChangeWatchedFiles` and store the registrations on the
  `LspClient`. **Don't re-stub those handlers**; they look harmless but
  break the entire feature. If a server doesn't register, we don't watch.
- **Servers send mixed pattern forms.** Gopls registers absolute-path
  globs (`/abs/root/**/*.go`); others send relative (`**/*.ts`) or
  `RelativePattern` objects. `compileWatchers()` tries both relative and
  absolute matching against each event so we accept all forms.
- **Ignore layering.** Always-ignore baseline (`.git/`, `.DS_Store`) +
  root `.gitignore` parsed via the `ignore` package + a small fallback
  for non-git workspaces. Nested gitignores aren't supported yet.
- **Startup readiness.** The daemon waits for chokidar's initial scan, capped
  at 5s, so first requests don't hang indefinitely on huge workspaces.
- **Debounce.** 50ms quiet period, capped at 500ms max wait so sustained
  event streams (branch switches) still flush in bounded time.
- **Watcher and mtime-sync coexist.** When the agent edits a file we'll
  emit `didChangeWatchedFiles` *and* the next request's `syncFile` will
  send a `didChange`. Servers treat the two as orthogonal (workspace
  index vs. editor buffer) and dedupe internally. This matches VS Code.
- **Rollback.** `PI_LSP_DISABLE_WATCHERS=1` short-circuits all watcher
  creation — if something goes wrong in a real workspace, this restores
  the prior "only the queried file is synced" behavior.

### Extension vs Daemon Responsibilities

| Concern | Where |
|---------|-------|
| Which server handles `.go` / `.ts`? | Both — `server.ts` is shared, but **extension** calls `pickServer()` for tools, **daemon** calls it for caching |
| Spawning/killing server processes | Daemon only |
| Formatting LSP responses for the LLM | Extension only (`formatHover`, `formatDefinition`, etc.) |
| Auto-diagnostics after `edit`/`write` | Extension (listens to `tool_result` event) |
| CLI one-shot mode (`--no-daemon`) | `cli.ts` directly uses `src/client.ts`, bypassing daemon |

## Project Structure

```
index.ts            — Extension entry point (tools, commands, auto-check flag)
server.ts           — Built-in LSP server registry (gopls, typescript-language-server, pyright, ...)
cli.ts              — CLI for testing/debugging (daemon-aware or --no-daemon)
daemon.ts           — Entrypoint that starts the daemon process

src/
  client.ts         — LspClient: spawns a language server, JSON-RPC handshake, file sync, file-watcher registrations
  watcher.ts        — WorkspaceWatcher: chokidar + picomatch → workspace/didChangeWatchedFiles batches
  commands.ts       — CLI command dispatcher (maps command names → LSP methods)
  config.ts         — Per-repo `.pi-lsp.json` loader: walk-up + merge with built-ins, mtime cache
  daemonClient.ts   — High-level helpers (daemonRequest, daemonDiagnostics, etc.)
  daemonProtocol.ts — Shared types, socket path, NDJSON send/receive, autospawn logic
  root.ts           — pickServer(), findServerById(), getServersForPath(), findRoot(), URI/path conversion
  types.ts          — ServerConfig interface, LspCommand union
```

### Per-Repo Config (`.pi-lsp.json`)

Users can add/override/disable servers without editing `server.ts`. `src/config.ts`
walks upward from a given path to find `.pi-lsp.json`, parses it, and merges
with the built-in `servers` list:

- New `id` → appended (must supply `match`, `command`, `args`, `rootMarkers`).
- Existing `id` → shallow-merged over the built-in (user fields win).
- `disable: []` → filtered out at the end.

Results are cached per config path, invalidated by mtime. `getServersForPath(p)`
is the **single entry point** — don't import the raw `servers` array from
`server.ts` outside `src/config.ts`. The daemon resolves servers at
`getOrCreateEntry()` time via `findServerById(filePath, id)`, so spawned
servers reflect the config of the file being acted on. **Already-running**
entries don't see config changes; users must `/lsp-destroy` to respawn.

## Adding a Server (Built-In)

For servers shipped with pi-lsp, edit `server.ts`. (For per-repo additions,
users should drop a `.pi-lsp.json` at the repo root — see README.) Add an entry
to the `servers` array:

```typescript
{
  id: "rust-analyzer",
  match: ["rs"],
  command: "rust-analyzer",
  args: [],
  rootMarkers: ["Cargo.toml"],
  languageId: "rust",
}
```

The `command` must be on PATH. No other code changes needed — the daemon and extension pick it up automatically via `pickServer()`.

## Adding a Command

1. Extend the `LspCommand` union in `src/types.ts`
2. Add a handler in `src/commands.ts` (maps command name → LSP method)
3. Register a tool in `index.ts` if it should be callable by the LLM
4. Update `cli.ts` method map if it should work via CLI

## Development Workflow

- **No build step** — everything runs via `tsx` (TypeScript executor)
- Extension is loaded by pi from `~/.pi/extensions/lsp/` or `.pi/extensions/lsp/`
- Daemon is autospawned on first LSP request; logs to `/tmp/pi-lsp-daemon.log`
- Set `LSP_DEBUG=1` to forward language server stderr to the daemon log
- Use `npm run lsp -- <file> <command> '<json>'` for CLI testing
- Use `npm run lsp -- daemon status` to inspect running servers

## Extension API Conventions

The extension uses pi's `ExtensionAPI` (from `@mariozechner/pi-coding-agent`):

- **Tools** — registered via `pi.registerTool()`, callable by the LLM. Parameters use TypeBox schemas.
- **Commands** — registered via `pi.registerCommand()`, invoked as `/cmd-name` in the TUI. Use `ctx.ui.notify()` for feedback.
- **Flags** — registered via `pi.registerFlag()`, accessed as CLI args (e.g., `--lsp-auto-check=false`).
- **Events** — subscribed via `pi.on()`. The auto-check feature listens to `tool_result` and runs diagnostics after `edit`/`write`.

All tool execute functions receive `(toolCallId, params, signal, onUpdate, ctx)` where `ctx` is the `ExtensionContext`.