feat: add e2e tests, fix server shutdown and map race, update docs

- Add end-to-end test suite covering HTTP tunnel round-trip, POST
  forwarding, unknown tunnel 404, duplicate name rejection, unauthorized
  access, info endpoint, multi-tunnel routing, and graceful shutdown
- Fix server graceful shutdown by closing TCP listener on context cancel
- Fix data race in pkg/maps Entries() iterator by holding RLock
- Rewrite README with architecture, configuration, and usage docs
- Add AGENTS.md with project conventions and architecture guide
- Update flake.nix (add gopls) and flake.lock
This commit is contained in:
2026-05-03 22:29:36 -04:00
parent fa8f4312df
commit 801f0f588f
7 changed files with 742 additions and 18 deletions

71
AGENTS.md Normal file
View File

@@ -0,0 +1,71 @@
# Conduit — Agent Guidelines
## Project Overview
Conduit is a self-hosted tunneling service (Go, single binary). A **server** (`conduit serve`) runs on a public host and routes incoming HTTP requests by subdomain to registered **tunnels**. A **client** (`conduit tunnel`) connects via WebSocket, receives forwarded traffic, and relays it to a local target using either an HTTP reverse-proxy or raw TCP dial.
## Build & Test
```bash
# Build all platforms
make build_local
# Run tests
make tests # includes coverage
# Lint
golangci-lint run
```
Go 1.25+ is required (`go.mod`). Nix devshell provides Go, gopls, golangci-lint.
## Architecture at a Glance
```
server/server.go — Raw TCP listener, reads Host header, routes to tunnel or control API
tunnel/tunnel.go — Core Tunnel struct, WebSocket message loop, stream management
tunnel/forwarder.go — Forwarder interface; factory selects HTTP or TCP forwarder
tunnel/http_forwarder.go — httputil.ReverseProxy served over net.Pipe via multiConnListener
tunnel/tcp_forwarder.go — Direct net.Dial TCP forwarding
tunnel/stream.go — Stream interface (io.ReadWriteCloser + Source/Target)
store/store.go — In-memory request/response recorder with pub/sub (SSE)
web/web.go — Local tunnel monitor (port 8181), SSE endpoint
config/config.go — Reflection-based config from struct tags → flags + env vars
pkg/maps/map.go — Generic sync.RWMutex-guarded map
types/message.go — WebSocket message envelope (data | close)
```
## Code Conventions
- **Go style**: standard `gofmt`, golangci-lint with `.golangci.toml`
- **Comment style**: Title Case heading above logical blocks (see root `AGENTS.md`)
- **Config**: add struct tags (`json`, `default`, `description`) to `ServerConfig` or `ClientConfig` — flags and env vars are auto-derived
- **Logging**: use `logrus` (`log` alias); structured fields preferred
- **Concurrency**: use `pkg/maps.Map` for shared maps; protect other shared state with `sync.Mutex`
- **Error handling**: return errors up; log at command/entry-point level. Use `fmt.Errorf` with `%w` for wrapping
## Key Patterns
1. **Raw TCP → HTTP upgrade**: The server reads from a raw TCP connection to inspect the Host header before deciding whether to handle as a control API request or tunnel the connection via WebSocket.
2. **Reconstructed connection**: After reading the HTTP request from the raw conn, `reconstructedConn` replays the consumed bytes so the full TCP stream can be forwarded.
3. **Forwarder abstraction**: `Forwarder` interface decouples tunnel transport from protocol handling. HTTP forwarder uses `net.Pipe` + `multiConnListener` to feed connections into a standard `http.Server`.
4. **Context-threaded records**: Request records are attached to context in `RecordRequest` and retrieved in `RecordResponse` via the `ModifyResponse` hook.
## Adding a New Forwarder
1. Implement `tunnel.Forwarder` interface (`Type()`, `Initialize()`, `Start()`)
2. Add a case in `tunnel.NewForwarder()` factory
3. Add corresponding `ForwarderType` const
## File Locations
| Concern | Files |
|---------|-------|
| CLI entry | `main.go`, `cmd/` |
| Server | `server/` |
| Tunneling | `tunnel/` |
| Config | `config/` |
| Storage | `store/` |
| Web UI | `web/`, `web/pages/` |
| Shared types | `types/` |
| Utilities | `pkg/maps/` |