commit 5f9b2a251ba3c261e8ff99e458de9b07df223e00 Author: Valentin Haudiquet Date: Tue Oct 27 21:30:13 2026 +0100 docs: the ecrou specification ecrou descends from ecr: the microVM runtime that survives strict snap confinement. The founding rule (no host mounts, no namespaces, every crossing a file or a socket) is stated with its syscall budget, sourced from snapd's seccomp allowlist; execution model, exit-code propagation through a private control share, kill-chain cancellation, the spike-snap gate, and seven falsification tests round out the v1 design. diff --git a/SPEC.md b/SPEC.md new file mode 100644 index 0000000..11fe3af --- /dev/null +++ b/SPEC.md @@ -0,0 +1,293 @@ +# ecrou - implementation specification + +## Identity + +ecrou is a microVM runtime for disposable Linux environments. It descends +from ecr, the ephemeral chroot: the name keeps the ancestor, and the +answer to "chroot ou VM" is now VM. As a bonus, an écrou is a nut: the +small part that keeps a bolt from walking out of the wall. + +One sentence: resolve an image reference, boot it as a one-shot microVM, +run one command with live stdio, propagate its exit code, keep whatever +crossed the workspace share, throw the machine away. + +The project is library-first like ecr: `crates/ecrou` is the library +(package `ecrou`), `crates/ecr-cli` becomes `crates/ecrou-cli` (binary +`ecrou`), a thin front-end that parses flags, maps them onto the library +API, and propagates exit codes. The library never writes to the +terminal: diagnostics go through the `log` facade, long operations +report through the `Progress` sink, all output is the CLI's. + +The first consumer is rhema's sandbox seam, which shells out to the +binary the way it shells out to `docker` and `lxc` today. The library +API is designed so a second consumer can arrive without forking the +runtime. + +## What carries over from ecr + +| module | disposition | +|---|---| +| `distro`, `download`, `extract`, `rootfs`, `utils`, `progress` | ported; the image machinery is confinement-neutral plain fs and network | +| `qemu_vm` (cpio builder, KVM probe, launch) | ported and grown; becomes `initramfs` + `qemu` + `vm` | +| `kernel` | ported; Alpine `linux-virt` fetch, cache entry renamed | +| `namespace`, `chroot`, `mount`, `exec` | dropped; they stay in ecr, their ancestor. Their error variants (`UserNamespacesUnavailable`, `BinfmtNotRegistered`, `MissingSubidRange`) leave with them | + +No cache compatibility with ecr: the formats may match but the directory +is `~/.cache/ecrou` and never shared. + +## The founding rule + +ecrou's host process never mounts a filesystem and never creates or +enters a namespace. Every boundary crossing is a file or a socket. This +is the property that lets ecrou run inside a strict snap's default-deny +seccomp allowlist, where every namespace-based runtime, runc, crun, +podman, and ecr's own chroot mode, dies at `mount`. + +The rule is enforced by audit, not by trust: test T6 straces a full run +and fails on a single hit from the denied set. It runs in CI. + +Guest-side mounts are not host mounts. Confinement binds the host +process; the guest kernel may mount proc, devtmpfs, and the workspace +share inside the machine, and nothing a guest does lands on the host +beyond what the shares expose. + +### Confinement budget + +Everything the host process may do, and nothing else: + +- fs: read, write, stat, rename, unlink, mkdir under the cache directory + and the share roots it was given, plus its own private tmpdir +- exec: the kernel it boots and `qemu-system-*` (later: libkrun), found + in PATH or staged in the snap +- `/dev/kvm`: open, ioctl, mmap. Optional; TCG is the fallback when the + probe finds no working KVM (`KVM_GET_API_VERSION` != 12) +- net: TCP and UDP client sockets for registry fetches, slirp's host + side, and localhost listeners only through explicit `hostfwd` +- process: fork, exec, waitpid, and signals for its own children + +Denied by design: `mount`, `umount2`, `unshare`, `setns`, `pivot_root`, +`open_tree`, `move_mount`, `mount_setattr`, `fsopen`, `fsmount`, +`ptrace`, kernel module loading, keyring calls. All absent from snapd's +seccomp allowlist, which is where the allowed list above is drawn from. +Host `mknod` is allowlisted but AppArmor mediates what it can create; +ecrou has no use for it either way. + +## Execution model + +One-shot machines. `ecrou run IMAGE -- CMD` boots one VM whose init +execs CMD as the only workload; when CMD exits, the machine powers off. +There is no exec into a running machine. No engine in the family has +that API: libkrun's `krun_set_exec` is per-context before start, and +crun's krun handler boots a fresh VM for every `crun exec`. A guest +agent and long-lived machines are a later row; per-command boots are the +v1 shape, and the boot budget below is what keeps them honest. + +Ephemeral rootfs, durable workspace. The base system is an initramfs: +it exists only in RAM and dies with the machine. Persistence crosses a +workspace share or it does not happen. + +**Workspace shares.** `--share HOST:GUEST[:ro]`, served by 9p +(`qemu -virtfs local`). The host side is plain fs under one more +directory the budget covers; the guest init mounts it. There is no +implicit share: an empty sandbox is the default, and the consumer +chooses what crosses. This is deliberate; a sandbox runtime does not +silently attach the caller's cwd. + +**Exit codes.** The machine has a private control share: a host tmpdir +the API does not surface, mounted read-write at a fixed guest path. The +init writes CMD's exit code to a file there, then powers off; ecrou +reads it back and exits with it. Signal deaths map to 128+signal, the +shell convention the init already produces. Alternatives considered and +rejected for v1: isa-debug-exit (x86-only, awkward from a shell init) +and semihosting exit (arch-dependent). If the exit file is absent and +QEMU itself failed, that is an engine failure, reported as one, never as +a guest code. + +**Cancellation is a kill.** ecrou-cli spawns QEMU with +`PR_SET_PDEATHSIG` and a kill-on-drop guard, so a killed consumer kills +the machine. No orphan QEMU survives the CLI. This is stronger than +`docker exec`'s cancel, which kills only the local client; the tests +keep it true (T4). + +**Console.** `-serial stdio`, no monitor mux. The QEMU monitor is a +host-level control hatch over the machine; a sandbox runtime does not +mux it into the workload's stdio by accident. `--monitor` restores +`mon:stdio` for development. + +**Networking.** slirp user-mode only. The init configures the guest +statically (10.0.2.15/24, gateway 10.0.2.2, DNS 10.0.2.3); no DHCP +client is assumed. No tap, no bridges, ever: they need host setup the +budget cannot afford. `hostfwd` stays available for a later +sandboxd-in-guest. + +**Env and argv** ride the kernel command line base64-encoded, ecr's +trick, extended to the environment. Sane size is a few KB; oversized +payloads are a caller bug and fail loudly. + +**Memory.** Default 1G, `-m` to override. ecr's CLI defaulted 2G; a +per-command machine is provisioned per command, and agent consumers run +several concurrently. + +## The guest side + +The init is ours, a generated shell script packed into the initramfs and +selected with `rdinit=`. It grows from ecr's (hostname, shell detection, +argv decode, sysrq poweroff fallback) with static net config, share +mounts, the workdir `cd`, and the exit-code handshake added. It uses +only what the image ships; `busybox` is not assumed. + +Kernel: Alpine `linux-virt` per target arch (the `linux-lts` fallback +when no virt flavor exists), fetched and cached as today. The pack step +adds the booted kernel's `/lib/modules` tree into the initramfs: 9p and +virtio_net are modules in `linux-virt`, and the distro image's module +tree does not match the kernel we boot. The size cost is measured in +T5, and gzip compression of the initramfs is the knob if it hurts. + +## Workspace layout + +Cargo workspace, two crates, the ecr split: + +- `crates/ecrou` - the library. Modules: `config`, `distro`, + `download`, `error`, `extract`, `initramfs`, `kernel`, `progress`, + `qemu`, `rootfs`, `utils`, `vm`. +- `crates/ecrou-cli` - the CLI (binary `ecrou`). + +The library must not depend on CLI types; `clap` is a CLI-only +dependency. Everything else from ecr's rules stands: anyhow internal, +`ecrou::Error` semantic variants at the sites that detect them, `log` +for narration, `Progress` for long operations. + +## Library API + +### Rootfs lifecycle (ported) + +`RootfsCache::new(dir)` / `default_dir()`, `prepare`, `prepare_provisioned`, +`persist`, `PrepareRequest`, `PreparedRootfs`: ecr's semantics verbatim, +including digest freshness on `:latest` and the `.provisioned` hot-cell +marker. See ecr's SPEC for the reference description; differences are +the cache directory and the kernel entry names only. + +### `vm::run` + +```rust +pub struct Share { pub host: PathBuf, pub guest: String, pub read_only: bool } +pub struct RunOptions { + pub image: String, // distro ref or OCI image ref + pub arch: String, // empty = host arch + pub memory: String, // default "1G" + pub workspace: Option, // the one rw share consumers name + pub shares: Vec, // additional named shares, ro allowed + pub command: Vec, // empty = the image's default shell + pub workdir: String, // guest cwd; empty = first share's guest path, else / + pub env: Vec<(String, String)>, // composed envp; host env never inherited + pub hostname: String, // empty = generated ecrou-vm-XXXXXX + pub quiet: bool, + pub require_kvm: bool, +} +pub fn run(options: &RunOptions, progress: &dyn Progress) -> Result +``` + +stdio is inherited; the return value is the guest exit code. A +`require_kvm` run with no working KVM is `Error::KvmUnavailable` with +the remediation text. Everything else is `Error::Other` over the anyhow +chain, ecr's shape. The v1 API has no captured-output variant and no +long-lived machine; consumers pump the child's stdio themselves when +they drive the CLI. + +## CLI + +``` +ecrou run [OPTIONS] [-- COMMAND...] +ecrou probe +``` + +| Flag | Default | Description | +|------|---------|-------------| +| `-a, --arch ` | host arch | Target architecture | +| `--share ` | none | Workspace share; repeatable | +| `-m, --memory ` | 1G | Guest memory | +| `--workdir ` | first share, else `/` | Guest cwd | +| `-e, --env ` | none | Guest env; repeatable | +| `--require-kvm` | false | Fail instead of falling back to TCG | +| `--monitor` | false | Dev: mux the QEMU monitor onto stdio | +| `--quiet` | false | Suppress kernel log on the console | +| `--no-cache` | false | Fresh download, ignore the cache | +| `-v, --verbose` | false | Diagnostics | + +`ecrou probe` reports what a run would find: KVM working or the TCG +fallback, the QEMU binary, the kernel in cache, the cache directory +writable. It is the confinement diagnostic and the support tool. + +## File layout + +``` +~/.cache/ecrou/ +├── ubuntu-noble-amd64.tar.gz +├── alpine-edge-x86_64.tar.gz +├── <...>.digest # manifest digest sidecar +├── <...>.provisioned # hot-cell marker +└── ecrou-default-kernel-x86_64.vmlinuz +``` + +`~/.config/ecrou.yaml` carries over (DNS overrides today). + +## Snap packaging + +Confinement: strict. Plugs: `network`, `home`, `kvm`. Manual +`snap connect` for kvm is accepted for now; auto-connect is a store +request and a later product decision. + +The first milestone is the spike snap, ecr's VM mode behind strict +confinement, before any porting: stage `qemu-system-x86_64` and the +kernel in the snap, or fetch both through the cache in $HOME. Both fit +the budget; T1 is the gate that proves the shape under real AppArmor +and seccomp, and everything else in this document is design until it +passes. + +## Falsification tests + +- **T1, confinement.** The spike snap runs `ecrou run alpine -- uname + -a` under strict confinement. The guest kernel version differing from + the host is the microVM proof, the same check crun's krun man page + prescribes. +- **T2, workspace coherence.** In a shared workspace, the guest runs + git init, add, commit, status; the host sees the commits; a second + machine sees the first's writes. Git's mmap'd index is the trap this + test exists for. +- **T3, exit codes.** `ecrou run alpine -- sh -c "exit 7"` exits 7. + A guest signal death exits 128+signal. A killed QEMU reports an + engine failure, never a guest code. +- **T4, kill chain.** SIGKILL ecrou-cli mid-run; QEMU is gone within a + second, no orphan, verified by scanning the process table. +- **T5, boot economics.** KVM boot-to-init for alpine with + `linux-virt` lands under 1.5 s including the module-tree pack cost. + TCG numbers are recorded, not gated. +- **T6, no-mount audit.** `strace -f -e trace=mount,umount2,unshare, + setns,pivot_root,open_tree,move_mount,fsopen,fsmount` over a full run + with shares: zero hits. This test is the founding rule. +- **T7, guest network.** An https fetch from the guest succeeds. + +## Later + +- **libkrun engine.** The FFI lives here; ecrou already carries unsafe + libc, and rhema keeps its `forbid(unsafe_code)`. Sub-second boots, + the kernel bundled in libkrunfw, TSI networking instead of slirp. + Same budget, one more engine. +- **Tiny initramfs.** A fixed small init with the rootfs itself served + over 9p, killing the per-run pack cost and the RAM floor. +- **Guest agent, long-lived machines.** Exec without a boot; the shape + to grow into when per-command VMs stop being right. +- **passt** if slirp throughput ever matters. +- **sandboxd-in-guest** over hostfwd: rhema's http backend pointed + into a machine. +- **Rootfs persistence** through `prepare_provisioned` hot cells, + already in the ported code. + +## Open questions + +- 9p versus virtiofsd as the share transport. 9p needs no extra daemon; + virtiofsd is the modern path but is another staged binary. Decide + when T2 runs; the `Share` API does not change either way. +- Initramfs compression default: uncompressed boots fastest, gzip + halves the RAM floor. Measure in T5. +- Whether `probe` belongs in the library or stays CLI-only.