mirror of
https://github.com/vhaudiquet/ecrou.git
synced 2026-11-12 22:22:44 +00:00
docs: trim SPEC.md to the contracts the code cannot carry
The code is the source of truth now; what stays is the part code cannot carry — the founding rule and confinement budget (fixed: the budget said no child process, the machine child is one), the run contracts (exit codes rewritten to match the engine-failure and reserved-code semantics), the trust model, the falsification tests, and the roadmap. Cut: the engine call list, the RunOptions listing, the CLI flags table, the cache layout tree, the workspace/module map, the ecr porting table, and the open questions — all answered or all mirrors of code that answers them faster.
This commit is contained in:
1 parent
b0ff40dd5d
commit
4aadc5d28a
1 file changed
+147
-344
@@ -1,83 +1,48 @@
|
|||||||
# ecrou - implementation specification
|
# ecrou - contracts and invariants
|
||||||
|
|
||||||
## Identity
|
The code is the source of truth for how ecrou works. This document
|
||||||
|
holds what the code cannot carry: the founding rule and the
|
||||||
|
confinement budget, the run contracts, the trust model, and the
|
||||||
|
falsification tests that keep them honest. Anything descriptive of the
|
||||||
|
current code lives with the code.
|
||||||
|
|
||||||
ecrou is a microVM runtime for disposable Linux environments. It descends
|
## What ecrou is
|
||||||
from ecr, the ephemeral chroot: the name keeps the ancestor, and the
|
|
||||||
answer to "chroot ou VM" is now VM. As a bonus, an écrou is a nut: the
|
|
||||||
small part that keeps a bolt from walking out of the wall.
|
|
||||||
|
|
||||||
One sentence: resolve an image reference, boot it as a one-shot microVM
|
Resolve an image reference, boot it as a one-shot microVM on libkrun,
|
||||||
on libkrun, run one command with live stdio, propagate its exit code,
|
run one command with live stdio, propagate its exit code, keep
|
||||||
keep whatever crossed the workspace share, throw the machine away.
|
whatever crossed a share, throw the machine away.
|
||||||
|
|
||||||
The project is library-first like ecr: `crates/ecrou` is the library
|
The split is library-first and enforced by the dependency edges: the
|
||||||
(package `ecrou`), `crates/ecrou-cli` is the thin front-end (binary
|
library links no clap and no rendering crate, and never writes to the
|
||||||
`ecrou`) that parses flags, maps them onto the library API, and
|
terminal. Diagnostics go through the `log` facade, filtered by
|
||||||
propagates exit codes. The split is enforced by the dependency edges:
|
`RunOptions::log_level`; long operations report through the `Progress`
|
||||||
the library links no clap and no rendering crate. It never writes to
|
sink; in-process failures come back as typed errors. The one exception
|
||||||
the terminal: diagnostics go through the `log` facade, filtered by
|
is the forked machine child's last-gasp diagnostics, which land in the
|
||||||
`RunOptions::log_level` (an engine-owned enum, not UI booleans), long
|
console file the parent streams. The CLI owns clap, rendering, log
|
||||||
operations report through the `Progress` sink (sequential
|
output, and exit codes. A consumer embeds the library exactly the way
|
||||||
`begin`/`advance`/`finish` operations plus uncounted `message`
|
the CLI does: a `RootfsCache` to prepare a rootfs, then `run` to boot
|
||||||
status lines), all output is the CLI's. In-process failures come back as typed errors; the one
|
it. The first consumer shells out to the binary the way it shells out
|
||||||
exception is the forked machine child's last-gasp diagnostics, which
|
to `docker`; the API must let a second consumer arrive without forking
|
||||||
land in the console file the parent streams. The CLI owns clap, the
|
the runtime.
|
||||||
indicatif rendering of progress events, the log sink, and exit codes.
|
|
||||||
A consumer embeds the library exactly the way the CLI does: a
|
|
||||||
`RootfsCache` to prepare a rootfs, then `run` to boot it.
|
|
||||||
|
|
||||||
The first consumer is rhema's sandbox seam, which shells out to the
|
## The engine
|
||||||
binary the way it shells out to `docker` and `lxc` today. The library
|
|
||||||
API is designed so a second consumer can arrive without forking the
|
|
||||||
runtime.
|
|
||||||
|
|
||||||
## What carries over from ecr
|
One engine, libkrun, driven through its stable C ABI (the
|
||||||
|
`stable-1.19.x` headers; the main branch is mid-rewrite to a vtable
|
||||||
| module | disposition |
|
API and is not the target). ecrou `dlopen`s `libkrun.so.1` at runtime
|
||||||
|---|---|
|
and fails with a remediation message naming the install when it is
|
||||||
| `distro`, `download`, `extract`, `rootfs`, `utils`, `progress` | ported; the image machinery is confinement-neutral plain fs and network |
|
missing; nothing links at build time, and the snap bundles the library
|
||||||
| `kernel` | dropped; the kernel is libkrun's, bundled in libkrunfw |
|
|
||||||
| `qemu_vm` | dropped from v1; its KVM probe logic (open `/dev/kvm`, `KVM_GET_API_VERSION` == 12) moves into the `krun` module |
|
|
||||||
| `namespace`, `chroot`, `mount`, `exec` | dropped; they stay in ecr, their ancestor. Their error variants (`UserNamespacesUnavailable`, `BinfmtNotRegistered`, `MissingSubidRange`) leave with them |
|
|
||||||
|
|
||||||
No initramfs and no cpio builder: the rootfs crosses as a virtiofs
|
|
||||||
share, never as a packed archive.
|
|
||||||
|
|
||||||
No cache compatibility with ecr: the formats may match but the directory
|
|
||||||
is `~/.cache/ecrou` and never shared.
|
|
||||||
|
|
||||||
## The engine: libkrun
|
|
||||||
|
|
||||||
v1 has exactly one engine, libkrun, driven through its stable C ABI
|
|
||||||
(the `stable-1.19.x` headers; the main branch is mid-rewrite to a
|
|
||||||
vtable API and is not the target). ecrou `dlopen`s `libkrun.so.1` at
|
|
||||||
runtime and fails with a remediation message naming the install when it
|
|
||||||
is missing; nothing links at build time, and a snap bundles the library
|
|
||||||
plus libkrunfw next to itself.
|
plus libkrunfw next to itself.
|
||||||
|
|
||||||
The VMM runs inside the ecrou process: `krun_start_enter` enters the
|
KVM is required; there is no TCG fallback. A machine without working
|
||||||
VM loop on the calling thread. There is no child process to shepherd,
|
KVM is `Error::KvmUnavailable` with the remediation text, and `ecrou
|
||||||
which makes the kill contract below trivial.
|
probe` reports it before any run. The VMM runs inside the machine
|
||||||
|
child, not on the caller's thread (see Cancellation).
|
||||||
|
|
||||||
The FFI lives in `crates/ecrou/src/krun.rs`. ecrou carries unsafe libc
|
libkrun's mapped-volumes entry point is explicitly unsupported on
|
||||||
code; rhema, its first consumer, keeps `#![forbid(unsafe_code)]` and
|
Linux — its header asks for user namespaces and bind mounts, precisely
|
||||||
drives the CLI.
|
what ecrou cannot do — so persistence crosses a virtiofs share mounted
|
||||||
|
by the guest init instead.
|
||||||
The v1 API surface is eleven calls: `krun_set_log_level`,
|
|
||||||
`krun_create_ctx`, `krun_set_vm_config`, `krun_set_root`,
|
|
||||||
`krun_add_virtiofs`, `krun_set_workdir`, `krun_set_exec`,
|
|
||||||
`krun_set_env`, `krun_set_console_output`, `krun_start_enter`,
|
|
||||||
`krun_check_nested_virt`. `krun_set_mapped_volumes` exists but is
|
|
||||||
explicitly unsupported on Linux (its header says to use user namespaces
|
|
||||||
and bind-mounts instead, which is precisely what ecrou cannot do); the
|
|
||||||
workspace is a tagged `krun_add_virtiofs` share mounted by the guest
|
|
||||||
init instead.
|
|
||||||
|
|
||||||
KVM is required. There is no TCG fallback; a machine without working
|
|
||||||
KVM is an `Error::KvmUnavailable` with the remediation text, and
|
|
||||||
`ecrou probe` reports it before any run. QEMU as a fallback engine is a
|
|
||||||
Later row, and its arrival would not change the API.
|
|
||||||
|
|
||||||
## The founding rule
|
## The founding rule
|
||||||
|
|
||||||
@@ -91,9 +56,9 @@ The rule is enforced by audit, not by trust: test T6 straces a full run
|
|||||||
and fails on a single hit from the denied set. It runs in CI.
|
and fails on a single hit from the denied set. It runs in CI.
|
||||||
|
|
||||||
Guest-side mounts are not host mounts. Confinement binds the host
|
Guest-side mounts are not host mounts. Confinement binds the host
|
||||||
process; the guest kernel may mount proc, devtmpfs, and the workspace
|
process; the guest kernel may mount proc, devtmpfs, and the shares
|
||||||
share inside the machine, and nothing a guest does lands on the host
|
inside the machine, and nothing a guest does lands on the host beyond
|
||||||
beyond what the shares expose.
|
what the shares expose.
|
||||||
|
|
||||||
### Confinement budget
|
### Confinement budget
|
||||||
|
|
||||||
@@ -106,14 +71,15 @@ Everything the host process may do, and nothing else:
|
|||||||
- `/dev/kvm`: open, ioctl, mmap
|
- `/dev/kvm`: open, ioctl, mmap
|
||||||
- net: TCP and UDP client sockets; with TSI these carry the guest's
|
- net: TCP and UDP client sockets; with TSI these carry the guest's
|
||||||
connections too, proxied host-side
|
connections too, proxied host-side
|
||||||
- process: threads, futexes, eventfds, timers; no children in v1
|
- process: threads, futexes, eventfds, timers; one forked child, the
|
||||||
|
machine, carrying PR_SET_PDEATHSIG
|
||||||
|
|
||||||
Denied by design: `mount`, `umount2`, `unshare`, `setns`, `pivot_root`,
|
Denied by design: `mount`, `umount2`, `unshare`, `setns`, `pivot_root`,
|
||||||
`open_tree`, `move_mount`, `mount_setattr`, `fsopen`, `fsmount`,
|
`open_tree`, `move_mount`, `mount_setattr`, `fsopen`, `fsmount`,
|
||||||
`ptrace`, kernel module loading, keyring calls. All absent from snapd's
|
`ptrace`, kernel module loading, keyring calls. All absent from snapd's
|
||||||
seccomp allowlist, which is where the allowed list above is drawn from.
|
seccomp allowlist, which is where the allowed list above is drawn from.
|
||||||
Two absences need a spike-time check because VMMs plausibly touch them:
|
Two absences need a spike-time check because VMMs plausibly touch them:
|
||||||
`sched_setaffinity` and `openat2`. The spike straces for EPERM on
|
`sched_setaffinity` and `openat2` (T1). The spike straces for EPERM on
|
||||||
allowed-looking calls, and anything libkrun genuinely needs that the
|
allowed-looking calls, and anything libkrun genuinely needs that the
|
||||||
allowlist denies goes on a snapd issue, never into a workaround.
|
allowlist denies goes on a snapd issue, never into a workaround.
|
||||||
|
|
||||||
@@ -122,272 +88,105 @@ allowlist denies goes on a snapd issue, never into a workaround.
|
|||||||
One-shot machines. `ecrou run IMAGE -- CMD` prepares a rootfs, boots
|
One-shot machines. `ecrou run IMAGE -- CMD` prepares a rootfs, boots
|
||||||
one VM whose payload is CMD, streams its stdio, and exits with CMD's
|
one VM whose payload is CMD, streams its stdio, and exits with CMD's
|
||||||
code when the machine powers off. There is no exec into a running
|
code when the machine powers off. There is no exec into a running
|
||||||
machine. No engine in the family has that API: `krun_set_exec` is
|
machine: no engine in the family has that API, and per-command boots
|
||||||
per-context before start, and crun's krun handler boots a fresh VM for
|
are the shape. A guest agent and long-lived machines are a Later row.
|
||||||
every `crun exec`. A guest agent and long-lived machines are a Later
|
|
||||||
row; per-command boots are the v1 shape, and the boot budget below is
|
|
||||||
what keeps them honest.
|
|
||||||
|
|
||||||
Ephemeral rootfs, durable workspace. The base system is a scratch copy
|
Ephemeral rootfs, durable workspace. The base system is a scratch copy
|
||||||
of the image (a `PreparedRootfs`, TempDir-backed): it exists only for
|
of the image; it exists only for the run and dies with it. Persistence
|
||||||
the run and dies with it. Persistence crosses a workspace share or it
|
crosses a share or it does not happen. No initramfs and no cpio
|
||||||
does not happen.
|
builder: the rootfs crosses as a virtiofs share, never as a packed
|
||||||
|
archive.
|
||||||
|
|
||||||
**Workspace shares.** `-v HOST:GUEST[:ro]` (`--volume`), one virtiofs
|
**Workspace shares.** `-v HOST:GUEST[:ro]`, one virtiofs share per
|
||||||
share per entry (`krun_add_virtiofs`, `krun_set_root` keeps the rootfs
|
entry. The host side is plain fs; the guest init mounts each share at
|
||||||
itself).
|
its guest path. There is no implicit share: an empty machine is the
|
||||||
The host side is plain fs; the guest init mounts each share at its
|
default, and the consumer chooses what crosses — a sandbox runtime
|
||||||
guest path. There is no implicit share: an empty sandbox is the
|
does not silently attach the caller's cwd.
|
||||||
default, and the consumer chooses what crosses. This is deliberate; a
|
|
||||||
sandbox runtime does not silently attach the caller's cwd.
|
|
||||||
|
|
||||||
Ownership needs no mapping layer. The guest runs as root inside its own
|
Ownership needs no mapping layer. The guest runs as root inside its
|
||||||
kernel, and libkrun's simplified virtiofs semantics report the
|
own kernel, and libkrun's simplified virtiofs semantics report the
|
||||||
requesting guest uid for ownership while storing permission bits
|
requesting guest uid for ownership while storing permission bits
|
||||||
host-side; writes through a share land owned by the host user. The
|
host-side; writes through a share land owned by the host user.
|
||||||
rootless subuid problem the podman shape has does not exist here.
|
|
||||||
|
|
||||||
**The guest init.** A small script provisioned into the scratch rootfs
|
**The guest init.** A small script provisioned into the scratch rootfs
|
||||||
before boot, for every run: it mounts each share, cds to the workdir,
|
for every run: it mounts each share, cds to the workdir, runs the
|
||||||
runs the payload, and writes the exit code to the control file. The
|
payload, and writes the exit code to the control file. It never execs
|
||||||
hostname and workdir ride `krun_set_exec`'s config natively (the init
|
— the init must outlive the payload to report it — and has no
|
||||||
blob applies both); the script's own hostname and cd lines stay as
|
share-less special case: the same contract covers a bare shell run.
|
||||||
fallbacks. There is no exec (the init must outlive the payload to
|
The payload is an argv vector taken literally; a single argument
|
||||||
report it) and no share-less special case: the same contract covers a
|
containing whitespace is a quoted shell line (`apk add x && y`) and
|
||||||
bare shell run. The payload itself is an argv vector taken
|
runs through the image's shell as `sh -c`. Ending the machine is not
|
||||||
literally; the one rewrite is a single argument containing whitespace,
|
the script's job: when it exits, the init blob reports the code and
|
||||||
which is a quoted shell line ("apk add x && y") and runs through the
|
the VMM powers the machine off.
|
||||||
image's shell as `sh -c`. Ending the machine is not the script's job
|
|
||||||
either:
|
|
||||||
when the script exits, the init blob reports the code and the VMM
|
|
||||||
powers the machine off (the libkrunfw kernel has no sysrq).
|
|
||||||
|
|
||||||
**Exit codes.** On a normal payload exit the VMM terminates its own
|
**Exit codes.** Only the guest init can mint a guest exit code: it
|
||||||
process with the blob-reported code, so `krun_start_enter` never
|
reports the payload's code through the control file and the VMM exits
|
||||||
returns: run() forks, the child enters the VM loop, and the parent
|
the machine child with it. A signal on the machine child means the VMM
|
||||||
reaps it and reads the code from the exit status. Signal deaths map to
|
or the engine died — the payload never reported — and `run` fails with
|
||||||
128+signal. The control file remains the diagnostic when the boot dies
|
an engine-failure error, never a 128+signal guess at a guest code.
|
||||||
without the init reporting. The mechanism is one hop shorter than
|
125 and 126 are reserved by the machinery itself: the child bails
|
||||||
designed here, and the contract for consumers is unchanged.
|
with 125 on a boot failure and the init refuses with 126 when a share
|
||||||
|
would not mount or the workdir is missing. Ending on either without a
|
||||||
|
control-file code is an error naming the cause; a payload that
|
||||||
|
genuinely exits either code reported through the file first, and the
|
||||||
|
file is the arbiter. The control file remains the boot diagnostic in
|
||||||
|
every other death.
|
||||||
|
|
||||||
**Cancellation is a kill.** The VMM lives in a child of the ecrou
|
**Cancellation is a kill.** The VMM lives in a child of the ecrou
|
||||||
process carrying PR_SET_PDEATHSIG, so a killed consumer kills the
|
process carrying PR_SET_PDEATHSIG, so a killed consumer kills the
|
||||||
machine with it. No orphan survives. This is stronger than `docker
|
machine with it. No orphan survives. This is stronger than `docker
|
||||||
exec`'s cancel, which kills only the local client; the tests keep it
|
exec`'s cancel, which kills only the local client (T4).
|
||||||
true (T4).
|
|
||||||
|
|
||||||
**Console.** Three wirings, chosen by the run's flags.
|
**Console.** Three wirings, chosen by the run's flags.
|
||||||
|
|
||||||
Headless (the default): guest output reaches real stdout through a
|
Headless (the default): guest output reaches real stdout through a
|
||||||
console file the parent streams while the machine runs, with a final
|
console file the parent streams while the machine runs, with a final
|
||||||
drain after the reap. The direct stdio wiring does not survive contact
|
drain after the reap. The child unwires fd 0 to `/dev/null`: with a
|
||||||
with reality: with a tty on fd 0 libkrun's terminal setup stops a
|
tty on fd 0 libkrun's terminal setup stops a background caller with
|
||||||
background caller with SIGTTOU, and without one its console bytes go
|
SIGTTOU, and /dev/null keeps the console plain while still forwarding
|
||||||
to the log sink, not the terminal. The child therefore unwires fd 0 to
|
a Ctrl-C sent to the process group into the guest.
|
||||||
/dev/null (no guest input; a Ctrl-C to the process group is still
|
|
||||||
forwarded into the guest).
|
|
||||||
|
|
||||||
`-t`: the machine owns the terminal. With no console output file set,
|
`-t`: the machine owns the terminal. The guest console rides the
|
||||||
libkrun wires the guest console to the process's own stdin and stdout,
|
process's own stdin and stdout, the tty goes raw, and the parent saves
|
||||||
puts the tty in raw mode, and restores it through its exit observers.
|
and restores the termios around the run — libkrun's exit observers do
|
||||||
The parent additionally saves and restores the termios around the run:
|
not run in a SIGKILLed machine. Input is the typed keyboard; Ctrl-C
|
||||||
the observers do not run in a SIGKILLed machine. Input is the typed
|
reaches the guest as a byte (raw mode has no ISIG) and the guest's
|
||||||
keyboard; Ctrl-C reaches the guest as a byte (raw mode has no ISIG)
|
line discipline signals its own foreground job. The run refuses to
|
||||||
and the guest's line discipline signals its own foreground job. The
|
start without a terminal on stdin and stdout, and `--capture` is
|
||||||
run refuses to start without a terminal on stdin and stdout, and
|
refused with `-t`: the payload's streams cannot land in files and on
|
||||||
`--capture` is refused with `-t` (the payload's streams cannot land in
|
the terminal at once.
|
||||||
files and on the terminal at once).
|
|
||||||
|
|
||||||
`-i` without `-t`: pipes. libkrun still owns the process stdio, and
|
`-i` without `-t`: pipes. libkrun exposes the run's non-tty stdio
|
||||||
its non-tty streams surface in the guest as virtio console ports
|
streams as virtio console ports (`krun-stdin`, `krun-stdout`,
|
||||||
(`krun-stdin`, `krun-stdout`, `krun-stderr` under `/dev/virtio-ports`);
|
`krun-stderr` under `/dev/virtio-ports`), and the guest init points
|
||||||
the guest init re-points the payload's stdio onto whichever port nodes
|
the payload's stdio onto whichever port nodes exist, so
|
||||||
exist, so `echo hi | ecrou run -i alpine -- cat` crosses the machine.
|
`echo hi | ecrou run -i alpine -- cat` crosses the machine. A stream
|
||||||
A stream the host left on a tty has no port node and stays on the
|
the host left on a tty has no port node and stays on the console.
|
||||||
console. There is no QEMU monitor to mux; `--verbose` maps to
|
|
||||||
`krun_set_log_level`.
|
|
||||||
|
|
||||||
**Networking.** TSI, libkrun's default: the guest has no interface, and
|
|
||||||
its socket calls are proxied on the host's stack by the VMM process.
|
|
||||||
ecrou pins it explicitly (a vsock with AF_INET hijacked) instead of
|
|
||||||
leaving the heuristics mode to sniff. The guest resolver is the host's
|
|
||||||
uplink config with loopback nameservers stripped (loopback never
|
|
||||||
crosses the proxy; verified by hand). DNS is proxied with the rest.
|
|
||||||
Known upstream limitation: per-stream throughput in this generation of
|
|
||||||
the ln driver is tens of KB/s against a CDN, TLS included; workable,
|
|
||||||
not fast, and worth re-measuring on every engine update. Port
|
|
||||||
publishing (`krun_set_port_map`) is how a later
|
|
||||||
sandboxd-in-guest serves the host. No tap, no bridges, ever: they need
|
|
||||||
host setup the budget cannot afford.
|
|
||||||
|
|
||||||
**Memory and CPUs.** `krun_set_vm_config`; default 1G and
|
|
||||||
all-logical-cpus, `-m` to override. A per-command machine is
|
|
||||||
provisioned per command, and agent consumers run several concurrently.
|
|
||||||
|
|
||||||
## Workspace layout
|
|
||||||
|
|
||||||
Cargo workspace, two crates, the ecr split:
|
|
||||||
|
|
||||||
- `crates/ecrou` - the library. Modules: `config`, `distro`,
|
|
||||||
`download`, `error`, `extract`, `guest` (the init script), `krun`
|
|
||||||
(the FFI), `progress`, `rootfs`, `utils`, `vm`.
|
|
||||||
- `crates/ecrou-cli` - the CLI (binary `ecrou`).
|
|
||||||
|
|
||||||
The library must not depend on CLI types; `clap` is a CLI-only
|
|
||||||
dependency. Everything else from ecr's rules stands: anyhow internal,
|
|
||||||
`ecrou::Error` semantic variants at the sites that detect them, `log`
|
|
||||||
for narration, `Progress` for long operations.
|
|
||||||
|
|
||||||
## Library API
|
|
||||||
|
|
||||||
### Rootfs lifecycle (ported)
|
|
||||||
|
|
||||||
`RootfsCache::new(dir)` / `default_dir()`, `prepare`,
|
|
||||||
`prepare_provisioned`, `pull`, `persist`, `PrepareRequest`,
|
|
||||||
`PreparedRootfs`, `Pulled`: `prepare` is the boot path — a present
|
|
||||||
entry is trusted, the registry is never contacted on a hit, a cold
|
|
||||||
cache downloads. `pull` is the explicit freshness check (unpinned
|
|
||||||
tags re-verified, moved digests re-downloaded, nothing extracted).
|
|
||||||
Plus the `.provisioned` hot-cell marker. The per-run scratch copy is
|
|
||||||
`PreparedRootfs` as ecr already builds it. See ecr's SPEC for the
|
|
||||||
reference description; differences are the cache directory and the
|
|
||||||
absent kernel entries only.
|
|
||||||
|
|
||||||
### `vm::run`
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct Share { pub host: PathBuf, pub guest: String, pub read_only: bool }
|
|
||||||
pub struct RunOptions {
|
|
||||||
pub image: String, // distro ref, OCI image ref, or local
|
|
||||||
// rootfs archive path (./x.tar.gz)
|
|
||||||
pub arch: String, // empty = host arch
|
|
||||||
pub memory: String, // default "1G"
|
|
||||||
pub workspace: Option<Share>, // the one rw share consumers name
|
|
||||||
pub shares: Vec<Share>, // additional named shares, ro allowed
|
|
||||||
pub command: Vec<String>, // empty = the image's default shell
|
|
||||||
pub workdir: String, // guest cwd; empty = /
|
|
||||||
pub env: Vec<(String, String)>, // composed envp; host env never inherited
|
|
||||||
pub hostname: String, // empty = generated ecrou-vm-XXXXXX
|
|
||||||
pub interactive: bool, // host stdin feeds the guest
|
|
||||||
pub tty: bool, // console on the terminal (implies interactive)
|
|
||||||
pub log_level: LogLevel, // Quiet | Normal | Verbose
|
|
||||||
pub capture: Option<PathBuf>, // payload stdout/stderr into files
|
|
||||||
}
|
|
||||||
pub fn run(options: &RunOptions, progress: &dyn Progress) -> Result<i32>
|
|
||||||
```
|
|
||||||
|
|
||||||
Guest output reaches real stdout through the streamed console; the
|
|
||||||
return value is the guest exit code. A run
|
|
||||||
without working KVM is `Error::KvmUnavailable` with the remediation
|
|
||||||
text. Everything else is `Error::Other` over the anyhow chain, ecr's
|
|
||||||
shape. There is no long-lived machine; consumers pump the child's
|
|
||||||
stdio themselves when they drive the CLI.
|
|
||||||
|
|
||||||
**Capture.** The console is one stream, kernel and guest mixed, and a
|
**Capture.** The console is one stream, kernel and guest mixed, and a
|
||||||
consumer that needs the payload's stdout and stderr apart - rhema's
|
consumer that needs the payload's streams apart cannot read it. With
|
||||||
sandbox seam reports them as separate fields - cannot read it. With
|
`--capture DIR` the run adds the directory as an internal rw share at
|
||||||
`capture` set, the run adds the directory as an internal rw share
|
the reserved guest path `/.ecrou-io`, and the init wraps the payload
|
||||||
(guest path `/.ecrou-io`, reserved), and the init wraps the payload in
|
so its stdout and stderr land host-side as `<DIR>/stdout` and
|
||||||
a redirection group pointing at it: the two streams land host-side as
|
`<DIR>/stderr`, written live and simply there when `run` returns. The
|
||||||
`<dir>/stdout` and `<dir>/stderr`, written live, owned by the host
|
console stays on stdout for diagnostics, which is what makes capture
|
||||||
user, and simply there when `run` returns. The console stays on stdout
|
and a nulled stdout composable. A boot failure leaves the files
|
||||||
for diagnostics, which is what makes capture and a nulled stdout
|
absent. No caller share may take the reserved guest paths.
|
||||||
composable. A boot failure leaves the files absent.
|
|
||||||
|
|
||||||
## CLI
|
**Networking.** TSI, libkrun's default, pinned explicitly instead of
|
||||||
|
the heuristics mode: the guest has no interface, and its socket calls
|
||||||
|
are proxied on the host's stack by the VMM process. The guest resolver
|
||||||
|
is the host's uplink config with loopback nameservers stripped (loopback
|
||||||
|
never crosses the proxy); DNS is proxied with the rest. Known upstream
|
||||||
|
limitation: per-stream throughput in this generation of the ln driver
|
||||||
|
is tens of KB/s against a CDN, TLS included; workable, not fast, and
|
||||||
|
worth re-measuring on every engine update. No tap, no bridges, ever:
|
||||||
|
they need host setup the budget cannot afford. Port publishing is how
|
||||||
|
a later in-guest agent serves the host (Later).
|
||||||
|
|
||||||
```
|
**Memory and CPUs.** Default 1G and all logical CPUs; `-m` overrides.
|
||||||
ecrou run [OPTIONS] <IMAGE> [-- COMMAND...]
|
A per-command machine is provisioned per command, and agent consumers
|
||||||
ecrou provision [OPTIONS] <IMAGE> [-- COMMAND...]
|
run several concurrently.
|
||||||
ecrou pull [OPTIONS] <IMAGE>
|
|
||||||
ecrou probe
|
|
||||||
```
|
|
||||||
|
|
||||||
`pull` fetches or refreshes the image's cache entry without booting
|
|
||||||
it: it re-resolves floating references (distro aliases like
|
|
||||||
`latest`/`lts`/`edge`, unpinned OCI tags against the registry
|
|
||||||
digest) and downloads only when the entry is absent or has moved.
|
|
||||||
It is the freshness operation — see the trust model below for how
|
|
||||||
`run` and `pull` divide the work. Local archives are rejected:
|
|
||||||
a file on disk has nothing to fetch.
|
|
||||||
|
|
||||||
The image is a distro name (`alpine`, `ubuntu:noble`), an OCI
|
|
||||||
reference (`ghcr.io/org/image:tag`, `ubuntu@sha256:…`,
|
|
||||||
`repo:tag@sha256:…` — the digest wins over a co-present tag, as in
|
|
||||||
docker), or a local rootfs archive named with a path
|
|
||||||
(`./rootfs.tar.gz`, `/tmp/rootfs.tar`): a path reference is read in
|
|
||||||
place on every run, never downloaded and never cached as a pristine
|
|
||||||
entry.
|
|
||||||
|
|
||||||
| Flag | Default | Description |
|
|
||||||
|------|---------|-------------|
|
|
||||||
| `-i, --interactive` | false | Keep the machine's stdin open: the host's stdin feeds the guest |
|
|
||||||
| `-t, --tty` | false | Console on the terminal: raw mode, the guest sees a tty (implies `-i`; needs a terminal on stdin and stdout) |
|
|
||||||
| `-v, --volume <HOST:GUEST[:ro]>` | none | Share a directory into the machine; the first rw one is the workspace; repeatable |
|
|
||||||
| `-m, --memory <size>` | 1G | Guest memory |
|
|
||||||
| `-w, --workdir <guest path>` | `/` | Guest cwd |
|
|
||||||
| `-e, --env <KEY=VALUE>` | none | Guest env; repeatable |
|
|
||||||
| `--quiet` | false | Suppress kernel log on the console |
|
|
||||||
| `--capture <DIR>` | none | Payload stdout/stderr into `<DIR>/stdout` and `<DIR>/stderr`; incompatible with `-t` |
|
|
||||||
| `--no-cache` | false | Fresh download, ignore the cache (a local archive is always read fresh) |
|
|
||||||
| `--verbose` | false | Diagnostics; raises the libkrun log level |
|
|
||||||
|
|
||||||
`ecrou provision IMAGE -- COMMAND` bakes a command into the image's
|
|
||||||
cache entry: the command runs in a machine over a freshly prepared
|
|
||||||
rootfs and the result is persisted, so every later boot of that image
|
|
||||||
carries what the command installed — packages, users, whatever a
|
|
||||||
one-shot boot cannot keep. A marked entry is reused as-is (the
|
|
||||||
command is skipped), `--no-cache` re-downloads and re-provisions, and
|
|
||||||
a failing command leaves the entry unprovisioned for the next attempt.
|
|
||||||
The marker dies with the entry: a digest refresh drops it, so a moved
|
|
||||||
upstream image is provisioned again, never silently served pristine
|
|
||||||
under an old marker.
|
|
||||||
|
|
||||||
`ecrou probe` reports what a run would find: KVM working (open, ioctl
|
|
||||||
version, nested-virt state), `libkrun.so.1` loadable, the cache
|
|
||||||
directory writable. It is the confinement diagnostic and the support
|
|
||||||
tool.
|
|
||||||
|
|
||||||
## File layout
|
|
||||||
|
|
||||||
```
|
|
||||||
~/.cache/ecrou/
|
|
||||||
├── ubuntu-noble-amd64.tar.gz
|
|
||||||
├── alpine-edge-x86_64.tar.gz
|
|
||||||
├── <...>.digest # manifest digest sidecar
|
|
||||||
└── <...>.provisioned # hot-cell marker
|
|
||||||
```
|
|
||||||
|
|
||||||
## Snap packaging
|
|
||||||
|
|
||||||
Confinement: strict. Plugs: `network`, `home`, `kvm`. Manual
|
|
||||||
`snap connect` for kvm is accepted for now; auto-connect is a store
|
|
||||||
request and a later product decision.
|
|
||||||
|
|
||||||
The snap stages `libkrun.so.1` and libkrunfw next to the binary; at
|
|
||||||
roughly 6 MB combined they cost less than any qemu-system package, and
|
|
||||||
the kernel ships inside libkrunfw, so there is nothing else to fetch or
|
|
||||||
stage.
|
|
||||||
|
|
||||||
Where the libraries come from, as of now: no Ubuntu release publishes
|
|
||||||
a `libkrun.so.1` package at all, and libkrunfw5 exists only in the
|
|
||||||
current development series' universe, not in any LTS the snap bases
|
|
||||||
ship from. Debian sid has both (source package `libkrun`), so the gap
|
|
||||||
is sync lag and it should close by the next LTS. Until then the snap
|
|
||||||
carries both halves itself: a part that builds libkrun from source
|
|
||||||
(the recipe is small: rust, clang for the C shim, `make build-release`
|
|
||||||
with the net feature for TSI) and stages `libkrun.so.1`; and a part
|
|
||||||
that fetches the `libkrunfw5` .deb from the Ubuntu pool and unpacks it
|
|
||||||
into the staging tree. snapcraft puts staged libraries on
|
|
||||||
`LD_LIBRARY_PATH` at runtime, which is exactly the resolution path the
|
|
||||||
`dlopen` in krun.rs takes; there is no ldconfig and no host dependency.
|
|
||||||
When an LTS base archive ships both packages, both parts shrink to
|
|
||||||
`stage-packages` entries and the from-source build is deleted.
|
|
||||||
|
|
||||||
The first milestone is the spike snap, before any porting: a minimal
|
|
||||||
libkrun binary behind strict confinement running a VM. T1 is the gate
|
|
||||||
that proves the shape under real AppArmor and seccomp, and everything
|
|
||||||
else in this document is design until it passes.
|
|
||||||
|
|
||||||
## Trust model
|
## Trust model
|
||||||
|
|
||||||
@@ -436,8 +235,9 @@ stop:
|
|||||||
test exists for, and libkrun's virtiofs DAX windows are exactly where
|
test exists for, and libkrun's virtiofs DAX windows are exactly where
|
||||||
it bites.
|
it bites.
|
||||||
- **T3, exit codes.** `ecrou run alpine -- sh -c "exit 7"` exits 7.
|
- **T3, exit codes.** `ecrou run alpine -- sh -c "exit 7"` exits 7.
|
||||||
A guest signal death exits 128+signal. A failed VMM reports an
|
A guest signal death exits 128+signal (the shell convention crosses
|
||||||
engine failure, never a guest code.
|
the control file). A failed VMM reports an engine failure, never a
|
||||||
|
guest code.
|
||||||
- **T4, kill chain.** SIGKILL ecrou-cli mid-run; the VM is gone with
|
- **T4, kill chain.** SIGKILL ecrou-cli mid-run; the VM is gone with
|
||||||
the process, no stray threads or KVM VMs, verified by scanning the
|
the process, no stray threads or KVM VMs, verified by scanning the
|
||||||
process table and `/dev/kvm` users.
|
process table and `/dev/kvm` users.
|
||||||
@@ -450,6 +250,26 @@ stop:
|
|||||||
- **T7, guest network.** An https fetch from the guest succeeds, DNS
|
- **T7, guest network.** An https fetch from the guest succeeds, DNS
|
||||||
included; a published port is reachable from the host.
|
included; a published port is reachable from the host.
|
||||||
|
|
||||||
|
## Snap packaging
|
||||||
|
|
||||||
|
Confinement: strict. Plugs: `network`, `home`, `kvm`. The manual
|
||||||
|
`snap connect` for kvm is accepted until auto-connect is a store
|
||||||
|
request and a later product decision.
|
||||||
|
|
||||||
|
The snap stages `libkrun.so.1` and libkrunfw next to the binary; the
|
||||||
|
kernel ships inside libkrunfw, so there is nothing else to fetch or
|
||||||
|
stage. No Ubuntu release publishes a `libkrun.so.1` package today, and
|
||||||
|
libkrunfw5 exists only in the current development series' universe,
|
||||||
|
not in any LTS the snap bases ship from — so one part builds libkrun
|
||||||
|
from source and one fetches the libkrunfw5 .deb from the Ubuntu
|
||||||
|
archives, per architecture (the main archive for amd64 and arm64, the
|
||||||
|
ports archive for riscv64), each pinned by digest. snapcraft puts
|
||||||
|
staged libraries on `LD_LIBRARY_PATH` at runtime, which is exactly the
|
||||||
|
resolution path the `dlopen` in krun.rs takes; there is no ldconfig
|
||||||
|
and no host dependency. When an LTS base archive ships both packages,
|
||||||
|
both parts shrink to `stage-packages` entries and the from-source
|
||||||
|
build is deleted.
|
||||||
|
|
||||||
## Later
|
## Later
|
||||||
|
|
||||||
- **QEMU engine.** The TCG fallback for machines without KVM. The
|
- **QEMU engine.** The TCG fallback for machines without KVM. The
|
||||||
@@ -461,29 +281,12 @@ stop:
|
|||||||
to grow into when per-command VMs stop being right.
|
to grow into when per-command VMs stop being right.
|
||||||
- **Provisioned layers, keyed.** `provision` today bakes one entry
|
- **Provisioned layers, keyed.** `provision` today bakes one entry
|
||||||
per image; the next step keys entries by (base digest, hash of the
|
per image; the next step keys entries by (base digest, hash of the
|
||||||
provisioning spec) so consumers — rhema's projects — each get their
|
provisioning spec) so consumers each get their own layer over a
|
||||||
own layer over a shared base, the spec file living in the repo and
|
shared base, the spec file living in the repo and the artifact
|
||||||
the artifact reproducible from it. Marker-hit semantics then skip
|
reproducible from it. Marker-hit semantics then skip the command
|
||||||
the command only for the identical spec, and eviction (LRU or an
|
only for the identical spec, and eviction (LRU or an explicit
|
||||||
explicit prune) keeps derived entries bounded.
|
prune) keeps derived entries bounded.
|
||||||
- **passt** (`krun_set_passt_fd`) if TSI throughput or UDP listening
|
- **passt** (`krun_set_passt_fd`) if TSI throughput or UDP listening
|
||||||
ever matters.
|
ever matters.
|
||||||
- **sandboxd-in-guest** over `krun_set_port_map`: rhema's http backend
|
- **An agent in the guest** over port publishing: an http backend
|
||||||
pointed into a machine.
|
pointed into a machine.
|
||||||
- **Rootfs persistence** through `prepare_provisioned` hot cells,
|
|
||||||
already in the ported code.
|
|
||||||
|
|
||||||
## Open questions
|
|
||||||
|
|
||||||
- Does `krun_start_enter` return the guest exit code natively on
|
|
||||||
poweroff? The control file works regardless; if the return value
|
|
||||||
carries it, the mechanism shrinks to a read.
|
|
||||||
- Console wiring: does the default virtio-console land on the VMM's
|
|
||||||
stdio, and is stdin carried? Decided by a first-run experiment, with
|
|
||||||
`krun_set_console_output` as the fallback.
|
|
||||||
- TSI DNS: which resolver file the guest needs, and whether lookups
|
|
||||||
just work. T7 decides.
|
|
||||||
- dlopen versus link-time linking for the snap; dlopen is the default
|
|
||||||
for the remediation error it enables.
|
|
||||||
- Which distros package libkrun (Fedora does; Debian and Ubuntu do not
|
|
||||||
yet) and what the dev-install story says meanwhile.
|
|
||||||
Reference in new issue
Block a user