prepare_provisioned and persist were sitting there with no consumer:
the machinery for "install once, every later boot carries it" that
one-shot machines cannot do on their own - packages, users, anything
short of a share - and no way to reach it.
ecrou provision IMAGE -- COMMAND is the missing surface. It boots a
machine over a freshly prepared rootfs, runs the command, and
persists the tree into the cache entry, marked provisioned; a marked
entry is extracted as-is on every later call, the command skipped.
A failing command leaves the entry unprovisioned for the next
attempt, and --no-cache re-downloads before re-provisioning.
The engine behind run splits out as vm::run_prepared: the boot half
of run, taking the rootfs the provisioner is about to persist. run
itself is prepare plus that.
One cache fix the provision flow surfaced: a digest refresh replaced
the tarball but left the old .provisioned marker, so a fresh pristine
image would present as provisioned and skip provisioning for good.
The marker now dies with the entry it belonged to.
With preservation on, the crate chmods every file to the mode it
ships as it extracts, and one refused bit killed the whole run:
alpine's usr/bin/chage ships setgid-shadow (02755, group shadow),
and an extracting context that cannot set the bit - an overlayfs
tmp, a confined sandbox - fails the chmod with EPERM and takes the
boot down with "failed to set permissions to 2755".
The crate's own chmod now runs with the special bits masked off,
which no filesystem refuses, and the shipped modes ride the fixups
like directory modes already do: applied after the walk, degraded
to plain rwx when the filesystem refuses them, exact everywhere
else. Unprivileged tar degrades the same way; the guest re-applies
whatever it needs. The setgid regression test pins the round trip.
The console is one stream, kernel and guest mixed, so a consumer that
needs the payload's stdout and stderr apart - rhema's sandbox seam
reports them as separate fields - could not get it. With capture set,
the run adds the directory as an internal rw share (guest path
/.ecrou-io, reserved) and the init wraps the payload in a redirection
group pointing at it: the streams land host-side as DIR/stdout and
DIR/stderr, written live, owned by the host user, and there when run()
returns. The console stays on stdout for diagnostics; the CLI exposes
the flag as --capture.
KVM runs only the host architecture, so the flag could not mean what
it suggested: a mismatched value downloaded a foreign rootfs that
extracted cleanly and died at boot with a cryptic guest-side exec
error. The flag's real job, picking the image variant to fetch, is
the library's; RunOptions.arch keeps that role and an empty value
still resolves to the host architecture.
The engine materializes a heuristics-driven TSI by default; disable
it and add an explicit vsock hijacking AF_INET so guest networking is
deterministic rather than sniffed. Throughput is unchanged in this
build (TLS crawls at tens of KB/s against a CDN either way) - that is
the new ln driver's own pace and belongs upstream; the flag at least
pins down which path the guest takes.
Copying /etc/resolv.conf verbatim gives the guest systemd-resolved's
stub at 127.0.0.53, and loopback never crosses the engine's socket
proxy: the name lookup dies inside the guest. Source the uplink
config from /run/systemd/resolve first and strip 127.x nameserver
lines from whatever is copied.
The engine proxies guest sockets to the host's stack, so the host's
resolv.conf works in the machine as-is, loopback stub included. The
images ship no usable resolver, so without this every fetch inside
the guest fails to resolve.
RunOptions carried quiet/verbose, which are flags, not engine state.
They are now a LogLevel enum (Quiet/Normal/Verbose) mapped onto
libkrun's filter; the CLI maps its flags onto it. The forked child's
last-gasp boot diagnostics also stop going straight to stderr: they
land in the console file the parent streams, with stderr kept for the
case where the console was never configured. lib.rs now states the
crate contract: no UI in the library, progress through the sink,
logging through the facade, typed errors out.
'ecrou run alpine -- "apk add x && y"' reached the guest as one argv
element and execvp'd a program named after the whole line. One
argument containing whitespace is a quoted shell line: hand it to the
image's shell as 'sh -c'. Argv vectors stay literal, including ones
that begin with sh -c.
The exit-code contract needed a rethink once the real libkrun was in
the loop: on a normal payload exit the VMM ends its own process with
the payload's code, so run() forks and the child enters the VM loop.
The parent reaps it and the code flows to the consumer; the control
file stays as the boot-failure diagnostic, and the guest script no
longer powers off - the init blob does that when the script exits,
since the libkrunfw kernel has no sysrq.
The console cannot ride stdio: with a tty on fd 0 libkrun stops a
background caller with SIGTTOU, and without one it routes guest bytes
through its logger. So the console goes to a file in the scratch that
the parent streams to real stdout, with a final drain after the reap;
the child unwires fd 0 to /dev/null so the terminal is never touched
and a Ctrl-C to the process group still reaches the guest. PDEATHSIG
keeps T4: a killed consumer is a killed machine. The hostname and
workdir now ride the set_exec config natively (the init blob applies
them); log level follows quiet/verbose.
Also in the CLI: the command after '--' was parsed but never copied
into RunOptions, so every run executed the image's default shell.
run maps the flags onto RunOptions, promotes the first read-write share
to the workspace, inherits stdio, and exits with the guest's code; probe
reports what a run would find (KVM, libkrun loadability with the
remediation, nested virt, cache writability) and fails when a run
would. T6, the no-mount audit, lives here where the binary is built,
ignored behind ECROU_BIN for CI.
run() probes KVM, prepares the rootfs through the cache, generates the
guest init into the scratch, wires a libkrun context to it, and blocks
in krun_start_enter on this thread; the exit code comes back through
the control file, and a killed consumer kills the machine with the
process. Shares are validated and tagged in one place so the script and
the virtiofs setup stay ordered; anything overlapping the scratch
rootfs is refused, since the guest could otherwise rewrite its own
init. The guest environment is composed from scratch; the host
environment never crosses.
Generated into the scratch rootfs before boot: mounts each virtiofs
share (ro through the mount option), runs the payload as a child so the
init survives to write its exit code to the control file, then powers
off through sysrq, which works without a poweroff binary in the rootfs.
Shell quoting keeps payload argv and share paths free of interpolation.
Eleven calls from the stable-1.19.x headers plus krun_free_ctx, loaded
at runtime so a missing install names the remediation instead of failing
to link. Symbol lookup goes through a macro that carries the signature
from libkrun.h, every unsafe block states its invariant, and negative
returns map onto errno errors. The KVM probe ports ecr's open-then-
KVM_GET_API_VERSION check and reports KvmUnavailable with the full
remediation text.
ecrou's workspace starts from ecr's confinement-neutral half: image
reference resolution, the content cache with digest freshness, layer
extraction with whiteouts and deferred directory modes, provisioned
hot cells, and the progress/error/config plumbing. Adaptations: the
cache lives in ~/.cache/ecrou, the namespace-era error variants and
utils constants are gone, KvmUnavailable arrives ahead of its caller,
rootfs loses its exec() delegation, and uname(2) goes through libc
now that nix is no longer a dependency. The CLI crate is a stub until
its commit.
QEMU was the committed v1 engine with libkrun later; that order is
flipped. libkrun deletes the initramfs pack, the kernel fetch, slirp
config, and the uid mapping problem, and its VMM runs in-process, which
collapses the kill chain. Verified against the stable-1.19.x headers:
eleven-call v1 surface, krun_set_workdir native, mapped volumes dead on
Linux (tagged virtiofs plus guest init mount instead), KVM required
with QEMU as the later TCG fallback. Snap spike now stages libkrun
plus libkrunfw instead of qemu-system.
ecrou descends from ecr: the microVM runtime that survives strict snap
confinement. The founding rule (no host mounts, no namespaces, every
crossing a file or a socket) is stated with its syscall budget, sourced
from snapd's seccomp allowlist; execution model, exit-code propagation
through a private control share, kill-chain cancellation, the spike-snap
gate, and seven falsification tests round out the v1 design.
vhaudiquet
Valentin Haudiquet