docs: document library crate, exec and rootfs cache apis

README gains a Library section covering prepare, the exec options (envp, bind targets, arch) and the provisioned-rootfs hot-cell flow. SPEC documents the workspace layout, the library API, the cache sidecars (.digest, .provisioned) and the updated execution flow. AGENTS.md scope paths follow the new crates/ layout.
2026-09-21 00:08:49 +02:00
parent b6ddd85525
commit af06264e60
3 changed files with 154 additions and 28 deletions
@@ -30,19 +30,26 @@ Rules:

 - Types: `feat`, `fix`, `refactor`, `perf`, `test`, `docs`, `build`,
  `ci`, `chore`.
- Scopes match the component touched — the module name under `src/`:
-  - `cli` — entry point and flags (`src/main.rs`, `src/cli.rs`)
-  - `config` — config file (`src/config.rs`)
-  - `distro` — distro definitions and mirrors (`src/distro.rs`)
-  - `download` — image download (`src/download.rs`)
-  - `extract` — rootfs extraction (`src/extract.rs`)
-  - `chroot` — chroot setup (`src/chroot.rs`)
-  - `namespace` — Linux namespaces (`src/namespace.rs`)
-  - `mount` — bind mounts and mount table (`src/mount.rs`)
-  - `kernel` — kernel/initramfs handling for `--kernel` (`src/kernel.rs`)
-  - `qemu` — QEMU VM mode (`src/qemu.rs`, `src/qemu_vm.rs`)
-  - `utils`, `verbose` — shared helpers (`src/utils.rs`,
-    `src/verbose.rs`)
+- Scopes match the component touched — the module name under
+  `crates/ecr/src/` (library) or `crates/ecr-cli/src/` (CLI):
+  - `cli` — entry point and flags (`crates/ecr-cli/src/main.rs`,
+    `crates/ecr-cli/src/cli.rs`)
+  - `rootfs` — cache-aware preparation and persistence
+    (`crates/ecr/src/rootfs.rs`)
+  - `exec` — namespace-mode execution API (`crates/ecr/src/exec.rs`)
+  - `config` — config file (`crates/ecr/src/config.rs`)
+  - `distro` — distro definitions and mirrors (`crates/ecr/src/distro.rs`)
+  - `download` — image download (`crates/ecr/src/download.rs`)
+  - `extract` — rootfs extraction (`crates/ecr/src/extract.rs`)
+  - `chroot` — chroot setup (`crates/ecr/src/chroot.rs`)
+  - `namespace` — Linux namespaces (`crates/ecr/src/namespace.rs`)
+  - `mount` — bind mounts and mount table (`crates/ecr/src/mount.rs`)
+  - `kernel` — kernel/initramfs handling for `--kernel`
+    (`crates/ecr/src/kernel.rs`)
+  - `qemu` — QEMU VM mode (`crates/ecr/src/qemu.rs`,
+    `crates/ecr/src/qemu_vm.rs`)
+  - `utils`, `verbose` — shared helpers (`crates/ecr/src/utils.rs`,
+    `crates/ecr/src/verbose.rs`)
  - `deps` — dependency additions/bumps (manifests, lockfile)
 - Omit the scope entirely for repo-wide changes that do not belong to a
  single component (README.md, SPEC.md, root config).
@@ -16,6 +16,56 @@ ecr fedora          # any Docker Hub image

 `ecr` pulls a root filesystem (Alpine/Ubuntu direct from their CDNs; everything else from Docker Hub), extracts it into a temporary directory, and execs a shell inside a user + mount + PID + UTS namespace. The process tree is isolated, the rootfs is discarded on exit, and your host is never touched.

+## Library
+
+`ecr` is a cargo workspace: `crates/ecr` is the library crate (package `ecr`), `crates/ecr-cli` the thin command-line front-end. Rust consumers can use the library directly — resolve an image, download it through the content cache, and run commands with a fully caller-controlled environment, bind targets and architecture:
+
+```toml
+[dependencies]
+ecr = { git = "https://github.com/…" } # or a path / registry source
+```
+
+```rust
+use ecr::{chroot, BindTarget, ExecOptions, PrepareRequest, RootfsCache};
+
+// Download alpine through the cache (~/.cache/ecr) and extract to a scratch dir
+let cache = RootfsCache::new(RootfsCache::default_dir()?);
+let rootfs = cache.prepare(&PrepareRequest::new("alpine:3.23"))?;
+
+// Caller-composed envp, explicit bind targets, target architecture
+let code = rootfs.exec(&ExecOptions {
+    arch: "amd64".into(),                                    // empty = host arch
+    binds: vec![BindTarget {                                 // host dir -> in-rootfs mount
+        source: "./data".into(),
+        target: "/mnt/data".into(),
+        read_only: false,                                    // true = overlay (ro)
+    }],
+    env: chroot::default_env(rootfs.dir()),                  // or your own envp
+    command: vec!["cat".into(), "/etc/os-release".into()],
+    ..ExecOptions::default()
+})?;
+
+rootfs.persist()?; // write the rootfs back into the cache (see below)
+```
+
+### Provisioned-rootfs caching (hot cells)
+
+Prepare once, provision once, amortize every later run — including runs from other processes sharing the same cache directory:
+
+```rust
+let rootfs = cache.prepare_provisioned(&PrepareRequest::new("alpine:3.23"), |rootfs| {
+    // runs at most once per cache entry
+    rootfs.exec(&ExecOptions {
+        command: vec!["apk".into(), "add".into(), "curl".into()],
+        env: chroot::default_env(rootfs.dir()),
+        ..ExecOptions::default()
+    })?;
+    Ok(())
+})?;
+```
+
+On the first call the pristine image is downloaded, the closure provisions it, and the result is persisted into the cache with a `.provisioned` marker. On every subsequent call the entry is hit directly: no download, no provisioning. Note a provisioned entry no longer tracks the upstream image; delete the entry (or pass `no_cache: true`) to re-provision.
+
 ## Usage

 ```
@@ -105,7 +155,7 @@ Requirements:

 ## Cache

-Downloaded images are cached in `~/.cache/ecr/`. For `latest` OCI tags the registry manifest digest is checked on each run — the image is only re-downloaded when it has actually changed. Use `--no-cache` to force a fresh pull regardless.
+Downloaded images are cached in `~/.cache/ecr/`. For `latest` OCI tags the registry manifest digest is checked on each run — the image is only re-downloaded when it has actually changed. Use `--no-cache` to force a fresh pull regardless. The library's `PreparedRootfs::persist` can replace a cache entry with a provisioned rootfs (see *Provisioned-rootfs caching* above).

 ## Requirements

@@ -1,5 +1,63 @@
 # ecr - implementation specification

+## Workspace Layout
+
+Cargo workspace with two crates:
+
+- `crates/ecr` — the library (package `ecr`). All functionality lives here:
+  image resolution, download, extraction, namespaces, mounts, chroot exec,
+  QEMU VM mode.
+- `crates/ecr-cli` — the CLI front-end (package `ecr-cli`, binary `ecr`).
+  Parses flags, maps them onto the library API, propagates exit codes.
+
+The library must not depend on CLI types (`clap` is a CLI-only dependency).
+
+## Library API
+
+### Rootfs lifecycle (`ecr::rootfs`)
+
+- `RootfsCache::new(dir)` / `RootfsCache::default_dir()` — the image tarball
+  cache (default `~/.cache/ecr`).
+- `cache.prepare(&PrepareRequest)` — parse the image reference, resolve
+  floating version aliases ("latest", "lts", "edge") to a concrete version
+  before computing the cache key, run the OCI `:latest` digest freshness
+  check, download through the cache when needed, and extract into a scratch
+  directory. Returns a `PreparedRootfs` (TempDir-backed; dropped on drop).
+- `cache.prepare_provisioned(&req, provision)` — hot-cell amortization: on a
+  cache hit with a `.provisioned` sidecar the entry is extracted as-is (no
+  freshness check, no provisioning); otherwise the rootfs is prepared,
+  `provision(&PreparedRootfs)` runs once, and `persist` writes the result
+  back. `no_cache` forces re-download + re-provision.
+- `PreparedRootfs::persist()` — packs the current rootfs into its cache
+  entry (compression chosen from the entry's filename extension: gz, xz,
+  zstd, plain tar; symlinks stored as links; permissions preserved) and
+  writes the `.provisioned` sidecar. The entry's contents are replaced:
+  plain `prepare` also returns the provisioned contents from then on.
+
+### Execution (`ecr::exec`)
+
+- `exec(rootfs, &ExecOptions)` — run a command inside a prepared rootfs in
+  fresh user/PID/mount/UTS namespaces; returns the exit code (128+signal on
+  kill). Empty option fields select defaults:
+  - `arch`: host architecture; a foreign arch requires binfmt_misc.
+  - `env`: caller-composed envp as `(key, value)` pairs; empty selects
+    `chroot::default_env`. The host environment is never inherited.
+  - `binds`: explicit `BindTarget { source, target, read_only }` — host
+    directory mounted at an absolute in-rootfs mount point; read-only via
+    overlay, read-write via bind. `..` targets are rejected.
+  - `dns`: nameservers written to /etc/resolv.conf; empty copies the host
+    resolver.
+  - `command`: argv; empty runs the rootfs default shell.
+  - `working_dir`: explicit in-rootfs cwd; empty picks the first read-write
+    bind target that exists, then `/root`, then `/`.
+
+### Mount plumbing (`ecr::mount`)
+
+`setup_mounts(rootfs, &[BindTarget])` mounts proc/dev/devpts/sys and applies
+the bind targets; `in_rootfs(rootfs, target)` maps in-chroot paths and
+rejects escaping targets. Overlay upper/work directories are temp dirs the
+caller must keep alive (returned by `setup_mounts`).
+
 ## Synopsis

 ```
@@ -36,11 +94,21 @@ ecr [OPTIONS] <DISTRO[:VERSION]> -- [COMMAND]...
 ~/.cache/ecr/
 ├── ubuntu-noble-amd64.tar.gz
 ├── alpine-latest-x86_64.tar.gz
-├── debian-bookworm-amd64.tar.gz
+├── oci-docker_io-library_archlinux-latest-amd64.tar.gz
+├── oci-docker_io-library_archlinux-latest-amd64.tar.gz.digest
 └── ...
 ```

-No metadata files. Tarballs are downloaded once and never redownloaded. Users can delete files manually or use `--no-cache` to fetch fresh.
+Sidecar files, never counted as image entries:
+
+- `<entry>.digest` — manifest digest of the last OCI download, used by the
+  `:latest` freshness check.
+- `<entry>.provisioned` — marker written by `PreparedRootfs::persist`;
+  `prepare_provisioned` treats an entry with this marker as provisioned.
+
+Tarballs are downloaded once and never redownloaded (unless the digest
+moves, `--no-cache` is passed, or a provisioned entry is deleted). Users
+can delete files manually.

 ### Config File

@@ -111,20 +179,20 @@ Error: No manifest found for architecture 'riscv64'. Available: amd64, arm64, pp

 ## Execution Flow

-1. Parse CLI arguments
-2. Resolve distro/version/arch to image source
+The CLI delegates to the library; the namespace-mode flow is:
+
+1. Parse CLI arguments, map flags onto library requests
+2. `cache.prepare`: resolve distro/version/arch to image source
 3. Check cache for existing tarball
 4. If not cached, download tarball (direct or OCI)
-5. Create temp directory for extraction
-6. Extract tarball to temp directory
-7. Create namespaces: user, pid, mount, uts
-8. Set up mounts: /proc, /sys (ro), /dev, /dev/pts
+5. Extract tarball to a temporary directory
+6. `ecr::exec`: create namespaces: user, pid, mount, uts
+7. Set up mounts: /proc, /sys (ro), /dev, /dev/pts
+8. Apply bind targets: overlays (ro) and bind mounts (rw)
 9. Write /etc/resolv.conf with DNS servers
-10. Set up overlay mounts for bind paths
-11. Set up read-write bind mounts
-12. Set environment variables
-13. Exec shell or command in chroot
-14. On exit, clean up temp directory
+10. Set the working directory
+11. Exec shell or command in chroot with the composed envp
+12. On exit, clean up the temporary directory; propagate the exit code

 ## Namespace Setup

@@ -300,7 +368,8 @@ dns:

 ## Environment Variables

-Default environment inside chroot:
+Default environment inside chroot (`chroot::default_env`, used by the CLI;
+library callers compose their own envp):

 - HOME=/root
 - USER=root