docs: document incremental builds and the session design
README: the build-session workflow (auto-recording, --resume, 'pkh deb list', --keep, prune retention) under a new section. plans/: the design spec with the decisions taken along the way (opt-in resume, --keep for the iteration loop, /var/tmp/pkh/ sessions, Ctrl+C keeping the session) and the implementation notes recording the as-built deviations.
This commit is contained in:
@@ -111,6 +111,28 @@ pkh put --ppa user/hello_xxx
|
|||||||
git push xxx user-fork
|
git push xxx user-fork
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Incremental builds (build sessions)
|
||||||
|
|
||||||
|
Every `pkh deb` build records a **session** under `/var/tmp/pkh/sessions`:
|
||||||
|
the bootstrapped chroot, the installed build dependencies and the build
|
||||||
|
artifacts of the staged tree. When a build fails (or is interrupted), the
|
||||||
|
session is kept and can be resumed:
|
||||||
|
|
||||||
|
```
|
||||||
|
pkh deb # fails after 25 minutes
|
||||||
|
pkh deb --resume # reuses the chroot, build deps and objects;
|
||||||
|
# only what changed is recompiled
|
||||||
|
pkh deb list # the sessions of this tree, with their ids
|
||||||
|
pkh deb --resume <id> # resume a specific session
|
||||||
|
pkh deb --keep # keep the session even after a successful build
|
||||||
|
# (iterate: edit, `pkh deb --resume --keep`, ...)
|
||||||
|
pkh prune # garbage-collect old sessions (7-day retention)
|
||||||
|
```
|
||||||
|
|
||||||
|
A plain `pkh deb` never reuses a session — everything is rechecked from
|
||||||
|
scratch — and it replaces the session of its target. `pkh deb --resume`
|
||||||
|
refuses to adopt a session built for a different series/architecture.
|
||||||
|
|
||||||
## Future improvement ideas
|
## Future improvement ideas
|
||||||
|
|
||||||
- pull: try to fetch the correct git branch for series on Debian
|
- pull: try to fetch the correct git branch for series on Debian
|
||||||
|
|||||||
@@ -0,0 +1,429 @@
|
|||||||
|
# `pkh deb` Incremental Builds — Spec
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`pkh deb` has no incremental story. Every invocation — including a rerun
|
||||||
|
seconds after a failed build — pays the full pipeline again:
|
||||||
|
|
||||||
|
1. mmdebstrap tarball **extraction** into a fresh `/tmp/pkh-<ts>` chroot
|
||||||
|
(the tarball itself is cached, but extraction of a kernel-sized chroot
|
||||||
|
is minutes),
|
||||||
|
2. device nodes + `/proc` bind mount,
|
||||||
|
3. `apt-get update`,
|
||||||
|
4. essentials install,
|
||||||
|
5. `quilt push -a`,
|
||||||
|
6. native build-dependency resolution + install,
|
||||||
|
7. `debian/rules build` **from scratch** in a freshly staged copy of the
|
||||||
|
tree,
|
||||||
|
8. `fakeroot debian/rules binary`.
|
||||||
|
|
||||||
|
The failures this hurts most are the late ones. A kernel package that
|
||||||
|
dies in `debian/rules build` after 25 minutes is restarted from zero;
|
||||||
|
fixing one compile error and rerunning recompiles everything, and a
|
||||||
|
cycle of "tweak, rebuild, fail again" costs a full pipeline each time.
|
||||||
|
|
||||||
|
The raw material for resumption already exists but is unreachable:
|
||||||
|
|
||||||
|
- **Failed chroots are kept.** `EphemeralContextGuard::drop`
|
||||||
|
(`src/deb/ephemeral.rs`) deliberately keeps the chroot when the build
|
||||||
|
did not succeed — but the next run creates a new `pkh-<ts>` directory
|
||||||
|
and cannot find or reuse the old one. The kept chroot is a leak until
|
||||||
|
`pkh prune`, not a cache.
|
||||||
|
- **The staged tree is an overlayfs mount** (`src/context/unshare.rs`):
|
||||||
|
host tree = lowerdir (read live), build writes = upperdir inside the
|
||||||
|
chroot. If the upperdir survives between runs, object files survive
|
||||||
|
too, and host-side edits propagate automatically — exactly the
|
||||||
|
semantics incremental builds need.
|
||||||
|
- **Ctrl+C deletes everything** (`sigint_cleanup_chroot`), which is the
|
||||||
|
worst behavior for a long build interrupted on purpose.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
`pkh deb --resume` after a failed build resumes from the point of
|
||||||
|
failure (plain `pkh deb` always rebuilds from scratch — see §5):
|
||||||
|
|
||||||
|
- same environment, source untouched → picks up at
|
||||||
|
`debian/rules build`, recompiling nothing (make sees a warm tree);
|
||||||
|
- source modified since the failure → still reuses chroot, apt state
|
||||||
|
and installed build-deps; `debian/rules build` recompiles only what
|
||||||
|
the modification affects;
|
||||||
|
- `debian/control` or patch series changed → environment-level state
|
||||||
|
(build-deps, quilt) is redone; the chroot and object files still
|
||||||
|
carry over when possible.
|
||||||
|
|
||||||
|
Non-goal: distributing or sharing sessions between machines; resuming
|
||||||
|
`pkh build` (the native source pipeline — it is seconds-fast already);
|
||||||
|
resuming inside ssh/schroot contexts (sessions are a local-unshare
|
||||||
|
feature; other drivers keep today's behavior).
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### 1. Build sessions: named, journaled, discoverable
|
||||||
|
|
||||||
|
A **session** is one (package, series, arch, cross) build environment
|
||||||
|
plus its journal. Replace the anonymous `pkh-<timestamp>` chroot
|
||||||
|
directory with a session root:
|
||||||
|
|
||||||
|
```
|
||||||
|
/var/tmp/pkh/sessions/<slug>/
|
||||||
|
session.json # manifest + phase journal (see below)
|
||||||
|
chroot/ # the chroot tree (was /tmp/pkh-<ts>)
|
||||||
|
tree-upper/ # overlayfs upperdir for the staged package tree
|
||||||
|
pkh-overlay/ # overlay workdirs (as today)
|
||||||
|
```
|
||||||
|
|
||||||
|
`<slug>` is derived from the session identity (package, series, arch,
|
||||||
|
cross) plus a short content hash, so a rerun can find a previous
|
||||||
|
session by identity without guessing timestamps. One live session per
|
||||||
|
identity — the newest attempt *replaces* the previous one (same
|
||||||
|
chroot environment, fresher state; keeping per-attempt history would
|
||||||
|
mean one full chroot per attempt, which is a cache nobody wants). A
|
||||||
|
locked session (concurrent build) is never replaced (§6). What the
|
||||||
|
session list shows for a tree is therefore one entry per identity
|
||||||
|
(series/arch/cross combination), each carrying its latest attempt's
|
||||||
|
id.
|
||||||
|
|
||||||
|
Moving off `/tmp` is deliberate: sessions must survive a reboot to be
|
||||||
|
worth keeping, and `/tmp` is often tmpfs (a kernel chroot on tmpfs is
|
||||||
|
RAM). `/var/tmp` is the conventional persistent-scratch location; the
|
||||||
|
base directory is overridable (`PKH_SESSIONS_DIR` env, then config).
|
||||||
|
Sessions are a cache, and `pkh prune` is their primary GC — if
|
||||||
|
systemd-tmpfiles sweeps `/var/tmp` on some setup, that is an
|
||||||
|
acceptable, if blunt, secondary cleanup.
|
||||||
|
|
||||||
|
`session.json` records:
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{
|
||||||
|
"id": "20260926T143505", // build-start timestamp, UTC; changes on every attempt
|
||||||
|
"host_tree": "/home/me/linux", // tree the session was built from (list scoping)
|
||||||
|
"identity": {
|
||||||
|
"package": "linux", "series": "stonking",
|
||||||
|
"arch": "arm64", "cross": true
|
||||||
|
},
|
||||||
|
"created": "...", "last_used": "...",
|
||||||
|
"chroot": {
|
||||||
|
"tarball": "stonking-arm64-buildd.tar.xz",
|
||||||
|
"tarball_sha256": "...", // reuse only if the cached tarball is the same
|
||||||
|
"ready": true // device nodes + /proc done
|
||||||
|
},
|
||||||
|
"phases": {
|
||||||
|
"apt_update": { "at": "..." }, // journal-only: apt update always reruns on resume
|
||||||
|
"essentials": { "at": "..." },
|
||||||
|
"patches": { "stamp": "<hash of debian/patches>", "applied": true },
|
||||||
|
"build_deps": { "stamp": "<hash of control+arch+cross+ppa+inject+pocket>", "at": "..." },
|
||||||
|
"build": { "at": "...", "result": "failed" }
|
||||||
|
},
|
||||||
|
"tree": {
|
||||||
|
"version": "7.3.0-5.6~local2", // changelog version of the last attempt
|
||||||
|
"host_snapshot": "<path or inline list>", // for deletion propagation
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The journal is written **after** each phase completes (and after a
|
||||||
|
phase fails, recording the failure), so a crash or Ctrl+C at any point
|
||||||
|
leaves a truthful journal. Discovery is: scan
|
||||||
|
`$PKH_SESSIONS_DIR/*/session.json`, match identity.
|
||||||
|
|
||||||
|
### 2. Two layers of resume
|
||||||
|
|
||||||
|
The journal separates **environment resume** (expensive, insensitive to
|
||||||
|
source edits) from **build resume** (sensitive to every edit):
|
||||||
|
|
||||||
|
| Phase | Reused when | Redone when |
|
||||||
|
|---|---|---|
|
||||||
|
| Chroot extract, nodes, /proc | chroot present, `ready`, same tarball hash | never for the same identity (otherwise the session is unusable) |
|
||||||
|
| `apt-get update` | — (always rerun on resume) | — |
|
||||||
|
| Essentials | done (verify cheaply via `dpkg-query` at most) | chroot redone |
|
||||||
|
| quilt push | `debian/patches` hash unchanged | hash changed → `quilt pop -a` + re-push |
|
||||||
|
| Build-deps | control/arch/cross/ppa/inject/pocket stamp unchanged | stamp changed → the resolver runs again (it is idempotent; apt installs only what is missing) |
|
||||||
|
| `debian/rules build` | always *attempted* in the warm tree | — |
|
||||||
|
| `fakeroot debian/rules binary` | always rerun | — |
|
||||||
|
|
||||||
|
Key consequences:
|
||||||
|
|
||||||
|
- **`apt-get update` is always rerun on resume.** It is the cheapest
|
||||||
|
phase, and stale package lists fail confusingly ("candidate version
|
||||||
|
not found") on fast-moving suites — there is nothing worth caching
|
||||||
|
here. The journal entry exists for observability only.
|
||||||
|
|
||||||
|
- **A changelog version bump** (`pkh chlog` between attempts) invalidates
|
||||||
|
build resume but not environment resume: the chroot, apt lists and
|
||||||
|
build-deps carry over. The staged tree is re-populated for the new
|
||||||
|
version (see §3); the session's `tree.version` tracks which attempt
|
||||||
|
the upperdir belongs to, and a version change discards the upperdir
|
||||||
|
(object files reference paths/flags of the old version).
|
||||||
|
- **The build phase is never skipped**, only made incremental — make
|
||||||
|
decides. pkh's job is to hand `debian/rules build` a tree whose
|
||||||
|
previously produced objects are still there and whose sources match
|
||||||
|
the host.
|
||||||
|
- Before rerunning the binary phase on a resume, clear
|
||||||
|
`debian/files` (and `debian/*.log`, `debian/substvars`) from the
|
||||||
|
staged tree so artifact collection (`collect_binary_artifacts`) only
|
||||||
|
sees the new attempt's outputs — today's "no globbing stale files"
|
||||||
|
contract is preserved.
|
||||||
|
|
||||||
|
### 3. Tree propagation on resume
|
||||||
|
|
||||||
|
Today each build stages a fresh copy/overlay of the host tree. On
|
||||||
|
resume the staged tree already exists inside the session; the problem
|
||||||
|
is syncing it with a possibly-modified host tree.
|
||||||
|
|
||||||
|
- **Overlay contexts (the default when overlayfs is available):** remount
|
||||||
|
the overlay with the *same* upperdir. Host modifications are visible
|
||||||
|
immediately (lowerdir is read live); objects written in the upperdir
|
||||||
|
during the previous attempt are still on top. Host-side *deletions*
|
||||||
|
are the one gap — overlayfs only hides a lower file via a whiteout in
|
||||||
|
the upper, and nothing creates whiteouts for files the user deleted
|
||||||
|
on the host. Fix: the session stores a snapshot of the host tree's
|
||||||
|
file list at first staging; on resume, files present in the upper
|
||||||
|
(or in the snapshot) but gone from the host get an explicit whiteout
|
||||||
|
(`mknod <path> c 0 0` inside the upper). This is a bounded,
|
||||||
|
manifest-driven operation, not a tree walk of the whole chroot.
|
||||||
|
- **Copy fallback (no overlayfs):** `rsync -a --delete` of the host
|
||||||
|
tree over the staged tree, *excluding* the build artifacts make
|
||||||
|
produced (this needs per-package exclusion knowledge and is best
|
||||||
|
effort). Deletions propagate naturally here. Sessions on the copy
|
||||||
|
path get build resume only for trees where the exclusion set is
|
||||||
|
sane; the honest fallback is: copy path → environment resume only.
|
||||||
|
|
||||||
|
The overlay logic stays inside the unshare driver, matching the
|
||||||
|
principle established in `plans/overlayfs-integration.md`: callers
|
||||||
|
stage a tree through `ensure_available()` and never learn how.
|
||||||
|
|
||||||
|
### 4. Interrupt behavior
|
||||||
|
|
||||||
|
Ctrl+C currently removes the chroot through the interrupt hook. Change:
|
||||||
|
**an interrupted build keeps the session**, journaling the interrupted
|
||||||
|
phase as incomplete — an intentional Ctrl+C on a 30-minute build is a
|
||||||
|
"pause", and deleting the environment on pause defeats the feature.
|
||||||
|
|
||||||
|
The cleanup hook (`sigint_cleanup_chroot`) becomes session-aware: it
|
||||||
|
still unmounts the overlays and `/proc` (leaving mounted state in a
|
||||||
|
session is what makes stale sessions dangerous), but keeps the tree
|
||||||
|
and leaves a `interrupted: true` marker in the journal so the next run
|
||||||
|
knows the session needs no special recovery (phases after the last
|
||||||
|
completed one simply rerun).
|
||||||
|
|
||||||
|
Users who want the old hard-discard behavior on interrupt get it via
|
||||||
|
`pkh prune`. The failure path is unchanged in spirit: failed builds
|
||||||
|
already keep their chroot; they now keep a *usable* one.
|
||||||
|
|
||||||
|
### 5. CLI surface and policy
|
||||||
|
|
||||||
|
**Recording is always on; reuse is opt-in.** The two are deliberately
|
||||||
|
separated, because the first attempt cannot know it will fail: a
|
||||||
|
plain `pkh deb` writes and maintains the session (journal, kept
|
||||||
|
chroot on failure or interrupt — today's keep-on-failure, made
|
||||||
|
usable) but never reads one. Only `pkh deb --resume` adopts an
|
||||||
|
existing session. The default run therefore always rechecks
|
||||||
|
everything from scratch — no stale-build risk unless the user asks
|
||||||
|
for one.
|
||||||
|
|
||||||
|
- `pkh deb` — today's behavior, plus session recording: any existing
|
||||||
|
session of the same identity is ignored and replaced by this run's
|
||||||
|
outcome. On failure or interrupt the session is kept (as today);
|
||||||
|
on success it is **removed**, unless `--keep` is given.
|
||||||
|
- `pkh deb --keep` — also keep the session after a successful build,
|
||||||
|
so a later `--resume` can iterate on it (the edit–rebuild loop).
|
||||||
|
Composes with `--resume`: `--resume --keep` chains sessions across
|
||||||
|
iterations; a resumed build without `--keep` consumes the session
|
||||||
|
on success — the natural end of the loop.
|
||||||
|
- `pkh deb --resume [<id>]` — adopt a session and continue it,
|
||||||
|
announcing what is being skipped ("Resuming session
|
||||||
|
20260926T143505 for linux/stonking-arm64: chroot, package lists
|
||||||
|
and build-deps reused").
|
||||||
|
- Without an id: the newest session recorded from this tree
|
||||||
|
(manifest `host_tree` matches the tree being built), whatever its
|
||||||
|
last outcome — failure, interrupt, or a `--keep` success.
|
||||||
|
- With an id (a build-start timestamp as shown by `pkh deb list`):
|
||||||
|
that exact session. A unique unambiguous prefix is accepted; an
|
||||||
|
ambiguous one is an error listing the candidates.
|
||||||
|
- Selector conflicts are errors, not surprises: if `-a`/`-s`/
|
||||||
|
`--cross`/`--ppa`/`--inject` are given explicitly and disagree
|
||||||
|
with the adopted session, refuse and point at `pkh deb list`. A
|
||||||
|
session for another series/arch is a *different* build
|
||||||
|
environment; silently building it under the requested selectors
|
||||||
|
would be exactly the stale-build risk this design avoids.
|
||||||
|
- With no session to adopt (none recorded, pruned, or the tree
|
||||||
|
moved): say so and build from scratch — never silently.
|
||||||
|
- `pkh deb list` — a subcommand, not a flag: `pkh deb` with flags
|
||||||
|
always means "start a build", and listing is a different action.
|
||||||
|
It prints the sessions recorded from the current tree, one row per
|
||||||
|
identity (§1), newest attempt first:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ pkh deb list
|
||||||
|
Sessions for /home/me/linux:
|
||||||
|
ID PACKAGE VERSION TARGET LAST RUN AGE
|
||||||
|
20260926T143505 linux 7.3.0-5.6~local2 stonking/arm64 failed 2h
|
||||||
|
20260925T090012 linux 7.3.0-5.5~local1 stonking/arm64* success 1d
|
||||||
|
20260924T171100 linux 7.3.0-5.6~local2 stonking/riscv64* failed 2d
|
||||||
|
|
||||||
|
* cross build. Resume with: pkh deb --resume [<id>]
|
||||||
|
```
|
||||||
|
|
||||||
|
`pkh deb list` never touches build machinery: it reads the session
|
||||||
|
manifests under the sessions directory and renders. As an optional
|
||||||
|
subcommand of `deb` (clap allows subcommands alongside args), the
|
||||||
|
build flags stay on `deb` itself and `list` takes none of them.
|
||||||
|
- `pkh prune` learns sessions: by default removes
|
||||||
|
sessions untouched for longer than a retention window (7 days) and
|
||||||
|
any session whose journal is missing/corrupt; `--all` removes every
|
||||||
|
session. Prune's report distinguishes "session (resumable build)"
|
||||||
|
from the other residual categories, since sessions are large and
|
||||||
|
users must be able to see and reclaim them.
|
||||||
|
|
||||||
|
### 6. Correctness guards
|
||||||
|
|
||||||
|
A session is only reused when **all** of these hold; otherwise it is
|
||||||
|
discarded (replaced) with a logged reason:
|
||||||
|
|
||||||
|
- identity matches (package, series, arch, cross);
|
||||||
|
- the cached chroot tarball hash matches the one the chroot was built
|
||||||
|
from;
|
||||||
|
- `chroot/` passes an integrity probe (marker file, `chroot/bin/sh`
|
||||||
|
exists and executes);
|
||||||
|
- pkh version compatibility: `session.json` records the pkh version
|
||||||
|
and manifest schema version; a mismatched schema discards the
|
||||||
|
session (the chroot is generic enough that only schema-relevant
|
||||||
|
changes matter, but being conservative is cheap).
|
||||||
|
- no other live session for the same identity (lock file in the
|
||||||
|
session root, in the spirit of the existing tarball lockfile).
|
||||||
|
|
||||||
|
Concurrent builds of the same identity take the lock and fall back to
|
||||||
|
a fresh ephemeral session (current behavior) rather than waiting —
|
||||||
|
correctness over caching.
|
||||||
|
|
||||||
|
One more guard on the resume path, at the CLI layer: explicit
|
||||||
|
selectors (`-a`, `-s`, `--cross`, `--ppa`, `--inject`) that disagree
|
||||||
|
with the adopted session refuse the build (§5) — the session is only
|
||||||
|
ever reused under the selectors it was created with.
|
||||||
|
|
||||||
|
## Implementation sketch
|
||||||
|
|
||||||
|
New `src/deb/session.rs` — session identity, manifest
|
||||||
|
(serde) read/write, discovery, locking, slug generation. Pure logic,
|
||||||
|
fully unit-testable.
|
||||||
|
|
||||||
|
- `src/deb/ephemeral.rs` — `EphemeralContextGuard` learns a `resume`
|
||||||
|
path: when a session is adopted, skip download/extract and reuse the
|
||||||
|
existing chroot (still verifying `ready` and remounting `/proc` if a
|
||||||
|
previous run unmounted it); on teardown, hand the tree to the
|
||||||
|
session instead of `rm -rf`-ing it. Chroot dir moves from
|
||||||
|
`create_temp_dir()` to the session layout — the local driver's temp
|
||||||
|
naming stays for non-session uses.
|
||||||
|
- `src/deb/local.rs` — journal writes around each phase; the
|
||||||
|
skip-or-run decisions per the table in §2; `debian/files` cleanup
|
||||||
|
before the binary phase on resume.
|
||||||
|
- `src/context/unshare.rs` — upperdir reuse + whiteout propagation for
|
||||||
|
host deletions (§3); `ensure_available()` gains a
|
||||||
|
"stage into existing session tree" mode.
|
||||||
|
- `src/interrupt.rs` + `src/deb/ephemeral.rs` — the interrupt hook
|
||||||
|
keeps the session (§4).
|
||||||
|
- `src/prune.rs` — session discovery, retention, reporting.
|
||||||
|
- `src/main.rs` — `--keep`, `--resume [<id>]` argument wiring,
|
||||||
|
the `pkh deb list` subcommand and selector-conflict checks;
|
||||||
|
`README.md` roadmap
|
||||||
|
update (this retires part of the "deb: asynchronous build,
|
||||||
|
detachable and monitorable" itch: a persistent session is also the
|
||||||
|
natural attach point for detached builds later).
|
||||||
|
|
||||||
|
Staged rollout, each step independently shippable:
|
||||||
|
|
||||||
|
1. Sessions with manifest + `pkh deb list` + adoption of failed-build
|
||||||
|
chroots via `--resume` (environment resume only — biggest win, no
|
||||||
|
overlay changes).
|
||||||
|
2. Build resume: upperdir reuse + whiteout propagation.
|
||||||
|
3. Interrupt keeps session; `--keep` after success; prune integration.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
- Unit: manifest round-trip, slug/identity matching, whiteout plan
|
||||||
|
computation (host snapshot vs current host listing), skip-decision
|
||||||
|
table given journals with various stamps.
|
||||||
|
- Integration (small fixture package, `#[cfg(test)]` style already
|
||||||
|
used by `deb` tests): fail `debian/rules build` on file B after
|
||||||
|
compiling file A; resume and assert A is *not* recompiled (make log
|
||||||
|
via the tee log); modify A; resume and assert only A recompiles;
|
||||||
|
modify `debian/control`; assert build-deps rerun and objects
|
||||||
|
survive; `pkh prune` GC behavior; a plain `pkh deb` ignoring an
|
||||||
|
existing session (fresh rebuild) while `--resume` adopts it.
|
||||||
|
- Unit/CLI: `pkh deb list` rendering against fixture manifests (scoping by
|
||||||
|
`host_tree`, cross marker, ordering); `--resume <id>` exact and
|
||||||
|
prefix resolution, ambiguity error; selector-conflict refusals;
|
||||||
|
`--keep` keeping the session on success while a plain build removes
|
||||||
|
it.
|
||||||
|
- Chroot reuse tests need the unshare/mount machinery — follow the
|
||||||
|
existing pattern of `#[ignore]` tests for deliberate ad-hoc runs
|
||||||
|
where root/unshare is required.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
- **Resume is opt-in (`--resume`), never the silent default.** A
|
||||||
|
plain `pkh deb` always rechecks everything from scratch, so the
|
||||||
|
no-flag behavior can never build anything stale; recording stays on
|
||||||
|
by default so the follow-up `--resume` after a failure has a
|
||||||
|
session to adopt.
|
||||||
|
- **Keeping a session after success is opt-in (`--keep`).** Today's
|
||||||
|
disk profile — success cleans up — stays the default; the
|
||||||
|
edit–rebuild iteration loop is served by `pkh deb --resume --keep`.
|
||||||
|
Failures and interrupts keep their session unconditionally, as
|
||||||
|
today.
|
||||||
|
- **Sessions are addressed by id (build-start timestamp), listed via
|
||||||
|
the `pkh deb list` subcommand.** Listing is a subcommand, not a
|
||||||
|
mode-switching flag: `pkh deb` with flags always starts a build.
|
||||||
|
One live session per identity keeps the cache
|
||||||
|
bounded (a per-attempt history would be one full chroot per
|
||||||
|
attempt); the list therefore shows one row per series/arch/cross
|
||||||
|
combination for the current tree, each carrying its latest attempt's
|
||||||
|
id. Explicit selectors that disagree with the adopted session are
|
||||||
|
an error, never a silent environment switch.
|
||||||
|
- **Ctrl+C keeps the session**, same as a failed build: an interrupt
|
||||||
|
on a long build is a pause, and there is nothing inherent in "stop
|
||||||
|
now" that means "discard half an hour of environment setup". The
|
||||||
|
hard-discard behavior stays reachable through `pkh prune`.
|
||||||
|
- **`apt-get update` always reruns on resume** (no stamp, no TTL) —
|
||||||
|
see §2.
|
||||||
|
- **Sessions live under `/var/tmp/pkh/sessions`** (overridable via
|
||||||
|
`PKH_SESSIONS_DIR` then config), and are cleaned by `pkh prune`
|
||||||
|
(retention window by default, `--all` for everything). Cleanup by
|
||||||
|
systemd-tmpfiles from time to time is accepted; sessions are a
|
||||||
|
cache.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Implementation Notes (as built)
|
||||||
|
|
||||||
|
Implemented across `src/deb/session.rs` (new), `src/deb/{mod,local,
|
||||||
|
ephemeral}.rs`, `src/context/{api,unshare}.rs`, `src/prune.rs` and
|
||||||
|
`src/main.rs`. Deviations from the design above, all deliberate:
|
||||||
|
|
||||||
|
- **Phase stamps gate less than specced.** The journal records every
|
||||||
|
phase (with the patch-tree and control stamps), but on resume only
|
||||||
|
the expensive non-idempotent work is actually skipped: the chroot
|
||||||
|
bootstrap (extraction, device nodes, `/proc`) and, via the reused
|
||||||
|
overlay upperdir, the compiled objects. `apt-get update`, the
|
||||||
|
essentials install, the quilt push and the build-dep resolution
|
||||||
|
always rerun — each is idempotent and seconds-cheap, and rerunning
|
||||||
|
them removes a whole class of stale-state bugs. The stamps still
|
||||||
|
drive the quilt `pop -a` decision (patch tree changed since the
|
||||||
|
recorded attempt) and the version-change handling.
|
||||||
|
- **The advisory lock lives outside the session root**
|
||||||
|
(`<sessions root>/<slug>.lock`): a teardown deletes the root while
|
||||||
|
still holding the lock, and a lock file inside it would be removed
|
||||||
|
under the holder — a concurrent opener would then create and lock a
|
||||||
|
fresh inode, and mutual exclusion silently dies (observed with two
|
||||||
|
concurrent same-identity e2e builds before the move).
|
||||||
|
- **`pkh prune` session GC is confined to the production `prune()`.**
|
||||||
|
`prune_in()` (the testable core) deliberately does not scan the real
|
||||||
|
sessions root: the test suite runs prune tests and e2e builds
|
||||||
|
concurrently, and the prune tests would delete live sessions
|
||||||
|
mid-build (observed). `prune_in_roots()` takes the sessions root as
|
||||||
|
an explicit opt-in parameter.
|
||||||
|
- **Fallbacks, in order:** no local base context → the historical
|
||||||
|
anonymous temp chroot (no session); session locked by a concurrent
|
||||||
|
build → same; overlay mount unsupported/failed on resume → fresh
|
||||||
|
copy staging (environment resume only, artifacts discarded); no
|
||||||
|
host-tree snapshot → upper wiped and re-snapshotted.
|
||||||
Reference in New Issue
Block a user