The size cap was the wrong shape: an --incremental user asked for a
carried tree, and the invariant that matters is which trees survive —
the newest successful one (the next run's automatic fork base) and the
newest one (what a failed run is re-forked from explicitly). The cap
is gone, with the tree measurement and manifest field it needed; the
retention planner now works on the records' outcomes alone and fills a
budget: staged_trees staged trees per target, the newer ones first,
the pair always among them. When the budget cannot hold the pair (one
staged tree configured, the newest build failed and an older
successful tree present) only one of the two is successful and that
one is kept; with no successful tree at all, the most recent one is.
The budget is a setting of the nested config file:
deb:
incremental:
staged_trees: 2
(default 2; 0 clamps to 1 with a warning — the newest successful tree
always survives). The fork-base selection no longer truncates the
candidate list to the budget before ranking it: with a budget of one
and a failed newest tree, that truncation hid the successful record
and forked the failed one instead.
The retention kept the newest five staged trees per target, so the
staged trees of a kernel-scale target alone could hold three-digit
GiB. The policy is now the pair that earns its disk: the newest
staged tree (what a failed run is re-forked from explicitly) and the
newest successful one (what the next --incremental run forks
automatically) — one record when they coincide; every other staged
tree degrades to metadata-only at the next build's retention. The
retention decision moved into a pure planner over (success, size) so
the policy is table-testable.
The size cap no longer drops a tree outright: an --incremental run's
own tree is always kept by its own run — the user asked for a carried
tree, and at least one always exists — and the cap only thins the kept
pair while another member covers the resume, the newest successful
tree being the last one standing. Oversized trees left by older pkh
versions or interrupted builds come down at the next build's
retention, once their lock is free.
The cap moves to the first user settings file, config.yml in
~/.config/pkh (YAML, like the data files; missing file or keys mean
defaults, a corrupt file is backed up, PKH_STAGED_MAXSIZE overrides),
replacing the JSON sketch from the previous commit. Default 20G, 0
disables the size rule. Every retained tree's size is measured once at
retention and recorded in the manifest, so sweeps never re-walk the
trees.
Confirming a 500 GiB prune went silent under "Removing:" for minutes:
drive an indicatif bar from the plan's sizes through a new
apply_with_progress observer, in the deb live view's layout — the
running count behind the spinner on the first line, the byte bar with
its counters on the second. The removed path is not rendered: it
fights the bar on a narrow terminal, and the failed-removal errors
name the paths that matter. --force plans no sizes and gets the count
line only.
The dry-run size spinner lacked the "> " prompt prefix the other
one-line waits carry; wait_spinner renders it too now.
The incremental fork's staged-tree copy wants to report byte progress
against the same measurements the prune report is built from: move the
concurrent size walk and the human-readable byte formatting out of
prune into utils::size and rewire prune to them. Behavior is unchanged:
same-device walks, unreadable entries count as zero.
- -f/--force skips the size pass and the confirmation prompt: remove
immediately, reporting counts but no sizes. The fastest cleanup
path; --dry-run still previews.
- The default path shows the grouped digest and asks before removing
(the raw-mode yes/no prompt; non-interactive sessions keep their
headless path, so scripts reach for --force).
- Sizes are computed under a spinner ("Computing sizes of
directories to prune..."), and the digest colors the counts (bold)
and the sizes (cyan) on terminals only.
The dry run still spammed one info line per artifact before the
grouped digest, and the size walk was serial: tens of seconds of pure
readdir/stat over the records' hundreds of thousands of files.
The per-path lines drop to debug (the grouped report is the
user-facing output), and the size pass now fans out over a shared
work queue with scoped threads — directory-level parallelism, same
du -x mount-crossing rule. 16s to 6.4s wall on a 440 GiB sessions
tree, totals unchanged.
The prune output spammed one line per path. The report now groups
entries by artifact kind with their total size, and the size is
measured before removal whatever the mode, so --dry-run prints a
digest ("24 residual build directories for 1.2 GiB") and both modes
end with the freed total. The size walk never crosses a mount point:
residual chroots hold bind-mounted /proc trees whose virtual files
(/proc/kcore alone reports 128 TiB) would poison the total.
--all is also honest now: it removes recent build records and staged
trees (that is where the space is), and the flag description says so.
One behavior fix rides along: a record or lock a live build holds was
previously prunable with --all, which would wreck a running build;
held locks are now out of bounds whatever the policy.
With the chroot now per-run scratch, every build re-downloaded the
packages its dependency install needed — the very cost the chroot reuse
used to hide. Every chroot's /var/cache/apt/archives is now a bind
mount over a shared cache at /var/tmp/pkh/apt-cache/<series>-<arch>/
(the chroot's architecture, like the tarball cache), which one run's
installs fill and the next runs reuse: apt keeps downloaded packages
through a dropped conf (APT::Keep-Downloaded-Packages), and an
apt-get autoclean at the end of each run drops the debs the archive no
longer serves. pkh prune removes caches untouched for a week.
The run holds a host-side flock on the cache for its whole lifetime:
apt's own archives lock is acquired inside the chroot's pid namespace,
where a concurrent build's apt shows the holder as "process 0" — an
invisible process it refuses to wait for — so pkh serializes the
traffic itself; concurrent builds of the same target wait for each
other. Any bind-mount failure falls back to the chroot's own cache
with a warning (correct, just slower), and the temp-chroot teardown now
unmounts everything under the chroot before rm -rf, so the removal can
never reach through the cache bind mount and delete the shared cache
itself.
Records carried a full chroot each, and space turned out to be the hard
constraint: five records per target multiplied by a multi-GB chroot does
not fit. The chroot is now per-run scratch — every build extracts a
fresh one from the sha-verified mmdebstrap tarball cache into its
record's chroot/ dir and removes it at the end (success, failure and
interrupt included); resume_chroot, the .pkh-ready probe and
chroot_is_usable are gone, and crashed-run leftovers are swept when the
next run of that identity starts.
What a record keeps instead: manifest.json (renamed, now with the
tarball sha it used, a kept_upper flag and a tree-only journal — the
environment phases rerun in every fresh chroot), a log/build.log full
tee of the build output (kept for EVERY build; plain runs become
metadata-only records, so keep-iff-incremental now applies to the
staged tree alone), and — for --incremental runs — tree-upper/ with its
host-files.list. The fork gate carries a base's staged tree only while
its recorded tarball sha still matches the cache and the changelog
version is unchanged; a fork candidate is any record with a staged tree.
The keep-last-5 cap now applies to staged trees: a sixth retention
degrades the oldest record to metadata-only instead of deleting it.
New `pkh deb log [-f] [<id>]` prints (or follows) a record's build log,
and `pkh deb list` shows the staged-tree state and plain-run records
too. README rewritten around the shrunk model.
The single mutable session per identity loses the last-good state: a
failed run after a good one adopts the session in place and leaves only
the failed state behind, so "continue from the last success" was
impossible. Each build now records an immutable record under
/var/tmp/pkh/sessions/<slug>/<id>/ (chroot, tree-upper, journal,
manifest), and --incremental forks a NEW record — a full copy of the
base's chroot and, at the same changelog version, its staged artifacts
— under the base's flock, never mutating the base itself.
Selection picks the newest successful record of the identity among the
last five usable, falling back to the newest when none succeeded, and
--incremental <id> pins an exact record (failed ones allowed: their
environment may be fine). The lifecycle is strict keep-iff-incremental:
plain runs build in a throwaway record discarded on success, failure
and interrupt alike (the SIGINT cleanup hook follows the same policy),
while --incremental records are always kept. Each identity keeps its
newest five records; pkh prune's 7-day sweep and orphaned-lock cleanup
follow the nested layout.
--resume and --keep are removed (no aliases): their adoption-in-place
and keep-on-success semantics are exactly what the record model
replaces. pkh deb list shows id, outcome, tree version, fork lineage
and age. README rewritten around the record workflow.
Session locks live next to the session root, and failed or
interrupted runs left them behind after the root was gone (seven
stray locks on a used machine). Session removal now unlinks its own
lock file, and prune collects the orphans: by default only the ones
no live build holds (probed with a non-blocking flock), with --all
every one of them. Locks guarding an existing session root are
never touched.
Test artifacts land under /var/tmp/pkh/tests (<source>-<series>-<arch>-
<timestamp>/), one directory per run: they are disposable results, not
state worth keeping. Prune removes them on the same 7-day retention as
build sessions, and with --all; the root is overridable through
PKH_TESTS_DIR like the sessions root.
Sessions accumulate under /var/tmp/pkh/sessions for every failed,
interrupted or --keep build and can reach tens of gigabytes for
kernel-sized chroots: prune is their primary GC. By default remove
the sessions untouched for longer than the retention window (7
days) plus the corrupt leftovers; --all removes everything. Session
roots hold the same mounts as residual chroots (/proc bind mount,
overlays), so they are unmounted before removal.
prune_in() (the testable core) deliberately stays session-free:
the test suite runs prune tests and e2e builds concurrently, and
scanning the real sessions root from the tests deleted live
sessions mid-build. Session pruning is an explicit opt-in of
prune_in_roots(), used by the production prune().
Log retention only matched 'deb-*' logs, so source-build ('build-*')
and placeholder ('pkh-*') logs accumulated forever, and the 'keep the
newest' sort was lexicographic on names that sort by package/version
first, so arbitrary logs were kept. All three log shapes are matched
now and retention orders by the timestamp embedded in the name (mtime
fallback). Stale-lockfile pruning no longer deletes lockfiles younger
than 24h: a fresh <tarball>.lock is the mutual-exclusion signal of a
concurrent download and deleting it could corrupt the shared tarball
cache.
The shared world-writable /tmp/pkh-keyrings directory, combined with the
skip-if-exists logic, let any local user pre-plant keyrings that pkh
then trusts into the chroot's trusted.gpg.d. Use a per-uid 0700
directory instead, refuse to reuse a pre-existing directory that is not
owned by the current user or is group/other-writable, and drop the now
unnecessary world-accessibility chmods (mmdebstrap in unshare mode runs
with the same real uid).
vhaudiquet