A pushed tag only gets Gitea's automatic tree archive: a snapshot
named after the raw tag, unusable as a Debian orig (the upstream
version may not carry a hyphen — 0.2.0-rc1 parses as upstream 0.2.0,
revision rc1, where the Debian encoding of the prerelease is
0.2.0~rc1) and with no release object to hold attachments. A v* tag
now publishes a release with the dpkg-shaped orig tarball (git
archive, prefixed pkh-<version>/), the packed snap and their
checksums; prerelease tags mark the release as such, and an existing
release is reused on a tag re-push. Still nothing published beyond
this repo's release page.
The audit of the lock against Debian's archive (qa.debian.org
madison, sid/trixie/bookworm) showed 255 of 291 crates packaged, the
36 missing ones all Windows/wasm/apple/redox-only. Making the tree
build against the Debian-packaged crates — the difference between a
vendored source package and a thin debcargo one — needed:
- sha1/sha2/md-5 bumped to the digest 0.11 line (sid moved, 0.10 is
trixie-only): finalize() now returns hybrid-array Output without
LowerHex (hex::encode it) and no io::Write impl (sha256_file reads
chunks into update()). The digest outputs are unchanged; the full
suite including the checksums tests passes.
- crossterm 0.28 -> 0.29: Debian skipped 0.28 entirely.
- reqwest 0.12 -> 0.13: the blocking/json/stream usages carried over
unchanged.
- suppaftp behind an opt-out 'ftp' feature: the one dependency absent
from the archive, used only by the plain-FTP upload fallback of
'pkh put' — the Debian build drops it and the SFTP failure path
reports the transport as unavailable instead.
- every patch-level requirement pin relaxed to the minor level
(4.5.51 -> 4.5, 1.0.228 -> 1.0, ...) — semver-compatible, and what
lets trixie's slightly older crates satisfy the manifest.
Result: 30 of 32 direct dependencies resolve from sid as-is; the
remaining two are indicatif 0.18 (sitting in experimental, migrating)
and the feature-gated suppaftp. No vendoring on sid.
The release-candidate version lands in 'pkh --version' and in the
build records' manifests, so an RC-tested tree is distinguishable
from the final release.
The size cap was the wrong shape: an --incremental user asked for a
carried tree, and the invariant that matters is which trees survive —
the newest successful one (the next run's automatic fork base) and the
newest one (what a failed run is re-forked from explicitly). The cap
is gone, with the tree measurement and manifest field it needed; the
retention planner now works on the records' outcomes alone and fills a
budget: staged_trees staged trees per target, the newer ones first,
the pair always among them. When the budget cannot hold the pair (one
staged tree configured, the newest build failed and an older
successful tree present) only one of the two is successful and that
one is kept; with no successful tree at all, the most recent one is.
The budget is a setting of the nested config file:
deb:
incremental:
staged_trees: 2
(default 2; 0 clamps to 1 with a warning — the newest successful tree
always survives). The fork-base selection no longer truncates the
candidate list to the budget before ranking it: with a budget of one
and a failed newest tree, that truncation hid the successful record
and forked the failed one instead.
The retention kept the newest five staged trees per target, so the
staged trees of a kernel-scale target alone could hold three-digit
GiB. The policy is now the pair that earns its disk: the newest
staged tree (what a failed run is re-forked from explicitly) and the
newest successful one (what the next --incremental run forks
automatically) — one record when they coincide; every other staged
tree degrades to metadata-only at the next build's retention. The
retention decision moved into a pure planner over (success, size) so
the policy is table-testable.
The size cap no longer drops a tree outright: an --incremental run's
own tree is always kept by its own run — the user asked for a carried
tree, and at least one always exists — and the cap only thins the kept
pair while another member covers the resume, the newest successful
tree being the last one standing. Oversized trees left by older pkh
versions or interrupted builds come down at the next build's
retention, once their lock is free.
The cap moves to the first user settings file, config.yml in
~/.config/pkh (YAML, like the data files; missing file or keys mean
defaults, a corrupt file is backed up, PKH_STAGED_MAXSIZE overrides),
replacing the JSON sketch from the previous commit. Default 20G, 0
disables the size rule. Every retained tree's size is measured once at
retention and recorded in the manifest, so sweeps never re-walk the
trees.
The per-target retention kept the newest five staged trees however
large they are: five carried kernel trees are three-digit GiB, most of
it compiled objects whose resume value does not scale with their size.
A staged tree larger than the retention cap (a user setting, see
below) is now dropped at the end of its own build — the record
degrades to metadata-only and the next build re-stages from scratch —
and the retention sweep also degrades older oversized trees, covering
records kept by older pkh versions and interrupted builds. The cap of
0 disables the size rule; the count cap is unchanged.
The cap is the first setting of a user configuration file,
~/.config/pkh/config.json (the directory already holds contexts.json):
missing file and missing keys mean defaults, a corrupt file is backed
up and the defaults apply, and PKH_STAGED_MAXSIZE overrides the file.
The default is 20G, the scale where a carried kernel tree stops
earning its disk.
Every retained tree's size is measured once at retention (the same
concurrent, mount-crossing-safe walker the prune report uses) and
stored in the record manifest, so sweeps reuse it instead of re-walking
the trees on every build.
resolve_pkh preferred target/release/pkh however old it was: after a
commit without a release rebuild, the harness measured the previous
code — the 54-package run recorded before this change exercised the
pre-nested-namespace builds and recorded no phase profiles, the
profile feature itself being newer than that binary. A repo binary is
now rebuilt when it is older than HEAD or than the newest source edit;
an explicit --pkh still pins whatever binary it names.
Rootless builds handed the staged tree to the build user with a
recursive chown; through the overlay staging every lower-layer file
the chown touched copy-ups — the full data movement of the tree, 19%
of a kernel build's wall clock.
The rules steps are now flagged and the unshare driver enters the
chroot by pivot_root instead of chroot(2), then creates a nested user
namespace mapping the context's root onto a fixed build id: the
chroot's files are the invoking user's host-side and the context maps
that user to uid 0, so the staged tree is owned by the build identity
by construction — no chown, no data movement. The kernel refuses user
namespace creation to chrooted processes, hence the pivot, and the
bind is recursive and taken inside the namespace: a plain bind of the
chroot's host-made submounts fails EINVAL on locked children, while
the rbind carries them across the pivot, and a fresh /proc is mounted
before the old root is shed (umount resolves through /proc/mounts).
fakeroot's chown wrappers attempt the real chown and swallow only
EPERM; as the mapped identity, root targets are unmapped and fail
EINVAL, so mapped builds export FAKEROOTDONTTRYCHOWN — fakeroot's own
knob — and the fake ownership is still recorded. Files the build
writes come out as the invoking user host-side, which also fixes the
silently missing .buildinfo/.changes of rootless builds: the host-side
rewrite of debian/files hit a subuid-owned file with EACCES.
The wrapper is probed per build; when nested namespaces are
unavailable (old util-linux, userns disabled), the previous setpriv
plus selective parallel chown handover runs unchanged. A setpriv-era
staged tree is handed to the invoking user once, host-side and
metadata-only, before the overlay mounts it — afterwards overlayfs
serves stale owners from its attribute cache.
The handover chowned the whole build root with a serial 'chown -R'.
Through the overlay staging every lower-layer file it touched is
copy-up'd — the full data movement of the staged tree — which the
profile of a kernel build showed as 15m53s in 'Preparing the build
user', 19% of the whole run.
The handover is now a 'find ! -uid | xargs -P chown' pipeline: only
entries the build user does not own yet are chowned (staging and quilt
re-application flip entries back to root; a resumed tree is already
owned and the phase collapses to a metadata-only walk), and the
copy-ups of independent files parallelize across the chroot's cores.
On a 200k-file tree the handover drops from 9.9s to 0.9s.
The ccache handover ran only when the build user was fresh, which is
every build in practice (the chroot is per-run scratch); it now rides
the same selective pipeline on every run, still best-effort — a failed
handover costs cache hits, never the build.
The pull downloads reported through the plain count port only because
byte_progress landed on main after the branch was cut. PullView now
implements it with a byte-counter template in the transfer-style
family (same shape, {bytes}/{total_bytes} counters, human-formatted by
indicatif), downloads emit through it, and the trait doc lists the
event among the ones the pull flow emits — un-duplicating the merged
'message, progress and progress' sentence.
* cleanup/errors-deps:
apt/release: type the file-verification and armor errors
new: type the scaffold validator errors
debian: type the errors of the format primitives
deps: add thiserror
pull,package_info,ui,cli: report the pull flow via BuildView
deps: trim the tokio feature set to the APIs in use
deps: drop the lazy_static dependency
data,apt/release,distro_info,new: replace lazy_static with LazyLock
deps: drop the unused cmd_lib dependency
# Conflicts:
# src/ui.rs
verify_file returns FileVerifyError (no Release file, unlisted path,
checksum mismatch, size mismatch — the structured pieces carried, so
the by-hash retry in package_info can match on the failure instead of
parsing text) and the in-process armor/base64 decoders return
ArmorError; the gpgv staging and keyring-download paths keep their
diagnostic strings, rendered at the boundary. The existing
VerifyError(String) keeps its name and downcast role for suite
signature verification. Display texts are byte-identical to the old
format! strings.
The leaf validators of the wizard (source name, command, upstream
version, maintainer, homepage, Depends, orig path, template id, file
collisions) now return a shared options::ValidationError carrying the
structured pieces (the rejected string, the wrapped DebianVersion or
Deps parse failure) instead of pre-formatted Strings; the wizard
resolve flow keeps its aggregate String errors and renders at the
boundary, and the Prompter validator closures adapt with to_string()
since the port's Validator deliberately exchanges re-ask explanations
as plain strings. Display texts are unchanged, as are the tests that
assert on them.
Give every parser of the shared format layer a proper error enum
(thiserror): VersionParseError, ArchError, deps::ParseError (wrapping
the version and arch failures as #[source] so the context chain
survives), ControlParseError, ChecksumParseError, FilesParseError and
TestsControlParseError, each carrying its structured pieces (the
offending string, the stanza number, the field name) instead of a
pre-formatted String.
Display texts are byte-identical to the old format! strings, so
user-facing output and the message-asserting tests are unchanged; the
few tests asserting on error text now render the error first. Flow
boundaries that still take String (build::env, quirks, test::expand)
render with to_string() at the edge — typing those flows is a later
step. ControlInfo's FromStr impl follows parse_content's error type.
Retire the ProgressCallback option-of-closure, the one pre-port
reporting idiom left in the tree: pull and package_info::lookup now
take &dyn BuildView like every other core flow, and the announcements
map onto message (unsized work), progress (item counts: series probes,
git objects, extraction) and progress again for byte downloads, which
the old callback carried in the same 4-argument shape.
The terminal side swaps the (ProgressBar, closure) pair for PullView, a
BuildView implementation owning one bar of the shared MultiProgress
with the same spinner/transfer style switching; it releases the line on
drop, which the CLI triggers explicitly on both outcomes since
exit(1) skips destructors. Headless callers (tests, internal lookups)
pass &report::Quiet. git clone sideband parsing is skipped entirely
when the view is not enabled, matching the old None-callback shortcut.
The runtime (multi-thread, for Runtime::new + block_on and task
spawning), time (retries, timeouts), sync (channels) and macros
(#[tokio::test]) cover every tokio:: path in the tree; 'full' also
pulled fs, io, net and process support nothing uses. Cargo.lock loses
the corresponding mio/signal deps if nothing else needs them.
Unify the lazily-initialized statics on std::sync::LazyLock, which the
newer code already uses: the embed_data! macro expands to a plain
static now, which also removes the visibility-forwarding workaround the
lazy_static! matcher forced on it (the @expand arm wrapping vis in
parentheses), and the 'static ref' spelling is kept at the call sites.
distro_info's HTTP client, the apt keyring cache and the new-template
registry follow the same pattern.
Confirming a 500 GiB prune went silent under "Removing:" for minutes:
drive an indicatif bar from the plan's sizes through a new
apply_with_progress observer, in the deb live view's layout — the
running count behind the spinner on the first line, the byte bar with
its counters on the second. The removed path is not rendered: it
fights the bar on a narrow terminal, and the failed-removal errors
name the paths that matter. --force plans no sizes and gets the count
line only.
The dry-run size spinner lacked the "> " prompt prefix the other
one-line waits carry; wait_spinner renders it too now.
Forking an incremental base copies its whole staged tree — bounded,
and without reflinks a matter of minutes — but the copy emitted no
build events: the live view showed the target line with nothing below
it, and an interrupt there printed raw cp errors with no hint of what
was running (the fork happens before the first bootstrap phase owns
the view).
Announce the copy through a new BuildView::byte_progress port. The
fork sizes the source up front (the copy's total), re-measures the
growing destination from a scoped thread while cp runs (cp has no
progress output to tap), and reports the completed copy when it
returns; unmeasurable or empty sources copy unreported. The terminal
view renders it like the other determinate bars, with byte-unit
counters.
The incremental fork's staged-tree copy wants to report byte progress
against the same measurements the prune report is built from: move the
concurrent size walk and the human-readable byte formatting out of
prune into utils::size and rewire prune to them. Behavior is unchanged:
same-device walks, unreadable entries count as zero.
The shared apt download cache is bind-mounted from outside the
chroot, so when an apt of ANOTHER build holds the archives lock from
its own pid namespace, a concurrent build's apt sees the holder as
"process 0", cannot wait for it, and fails the build immediately
(observed as an e2e samba essentials install racing a kernel build's
build-deps transaction on the stonking-amd64 cache).
Set DPkg::Lock::Timeout to ten minutes in the shared-cache apt conf:
apt then waits for the holder instead of erroring. pkh itself
already serializes builds per cache with a host-side flock; the
timeout covers the window where a killed run's orphaned apt is still
winding down.
- -f/--force skips the size pass and the confirmation prompt: remove
immediately, reporting counts but no sizes. The fastest cleanup
path; --dry-run still previews.
- The default path shows the grouped digest and asks before removing
(the raw-mode yes/no prompt; non-interactive sessions keep their
headless path, so scripts reach for --force).
- Sizes are computed under a spinner ("Computing sizes of
directories to prune..."), and the digest colors the counts (bold)
and the sizes (cyan) on terminals only.
The dry run still spammed one info line per artifact before the
grouped digest, and the size walk was serial: tens of seconds of pure
readdir/stat over the records' hundreds of thousands of files.
The per-path lines drop to debug (the grouped report is the
user-facing output), and the size pass now fans out over a shared
work queue with scoped threads — directory-level parallelism, same
du -x mount-crossing rule. 16s to 6.4s wall on a 440 GiB sessions
tree, totals unchanged.
The prune output spammed one line per path. The report now groups
entries by artifact kind with their total size, and the size is
measured before removal whatever the mode, so --dry-run prints a
digest ("24 residual build directories for 1.2 GiB") and both modes
end with the freed total. The size walk never crosses a mount point:
residual chroots hold bind-mounted /proc trees whose virtual files
(/proc/kcore alone reports 128 TiB) would poison the total.
--all is also honest now: it removes recent build records and staged
trees (that is where the space is), and the flag description says so.
One behavior fix rides along: a record or lock a live build holds was
previously prunable with --all, which would wreck a running build;
held locks are now out of bounds whatever the policy.
A rootless build's upperdir entries are copy-up'd by whoever wrote
them during the build: the unprivileged build user, which host-side
is a subuid of the invoking user. The host-side sync of a forked
record removes the stale upper copy of every host-modified file, and
the plain fs removals failed with EPERM on those build-user-owned
entries: the kernel incremental run died with a bare
'Permission denied (os error 13)' and no context, ten minutes into
the run.
Route the sync removals through the privileged path (the same sudo
fallback the record teardowns use) and wrap the sync error at the
call site, so a failure names the operation instead of surfacing a
bare errno string.
The specs describe shipped features — incremental builds, the live
view, overlayfs staging, pkh new, pkh test, pkh lint phase 1 — and
the satellite-tools plan is implemented in full (no shell-outs to
dpkg-architecture, dpkg-checkbuilddeps or dpkg-parsechangelog
remain). The two unshipped proposals (the lint porting track and the
data/configuration architecture) stay in whatever notes the project
keeps outside the tree; the README remains the user-facing
documentation.
Plain-speech pass: drop the em-dash asides and mid-sentence colons,
reword the boldface digest constructions ("What --incremental buys
is ..."), and replace the fluffy intro ("aims at wrapping ... while
being smarter at integrating all workflows") with what pkh actually
does. No content changes.
The profile's build-dependency phase spanned everything between the
apt passes and the build step, and for a kernel incremental build
that hid ~13 minutes of housekeeping: the recursive chown handing
the staged tree to the build user, and the resume_clear quirk
dropping the previous attempt's stamp caches. Both now enter the
profiler (ClearingResumeCaches only when the quirk has paths and a
tree is carried, PreparingBuildUser only for rootless builds), so
the status bar and pkh deb profile show where the time actually
goes — on a large carried tree the chown dominates, which is the
honest answer to a 6m30s DEPS column next to a 19m19s phase.
A rules chown failing for the unprivileged build user is expected
and harmless (see the revert of the build-step fakeroot wrap), but a
sysroot tree can print thousands of 'chown: changing ownership of
...: Operation not permitted' lines. MakeClassifier now shows one
warning digest for the flood and hides the rest; every other line,
make errors included, keeps its classification. The full tee in the
record's build.log is untouched.
This reverts commit 5f36a54db42e0e5b8f5243dded89f68cd93fa689.
Fakeroot's IPC degrades under kernel-scale parallel builds: a
linux-riscv build with make -j plus LTO floods the faked daemon and
libfakeroot prints 'internal error: payload not recognized' per
mangled message, with the ownership database losing entries. The
benefit the wrap bought is vacuous anyway: the .deb is assembled in
the binary step's own fakeroot session, which records files unknown
to its database as root-owned — verified with a sysroot package
whose build chown fails: the produced .deb still carries root/root.
A rules chown failing for the unprivileged build user stays expected
and harmless; the visible noise is collapsed at the classifier level
instead (next commit). The bare build step also matches modern dpkg
semantics: with Rules-Requires-Root: binary-targets (pkh's default
export), dpkg-buildpackage itself runs the build targets unprivileged
and fakes root only for the binary targets.
The staged tree was the parent of the tree the caller pointed at,
so building a tree sitting in a busy directory dragged every
sibling into the chroot: a /tmp checkout copied the whole host /tmp
(snap sockets, other jobs' scratch), and the build user's chown -R
then failed per foreign-owned file, or aborted the build outright.
Stage the tree itself: its content is the merge point of the build
root, the overlay lower is the tree and the record's upper stacks
on top. The package resolution treats the staged root as the
package when it carries debian/, keeping the pattern search for
nested layouts. The .debs, the upload metadata and the dose3
scratch follow dh_builddeb's .. (the package directory's parent)
instead of assuming the build root, and the host-tree snapshot
covers the tree only. Records staged the old way self-heal: their
snapshots carry parent-relative paths, so the next incremental
run's sync wipes the stale upper entries and rebuilds once.
Siblings are no longer visible to debian/rules — dpkg packaging
never crosses that boundary, and pkh test already staged the tree
alone.
The rules run as the unprivileged build user, so an ownership change
they make during the build (a sysroot chown) failed per file with
EPERM, spamming the build view and log and silently not happening.
Wrap the build step in fakeroot like the binary step already is, and
share one session state between the two — the way dpkg-buildpackage
-rfakeroot wraps every rules target: the chown succeeds faked and
carries into the .deb assembly. With the context's own user the
build step stays bare. The state file lives in the staged build
root, outside the package tree, and is removed before each build so
an interrupted previous attempt cannot seed the session.
Only one build mode exists and the flag only ever accepted its
default 'local'. The --mirror help keeps what a user acts on. The
URL feeds mmdebstrap so every apt resolves from it, and the mirror
must serve every suite and architecture the build resolves.
--incremental says the same things without the asides, and
--no-ccache is one line.
Time every Phase the build flow announces through enter_phase — the
chroot bootstrap (tarball download, extraction, finalization), the
apt phases, patch application, the build and binary steps and the
artifact retrieval — and land the ordered profile in the record's
manifest of every run, failed builds included: their profile ends
at the phase the error propagated out of.
Timing rides the enter_phase choke point instead of new measurement
sites, so a phase added later profiles itself. The manifest field
is additive (serde default, skipped when absent), so older manifests
keep loading without a schema bump. The coarse timings stay the
source of the deb list columns: they measure the apt transactions
themselves, where the profile measures whole phases.
pkh deb profile renders a record's profile through the same
read-only record discovery as pkh deb log — the newest record of
the tree, or an id prefix as shown by pkh deb list — so
record_for_log becomes the neutral find_record. Phase keys are
storage names, distinct from the display labels; Phase::from_key
turns them back and a test keeps both in sync.
deb: build the rules as an unprivileged user in pkh chroots, the
same way Launchpad's buildds and sbuild do - setpriv drops to an
on-demand pkh-build user owning the staged tree, the scratch HOME
and the ccache; dependency installation and quilt stay root.
Validated by the e2e canary: coreutils (which refuses a root
configure) now passes.
* rootless-builds:
test: drop the coreutils xfail, rootless builds clear it
deb: run the user probes through sh with stderr dropped
deb: build the rules as an unprivileged user in pkh chroots
The e2e canary against the rootless build records coreutils as
xpass: with the rules running as the build user, its configure no
longer refuses the build.
A missing build user is the expected state in every fresh ephemeral
chroot: without the redirect the id failures leak into the build log
as errors on every run.
Launchpad's buildds and sbuild compile as a non-root user, and
packages rely on it: coreutils' configure refuses to run as root
(the one remaining standard-tier e2e failure). Run the rules - and
the fakeroot-wrapped binary step - as an on-demand 'pkh-build' user
whenever the build runs in a context pkh controls (the ephemeral
chroot); ssh and schroot contexts keep their own user.
setpriv drops to the user without touching the environment, so the
dpkg architecture variables and DEB_BUILD_OPTIONS survive the
transition. The staged tree, the scratch HOME and the shared ccache
directory are handed to the user on every build; everything before
the rules (dependency installation, quilt) legitimately stays root.
Minimal mmdebstrap chroots carry no passwd package, so the user is
appended to /etc/passwd and /etc/group directly when useradd is
missing, as sbuild's build-user setup does.
All three fail on the current archive, none by pkh's doing: gdb's
PDF dies in texi2dvi against texinfo 7.3; sid's grub2 is the 2.06-2
branch whose control pins gcc-10, which sid no longer ships; and
cross u-boot compiles its kbuild host tools with the arm64 flags
dpkg-buildflags serves in cross mode (-mbranch-protection is amd64
rejected), the same under sbuild cross.
glibc 2.42 turned strstr into a macro with C2x redirect annotations
in string.h, and gprofng's libcollector does not survive it:
iolib.c:898 fails with 'expected identifier before _Generic'. Fixed
upstream after the 2.45 snapshot this rebuild uses - a distro
toolchain change hitting every rebuild, not pkh.
The build context inherits pkh's environment, so the rules ran with
HOME set to the invoking user's host path - which does not exist
inside the chroot. Tools keying user config on HOME then fail
mid-build: gdb's doc build dies in texi2dvi because the TeX live
setup cannot use the nonexistent home.
Point HOME at /tmp/pkh-home, created through the build context (all
drivers: local, unshare chroot, ssh, schroot). sbuild uses a
non-writable path, Launchpad's buildd user a real one; a writable
scratch directory matches the latter.
A component tarball whose entries already carry the addon prefix
('www/...' in sqlite3's 'sqlite3_3.53.4.orig-www.tar.xz') extracts
to its end state, but the wrapper detection named it a wrapper and
unwrap_into then cleared the addon target - destroying the component
just extracted into it - and unwrapped the emptied directory into
itself: no 'www' at all, and the deb failed at dh_installdocs over
the missing 'www/*' docs.
dpkg-source unpacks a component into '<root>/<addon>': when the
tarball's own layout already is that, the extraction is the end
state; leave it alone.
unwrap_into pointed the component's unwrap at '<root>/<addon>' but
never created that directory: unwrap_wrapper moves the wrapper's
entries into it by rename, and a rename whose target parent is
missing fails with ENOENT - every entry, aborting the whole pull.
The multi-orig unit test exercised unwrap_wrapper directly and could
not see it (dovecot).
Also exercise the component through unwrap_into in the test, and
assert the wrapper's own layout does not leak to the root - the
shape dpkg-source -b rejects as unexpected upstream changes.
python3-lxml 6.1 no longer consults custom resolvers for parameter
entities, and systemd's man build leans on exactly that (the
custom-entities.ent resolver in tools/xml_helper.py):
make-directive-index fails with 'Entity entities not defined'. A
distro toolchain change hitting every rebuild, not pkh - the same
breakage forced a backport in Yocto.
dpkg-buildpackage normally exports the rules' root requirement from
debian/control; running the rules directly does not, and tools key
their assembly strategy on it - without it debputy refuses to
assemble packages shipping root-owned paths at all (apache2). Policy
defaults to 'binary-targets' when control does not declare it.
A changelog may target a pocket suite ('trixie-security') rather than
the bare series: split the pocket off so the build resolves against
the base series with that pocket enabled - the same target an
explicit '-s trixie -p security' names. The pocket must be one of
the base series' distro-info pockets, which also guards against
series names merely containing a dash; an explicit --pocket wins.
Debian serves security updates from a dedicated archive: appending
the security suite to the main mirror would 404 on 'apt-get update'
(ubuntu serves every pocket from the same archive, where appending
works) - clone the pinned source against deb.debian.org/debian-security
instead, as mmdebstrap does.
patch reports rejected hunks on stdout, not stderr: the error used
to carry only the diff path and an empty stderr, naming neither the
failing file nor the reason.
Git-archive tarballs open with a pax global header record
('pax_global_header'): GNU tar consumes it and never lists or
extracts it, but the tar crate yields it as an entry. Counting it
for the wrapper detection made every git-archive orig look
multi-rooted - linux-riscv's orig stayed nested under 'linux-6.8/'
and its .diff.gz failed to apply with 'can't find file to patch'.
Skip the record outright: unpack would no-op on it anyway, so this
only keeps it out of the wrapper detection (and the tree).
dpkg-source assembles a multi-orig source by unpacking the main orig
first and the extra components over it, each component's content at
'<root>/<addon>' ('pigeonhole' for 'pkg_1.0.orig-pigeonhole.tar.gz'),
so component files win where they overlap the main tree: process the
archives in that order, and unwrap each archive's wrapper into where
dpkg-source expects it - the root for the main orig, '<root>/<addon>'
for a component, replacing any pre-existing addon directory.
Recognizing the wrapper per archive (the one first component every
entry of THIS archive shares) rather than from the shape of the tree
root keeps every component of a multi-orig package unwrapping after
the first one filled the root: dovecot's pigeonhole sources used to
be left nested one level too deep, and dpkg-source -b then refused
the tree over unexpected upstream changes.
Exercises pkh the way it is used in anger: 63 real source packages
(from hello and micro to dracut, grub2 and the kernel) run through
pull -> chlog --rebuild -> build -> deb, each stage timed, logged and
checked for its artifacts, with the catalog alternating between debian
and ubuntu within every category.
run.py keeps runs self-contained (per-package logs, progressive
results.json, a markdown summary), pulls in parallel while builds run
sequentially to keep pkh's shared chroot/apt/ccache caches warm,
resumes interrupted runs reusing the stages that already succeeded,
and marks known-broken entries as xfail with a reason.
The first catalog run caught four pkh defects within an hour - the
unstable/sid suite alias, the bz2 origs, the hard link extraction and
the wrapper unwrapping - which is exactly what it is for.
The rules run outside dpkg-buildpackage, which exports none of the
dpkg architecture variables anyway - only sbuild does. Packages using
$(DEB_HOST_MULTIARCH) or $(DEB_BUILD_GNU_TYPE) without including
/usr/share/dpkg/architecture.mk expanded them empty: micro's rules
cd'd into 'obj-' and file configured --libdir=/usr/lib, putting its
libraries where dh_install's multiarch glob could not find them.
Run 'dpkg-architecture -s' in the build context and export the whole
DEB_* set, sbuild-style, for native builds; cross builds already carry
the target's set from cross::setup_environment, whose output parser
this reuses.
The changelog (and -s) may carry a suite name instead of the series
codename: Debian conventionally writes 'unstable' where the series
data carries 'sid'. The name was used verbatim downstream, so every
NCR rebuild of a Debian package keyed its chroot tarball cache on
'unstable-buildd' (missing the cached sid tarball), tried to bootstrap
a tarball nobody waits for, and finally died on 'Unknown series:
unstable' in the distro-info lookup - with each queued build waiting
out the 300s stale-lock grace on the way.
Resolve the alias right where the series is picked, so the codename
identifies the target everywhere downstream; the codename is also what
the caches and distro-info data key on.
Three archive-side defects broke pulled trees, each surfacing as a
later build failure rather than at pull time:
- '.tar.bz2'/'.tbz2' origs (busybox) were downloaded but never
extracted: the dispatch only knew gz and xz. dpkg-source -b then
packed the missing tree as a mass of ignored deletions and the
chroot's quilt run failed on every patch. Extract bz2 through
bzip2-rs, and match every '.tar.*' of the source package so an
unsupported compression fails loudly instead of silently skipping
the extraction.
- Hard link entries (jq) aborted the whole extraction: resolve their
targets against the archive root, where tar guarantees the shared
entry was already extracted, with the same confinement checks as
regular entries.
- The 'package-version/' wrapper was detected from explicit tar
directory entries (tree has none, so its sources stayed nested one
level too deep) and relocated through a copy that flattened
top-level symlinks (htop's 'README -> README.md' became a plain
file, which dpkg-source refuses to represent). Recognize the
wrapper by the shape of the extraction result - exactly one real
directory - and unwrap it by rename, preserving links and
timestamps.
Decode-only pure-Rust bzip2, matching flate2's pure-Rust default: no
new system packages for building pkh, and the archive-side origin of
the need is the busybox class of '.tar.bz2' upstream tarballs.
dh_strip registers debug-symbol packages in debian/files under their
renamed .ddeb name; they are deliverables of the build just like the
regular .debs. Both artifact-type filters, the one collecting the
build's artifacts and the one feeding the .buildinfo/.changes
checksum sections, accepted only deb and udeb, so every ddeb a build
produced was silently discarded: nothing exported next to the .debs,
nothing in the upload metadata.
The filter is now a shared FilesEntry::is_binary_artifact() predicate
(plus unit tests) used by both call sites, so the two cannot drift
apart again.
One dpkg distinction has to be copied rather than shared:
dpkg-genchanges lists only deb/udeb packages in the changes' Binary:
and Description (a ddeb travels in Files and the checksum sections
but is not a listed binary of the upload), while dpkg-genbuildinfo
lists every binary artifact's package, ddebs included. The package
list is therefore accumulated twice, once per consumer.
The binary-metadata differential fixture gains a Package-Type: ddeb
package, so the native .changes/.buildinfo generation is compared
against real dpkg with a ddeb present. Without the fix the comparison
fails on the checksum sections (golden distributes the ddeb, ours
dropped it); with it every field matches.
The buildd tarballs carry their /dev entries as char-device members,
but the tar crate cannot create special files: unrecognized entry
types are written out as regular ones, so every device entry in the
chroot extracts to an empty regular file. The old readiness guard
only checked that /dev/null existed, which a regular file satisfies,
so the rm + mknod repair never ran.
Redirects into a regular /dev/null still succeed, so the build
proceeds until something reads the node back. The first reader in
practice is dh_strip's dbgsym path, where dpkg-gencontrol gets
-T/dev/null (dbgsym packages have no substvars file) and aborts on
"bad line in substvars file /dev/null at line 1". Packages without
debug symbols never reach that call, which is why earlier builds
looked fine.
The guard now requires every node to be a char device, and the node
set covers all entries the tarballs ship (console, full, null, ptmx,
random, tty, urandom, zero), not just null and zero: a regular-file
/dev/urandom or /dev/console is a silent hazard for anything reading
it later in the build.
Regression tests pin both shapes: the regular files extraction
leaves behind must not count as ready, and real char devices (via
symlinks to the host's nodes, so the test runs unprivileged) must.
Every binary build measures its dependency installation (apt-get
update, the essential and injected packages, each build-dependency
pass) and its package build (debian/rules build plus binary) into a
timings accumulator; the record manifest stores both as an optional
`timings` field when the run ends, whatever the outcome — a failed
build keeps its partial durations, and a run that died before any
timed command records nothing.
`pkh deb list` shows the two durations as DEPS and BUILD columns,
formatted like the age ("45s", "2m41s", "1h02m"); records predating
the field show dashes. The field is serde-optional, so the manifest
schema stays at 1 and existing records remain discoverable and
forkable.
Pinned runners (a caching proxy, a chosen mirror) must resolve every
package from one base URL. 'pkh deb --mirror <url>' passes the mirror
to mmdebstrap as its mirror positional, so the chroot tarball's apt
sources carry it from the bootstrap on and every apt inside the build
follows - no per-build source rewriting. The mirror must serve every
suite and architecture the build resolves (the security pocket and,
for cross builds, the target archive included); that contract is
documented, not enforced.
The tarball cache is keyed per mirror (a short sha prefix before the
'-buildd.tar.xz' suffix prune matches): a --mirror run over a stale
distro-default tarball would silently resolve from the distro, and
'--incremental' records must never fork across a mirror change.
The runtime source passes recognize the pinned URL as an archive
source (it carries no official host for the host-based checks): the
pocket pass and the universe pass treat it as the archive, and the
cross setup gives it every component and suite, no arch scoping (the
mirror serves every architecture by contract) and no ports entry.
'pkh test' passes no mirror: its tarballs stay distro-default.
Kernel builds abbreviate every invocation to an uppercase status tag
(CC, LD, HOSTCC, MODPOST, ...) instead of printing the full command
line, so the substring rules that catch classic make output never
fired and the rolling pane went static for the whole kbuild stream.
Recognize the common tags after leading whitespace and show those
lines.
Also surface warnings instead of hiding them: gcc's lowercase
'file.c:12:3: warning:' diagnostics and kbuild's line-leading
'WARNING:' banners (modpost, etc.) now map to Action::Warning.
The record-log wrapper provides a sink for every command, which removed
the runner's subprocess-inherit fallback -- and with it the raw output
of --verbose runs (and of piped stdout, where the live view disables
itself): Quiet's None sink got wrapped inside, every line went to
build.log, and the terminal showed nothing but pkh's own logs, breaking
what --help still promises. When the caller's view has no sink of its
own, a raw terminal sink (stdout/stderr writers, one per stream) now
takes its place inside the record-log wrapper: the terminal shows the
raw lines again while build.log keeps the full tee. A view that
consumes lines itself (the live view on a tty) is wrapped exactly as
before.
Every --incremental run claimed "kept its staged tree" and set
kept_upper=true at teardown — even when the run died before staging
created tree-upper, leaving a manifest that lied about the record being
a fork candidate and a message contradicting the disk. The retention
now follows the disk through upper_retention(requested, has_upper()):
a keep with a staged tree keeps it as before, a plain run drops it, and
a keep with nothing staged keeps the record as metadata + log with
kept_upper=false and says so ("no staged tree was created — the run
ended before staging"). The interrupt path records the same honest
state — failed outcome, kept_upper only if a tree exists — and the
SIGINT hook's message no longer claims a tree that is not there.
The snapshot walk runs over the whole staged PARENT, which contains
trees unrelated to the build; one unreadable path anywhere under it —
a root-owned sysroot left there by the user's own setup script — made
fs::read_dir or symlink_metadata fail and aborted the whole build
seven seconds in, before any staging could happen. The walk is now
best-effort below its root: unreadable directories and entries lost to
a mid-walk race are warned about and skipped, and only the root itself
being unreadable propagates (the package tree being invisible is a
real error). Absent entries are already the handled case for the
whiteout sync: they are simply untracked.
Fresh chroots also mean fresh compiles: nothing survived to make the
recompile-heavy middle of a run cheap. Every deb build now wires a
shared ccache — /var/tmp/pkh/ccache/<slug>/, bind-mounted at the
chroot's /var/cache/pkh-ccache — through a PATH shim rather than CC
environment variables: gcc/g++/cc/c++ symlinks in /usr/lib/pkh-ccache-
shim resolve to the ccache the essentials install, so rules that call
the plain compiler names are cached while explicit exotic compilers
stay untouched. CCACHE_COMPILERCHECK=content is critical — the default
is mtime-based and would never hit across fresh chroots, where every
toolchain's mtime is the extraction's — with CCACHE_BASEDIR=/tmp/pkh-
build and CCACHE_NOHASHDIR=true keying the staged tree's relative
paths, and CCACHE_MAXSIZE=10G (PKH_CCACHE_MAXSIZE overrides).
No mtime clamping anywhere: the tarball extraction preserves mtimes,
so the toolchain and libc headers stay older than carried objects and
make keeps them, while build-dependency reinstalls are newer and make
rebuilds their dependents — ccache turns those into hits, and a real
archive library change is a miss that rebuilds them, exactly the
signal clamping would silence. `pkh deb --no-ccache` skips the shim,
the env and the mount entirely: an escape hatch for rules that
mishandle compiler wrappers (kbuild-style flavour configs).
With the chroot now per-run scratch, every build re-downloaded the
packages its dependency install needed — the very cost the chroot reuse
used to hide. Every chroot's /var/cache/apt/archives is now a bind
mount over a shared cache at /var/tmp/pkh/apt-cache/<series>-<arch>/
(the chroot's architecture, like the tarball cache), which one run's
installs fill and the next runs reuse: apt keeps downloaded packages
through a dropped conf (APT::Keep-Downloaded-Packages), and an
apt-get autoclean at the end of each run drops the debs the archive no
longer serves. pkh prune removes caches untouched for a week.
The run holds a host-side flock on the cache for its whole lifetime:
apt's own archives lock is acquired inside the chroot's pid namespace,
where a concurrent build's apt shows the holder as "process 0" — an
invisible process it refuses to wait for — so pkh serializes the
traffic itself; concurrent builds of the same target wait for each
other. Any bind-mount failure falls back to the chroot's own cache
with a warning (correct, just slower), and the temp-chroot teardown now
unmounts everything under the chroot before rm -rf, so the removal can
never reach through the cache bind mount and delete the shared cache
itself.
Records carried a full chroot each, and space turned out to be the hard
constraint: five records per target multiplied by a multi-GB chroot does
not fit. The chroot is now per-run scratch — every build extracts a
fresh one from the sha-verified mmdebstrap tarball cache into its
record's chroot/ dir and removes it at the end (success, failure and
interrupt included); resume_chroot, the .pkh-ready probe and
chroot_is_usable are gone, and crashed-run leftovers are swept when the
next run of that identity starts.
What a record keeps instead: manifest.json (renamed, now with the
tarball sha it used, a kept_upper flag and a tree-only journal — the
environment phases rerun in every fresh chroot), a log/build.log full
tee of the build output (kept for EVERY build; plain runs become
metadata-only records, so keep-iff-incremental now applies to the
staged tree alone), and — for --incremental runs — tree-upper/ with its
host-files.list. The fork gate carries a base's staged tree only while
its recorded tarball sha still matches the cache and the changelog
version is unchanged; a fork candidate is any record with a staged tree.
The keep-last-5 cap now applies to staged trees: a sixth retention
degrades the oldest record to metadata-only instead of deleting it.
New `pkh deb log [-f] [<id>]` prints (or follows) a record's build log,
and `pkh deb list` shows the staged-tree state and plain-run records
too. README rewritten around the shrunk model.
The single mutable session per identity loses the last-good state: a
failed run after a good one adopts the session in place and leaves only
the failed state behind, so "continue from the last success" was
impossible. Each build now records an immutable record under
/var/tmp/pkh/sessions/<slug>/<id>/ (chroot, tree-upper, journal,
manifest), and --incremental forks a NEW record — a full copy of the
base's chroot and, at the same changelog version, its staged artifacts
— under the base's flock, never mutating the base itself.
Selection picks the newest successful record of the identity among the
last five usable, falling back to the newest when none succeeded, and
--incremental <id> pins an exact record (failed ones allowed: their
environment may be fine). The lifecycle is strict keep-iff-incremental:
plain runs build in a throwaway record discarded on success, failure
and interrupt alike (the SIGINT cleanup hook follows the same policy),
while --incremental records are always kept. Each identity keeps its
newest five records; pkh prune's 7-day sweep and orphaned-lock cleanup
follow the nested layout.
--resume and --keep are removed (no aliases): their adoption-in-place
and keep-on-success semantics are exactly what the record model
replaces. pkh deb list shows id, outcome, tree version, fork lineage
and age. README rewritten around the record workflow.
Iterating on a package currently needs two flags (--resume --keep):
without --keep a successful build consumes the session, and without
--resume nothing is reused, so incremental iteration never "just
works" opt-in.
--incremental is the single-flag form: with no --resume it adopts the
newest usable session of this tree (like `pkh deb --resume`), and it
always keeps the session on success, so the loop (edit, `pkh deb
--incremental`, edit, ...) reuses the chroot, build deps and objects
and only recompiles what changed. With an explicit `--resume <id>` it
only adds keep.
A plain `pkh deb` is unchanged: fresh session, consumed on success.
The changelog path was hardcoded to debian/changelog in the series
detection, the entry generation and the editor invocation. Packages
keeping the changelog elsewhere (e.g. a staging copy) could not use
chlog on it.
Add -f/--file defaulting to debian/changelog, threaded through all
three uses. The library side was already parameterized on the file,
so no changelog.rs behavior changes beyond a test pinning the
non-default path.
Without a version tag, the boundary of a new entry was found by walking
the whole history and diffing each commit's tree against the changelog
path. On large repositories that per-commit tree diff takes 10+ minutes.
Add --limit (default 500, 0 = unlimited) capping that walk: when no
changelog commit is found within the last N commits, the oldest visited
commit becomes the boundary — the version tag probe is skipped there,
since it could jump the boundary arbitrarily far back and defeat the
cap — and a warning explains the fallback. A found version tag still
takes precedence and is unaffected by the limit.
The f055b70 skip assumed a single-debian-patch tree carries all patch
content ambient in the working tree. A tree that declares
single-debian-patch while listing real quilt patches in
debian/patches/series (e.g. a pkh-scaffolded package that grew patches,
like lite-xl) was built from unpatched sources: the binary silently
lost the patch content, while plain dpkg-buildpackage - and Launchpad,
which never sees debian/source/local-options - applies the series.
Mirror dpkg-source --before-build semantics: apply the series minus the
ambient 'debian-changes[-<version>]' entries, dropping those entries
from the staged series so 'quilt push -a' tolerates the ambient content.
A series holding only the ambient entry keeps skipping, as before
(covered by apply_phase_skips_single_debian_patch_trees); the
classification core is unit-tested in
series_patches_to_apply_classification.
Two failures from resumed kernel sessions:
A resumed build re-runs debian/rules build, but the kernels keep
their wrapper-step stamps in debian/stamps and the flavour config
rule there exports .config from the annotations with no config
prerequisite: the resumed build silently kept the previous
attempt's configuration and a config edit never reached the .deb.
Dropping the stamp cache unconditionally would impose the kernel
packaging's shape on every package, so it is data-driven instead:
packages declare the caches under a resume_clear quirk, and
resumed builds remove those tree-relative paths before the build.
The inner kbuild keeps its own incremental state, so only the
cheap wrapper passes re-run and config-affected objects recompile.
A SIGKILLed build (OOM) leaves its overlay mounts and /proc bind
mount behind, and overlayfs creates root-owned work state inside
the workdir: the next plain build (recording is always on, it
replaces the session of its identity) failed to clear the
leftovers with a permission error. Session clearing now unmounts
everything under each entry at depth (re-reading /proc/mounts,
tolerating mount stacks from consecutive crashes) and escalates
through sudo; when the tree still cannot be cleared, the build
reports it and continues without a session instead of layering
over a half-cleared one.
Session locks live next to the session root, and failed or
interrupted runs left them behind after the root was gone (seven
stray locks on a used machine). Session removal now unlinks its own
lock file, and prune collects the orphans: by default only the ones
no live build holds (probed with a non-blocking flock), with --all
every one of them. Locks guarding an existing session root are
never touched.
On a resumed session whose overlay mount fails (no overlayfs, mount
error, non-overlay driver), the fallback used to copy the host tree
into the staged tree of the previous attempt: a merge that keeps the
old build stamps and resurrects files deleted on the host — a
silently half-stale build, exactly what resume must never produce.
The driver now reports the failure instead of copying, and the
caller wipes the staged build root and re-stages a fresh copy
(environment resumable, build artifacts discarded).
Add the DEP-8 runner to the command list and a usage section: default
chroot testbed sharing the deb tarball cache, binary reuse from pkh deb
with in-process rebuilds, the local and vm modes, and the 0/1/2 exit
contract. autopkgtest joins the shell-out list (vm mode) and the
roadmap bullet comes off — the feature exists.
A local run fails apt-get update exactly like a local build does when
the user is not root; say so in the error instead of leaving a bare
failure (the chroot modes bootstrap their own root environment).
The native runner's trust anchor: on a tree of the caller's choosing
(PKH_TEST_DIFF_TREE, --ignored, deliberate: both runs land on the host
testbed), the harness runs pkh's local mode and the installed
autopkgtest with its null runner over the same tree and binaries, then
fails on any per-test verdict disagreement — pinning the semantics the
DEP-8 documents under-specify (verdict classes, argid names, edge-case
expansions). The comparison itself is pure and always tested.
VM virtualization is the one backend pkh does not reimplement: the qemu
runner carries the serial-console protocol, image overlay management
and reboot support, and it is the only one satisfying
isolation-machine/needs-reboot tests. pkh execs the installed
autopkgtest with its qemu runner (--image required, --cpus/--ram-size
passthrough) in its own process group so Ctrl+C reaches the testbed
cleanup, parses the --summary file back into the pkh report model and
maps autopkgtest's exit classes onto the 0/1/2 contract (12/14/16/20
are runtime errors; the raw code rides the JSON report). Pockets, PPAs
and injection are refused in vm mode for now: they would need the
keyring setup inside the image.
Test artifacts land under /var/tmp/pkh/tests (<source>-<series>-<arch>-
<timestamp>/), one directory per run: they are disposable results, not
state worth keeping. Prune removes them on the same 7-day retention as
build sessions, and with --all; the root is overridable through
PKH_TESTS_DIR like the sessions root.
Argument wiring only, per the report ports: the live view carries
phases and the outcome, the per-test report renders after the run
(text or --json) and decides the exit code; runtime errors exit 2.
--list-tests prints the parsed debian/tests/control and exits without
booting a testbed.
The DEP-8 run-time contract is one page, so pkh implements it natively
instead of wrapping autopkgtest (see plans/pkh-test.md): the testbed is
a pkh context — chroot mode reuses the deb ephemeral chroots and their
cached mmdebstrap tarballs, local mode runs on the current context —
and the binaries under test are the freshness-guarded pkh deb output
next to the tree, rebuilt through the deb flow when stale or missing.
Per test: restrictions gate on the testbed (isolation-machine,
needs-reboot and container isolation skip with a pointer at --mode vm),
the stanza Depends are expanded (@, @builddeps@, @recommends@) and
installed, then the script or Test-Command runs in the staged tree with
the AUTOPKGTEST_* (and legacy ADT_*) environment, a per-test timeout
and the stderr/exit-77 verdict rules; flaky tests retry once. stdout,
stderr and $AUTOPKGTEST_ARTIFACTS are collected per test under
/var/tmp/pkh/tests/ (pruned separately). --shell/--shell-fail drop
into the testbed for debugging.
pin_pocket and install_injected_packages become crate-internal helpers
shared with the deb flow, which they mirror.
BuildTarget::source_only carried two states for what is a three-way
distinction now that pkh test exists: replace it with a TargetKind
(Source/Binary/Test) and derive the tee log prefix (build-/deb-/test-)
from it, so test runs get their own log family instead of borrowing
the deb one.
Parse debian/tests/control into runnable test descriptions: one test
per Tests: script and per Test-Command value, carrying the shared
Depends/Restrictions/Tests-Directory stanza fields and the autopkgtest
command1 enumeration (test-name overrides advance the counter).
Depends substitution is deliberately left raw: expanding @ and friends
needs the built binary list, so it belongs to the runner.
First piece of pkh test (see plans/pkh-test.md); the runner expands
these stanzas against the deb822 primitives.
README: the build-session workflow (auto-recording, --resume,
'pkh deb list', --keep, prune retention) under a new section.
plans/: the design spec with the decisions taken along the way
(opt-in resume, --keep for the iteration loop, /var/tmp/pkh/
sessions, Ctrl+C keeping the session) and the implementation notes
recording the as-built deviations.
Wiring for the session feature: --resume optionally takes a session
id (bare --resume adopts the newest session of the tree), --keep
keeps the session after a successful build, and 'pkh deb list'
renders the read-only session table for the current tree. Listing
is a subcommand, not a mode-switching flag: pkh deb with flags
always starts a build.
Sessions accumulate under /var/tmp/pkh/sessions for every failed,
interrupted or --keep build and can reach tens of gigabytes for
kernel-sized chroots: prune is their primary GC. By default remove
the sessions untouched for longer than the retention window (7
days) plus the corrupt leftovers; --all removes everything. Session
roots hold the same mounts as residual chroots (/proc bind mount,
overlays), so they are unmounted before removal.
prune_in() (the testable core) deliberately stays session-free:
the test suite runs prune tests and e2e builds concurrently, and
scanning the real sessions root from the tests deleted live
sessions mid-build. Session pruning is an explicit opt-in of
prune_in_roots(), used by the production prune().
Every local build records a session under /var/tmp/pkh/sessions:
the bootstrapped chroot, the installed build dependencies, the
phase journal and the staged tree with a persistent overlay
upperdir. Rebuilding after a failure currently redoes the tarball
extraction, apt update, build-dep resolution and the whole compile;
a kernel-sized package loses half an hour per iteration.
Recording is always on, reuse is opt-in (--resume [<id>]): the
newest session of the tree, or the one matching an id as shown by
'pkh deb list' (one session per series/arch/cross identity, ids
are build-start timestamps). Resume skips the chroot bootstrap
(integrity marker + tarball check), re-mounts the previous
upperdir so make recompiles only what changed, pops the quilt
series first when debian/patches changed, and clears debian/files
before re-packaging so artifact collection stays exact. A version
bump keeps the environment and discards the build artifacts.
Explicit selectors disagreeing with the adopted session are an
error, never a silent environment switch. Failures and Ctrl+C keep
the session (the interrupt hook unmounts but preserves the tree); a
success consumes it unless --keep. Concurrent same-identity builds
serialize on a lock file kept OUTSIDE the session root: teardown
removes the root while holding the lock, and a lock inside it would
be deleted under the holder, letting the next opener lock a fresh
inode. The apt phases rerun on resume (idempotent, seconds-cheap)
rather than being stamp-gated.
Resumable build sessions need to re-mount the staged tree over the
same overlay upperdir on every attempt: the build artifacts written
inside the chroot (object files) live there, while the host tree
stays visible as the live lowerdir. Add an optional
ensure_available_with_overlay staging path (OverlayStaging reports
whether the persistent upperdir was actually mounted or a fresh copy
was staged instead) with a default implementation rejecting the
request, so drivers without overlay support degrade cleanly.
The overlay mount itself is factored out of try_overlay_mount into
mount_overlay, taking optional pinned upper/work directories; a
failed mount only cleans up the directories it created, never the
caller-pinned ones.
dpkg-buildpackage and sbuild export no pkg-config redirection: the
environment is the package's to set. Pointing the whole build at the
target multiarch pkgconfig dirs makes every pkg-config-consuming tool
resolve against the target libraries, the host-side tools of the same
build included: the host linker is then handed target-arch -L paths
(the kernel's resolve_btfids logging 'skipping incompatible
/usr/lib/aarch64-linux-gnu/libelf.so') and only survives through
fallbacks when the build-architecture libraries happen to be installed
too.
Packages whose cross builds need target pkg-config data must arrange
for it themselves, the way the kernel packaging builds rtla statically
against no target system libraries.
The resolute and stonking kernels declare libelf-dev, libdw-dev and
libssl-dev unqualified, unlike the Debian control which carries the
same names qualified :native for the host-side tools (resolve_btfids,
gendwarfksyms, sign-file). The dpkg cross rules resolve an unqualified
Multi-Arch: same name against the host architecture only, so a cross
build installs no build-architecture variants and the kernel's
host-side tools cannot link (-ldw, -lelf, -lssl all fail).
Inject the :native variants for the affected series — the declared
unqualified dependencies stay, so the target-side tools keep their
host-architecture libraries. Series-scoped so the entry can be
dropped once the control is fixed upstream.
An unqualified Multi-Arch: same build-dependency resolves to the
host-architecture variant only, even when a build-architecture
candidate exists: that is what dpkg's checker accepts (KnownFacts
accepts foreign, host and all instances) and what sbuild installs,
whose dummy package carries the parsed Build-Depends unqualified and
whose sbuild-cross-resolver only filters foreign M-A:foreign and
Essential packages out of the apt universe — verified by resolving a
sbuild-style dummy:arm64 against the resolute/stonking indexes, which
yields libelf-dev:arm64 & co and never the amd64 variants.
The build-architecture side of such a library is only ever installed
through an explicit :native declaration. Pin both rules so a resolver
regression towards dual-installing unqualified dependencies cannot
reappear unnoticed.
pkh v0.1.0 is published to crates.io, so cargo install pkh is now the
primary installation path; building from source stays as the
alternative. Drop the 'no distribution channel' notice.
vhaudiquet