The widget's steady-tick redraws fight the prompt for the cursor, so a
raw-mode question never stayed on screen: pkh put showed the warning,
then appeared to hang, with the upload running blind behind the
spinner. Every prompt now releases the widget first (its full teardown,
through a hook the view registers; prompts stay view-agnostic). The
release is one-way — after a prompt the view narrates in plain lines
instead of through the gone spinner — and hooks are generation-tokened
so a view's drop never clears the registration of a view created after
it (pkh new runs two builds in a row).
The per-guard questions asked about one mistake at a time and, asked in
sequence, would have interrupted an upload several times for what is
really one decision. The guards now all run, every reason found is
shown before anything stops (one rebuild can fix everything at once),
refusals stop the upload, and only warnings ask, once, generically:
'Upload with warnings?'. Verdict::Ask loses its per-guard question
field.
Each Launchpad API round trip costs a few hundred milliseconds of
server processing, and put walked four of them sequentially (person,
archive, published sources) before the connection came up: the slowest
phase of the flow. The person and archive lookups are independent and
now go out together; the three pre-flights (PPA info, target series,
published versions) run as one join with results checked in the
original order, so a missing PPA still reports the PPA error rather
than the superseded check's.
The guards warned in lowercase fragments with no level marker, reading
like debris in the terminal. Capitalized sentences throughout; the
no-bug warning follows the suggested wording and allows for the entry
legitimately fixing no bug; a declined question stops with its own
message instead of repeating the reason just warned above it.
The logger format printed every record as a bare message, so a warning
read like a stray lowercase fragment and an error like plain progress
output. WARN: and ERROR: prefixes, yellow and red on a terminal, bare
when piped; the other levels keep the bare message, which progress
lines read better without.
Stop uploads whose Changed-By is a container or VM identity (ubuntu@,
root@) and uploads of native packages carrying debian/patches/series
(quilt patches never apply there). Question, through the prompter and
defaulting to no, uploads without Launchpad bug references, with
placeholder bug numbers (1234, 9999), with a backport suffix not
matching the target series, and Ubuntu-marked versions whose
Maintainer is not an ubuntu.com address (update-maintainer forgotten).
--force skips the guards, together with the duplicate and superseded
checks it already bypassed.
Two deviations from the hooks: the maintainer check also covers
point-update SRU versions (1ubuntu2.3), which dpkg's \dubuntu\d*$
reading misses, and the ~ppa version hook is left out: pkh put uploads
to PPAs only, where keeping the archive version (an SRU test build) is
a legitimate choice. The archive half belongs with the future archive
uploads.
parse changed_by, maintainer and launchpad_bugs_fixed from the
.changes, and share the source-tree decision with the Section check
(the tree-level guards skip when the upload was collected away from
the tree).
Ubuntu's dpkg inserts the field right after 'Closes' in both the
.changes and the changelog. Emitting it closes the gap with
dpkg-genchanges output and gives the put mistake guards something to
read.
dpkg's Ubuntu vendor module collects 'LP: #NNN' changelog trailers into
the .changes 'Launchpad-Bugs-Fixed' field, which Launchpad uses to link
uploads to the bugs they fix. Port the same parsing (case-insensitive
'lp:' prefix with a mandatory space, '#'-prefixed comma-separated
numbers) and rendering (numeric sort, de-duplication, space-joined) so
pkh-built uploads can carry the same field.
pkh's process carries inheritable descriptors (async runtime, gpgme
sockets, the log tee) that std passes to every child. Tools with
fd-number assumptions then break on the shifted table: mmdebstrap's
apt hook is `cat >&N`, and /bin/sh (dash) rejects any two-digit N
with "Bad fd number" — a fresh-chroot bootstrap died with
"mmdebstrap failed to run" whenever the hook pipe landed above
fd 9, and succeeded from a clean shell where it lands lower.
Sweep fds >= 3 pre-exec in the local driver's spawns: it is the one
funnel every context spawn goes through (schroot and unshare chain
through their parent, ssh execs remotely over libssh2). Verified
end to end: the sid bootstrap test fails on the unfixed code under
a polluted fd table and passes with the sweep under the same table.
The fork announcement, the base selection and the resume note all
described the same thing — building on the previous attempt's tree —
in three consecutive lines. One line now carries the record ids, the
carried/fresh state, and the selection why only when it deviates
from the default pick (a base that did not succeed).
The live view's bar template renders the byte counters after the
bar, so baking them into the message too duplicated them and pushed
the line past the terminal width. Announce the copy as
"Restoring <id>" and leave the counters to the view.
RC tags (a v* tag with a prerelease suffix) now publish nothing at
all — the release job joins the crates.io job behind the same gate;
only final release tags attach artifacts.
The tarball loses the 'orig' qualifier: 'orig' is the Debian side's
name for the imported upstream tarball (gbp/uscan rename it on
import), not something upstream ships. The asset is
pkh-<version>.tar.gz over a versioned pkh-<version>/ prefix, the
conventional upstream shape.
A pushed tag only gets Gitea's automatic tree archive: a snapshot
named after the raw tag, unusable as a Debian orig (the upstream
version may not carry a hyphen — 0.2.0-rc1 parses as upstream 0.2.0,
revision rc1, where the Debian encoding of the prerelease is
0.2.0~rc1) and with no release object to hold attachments. A v* tag
now publishes a release with the dpkg-shaped orig tarball (git
archive, prefixed pkh-<version>/), the packed snap and their
checksums; prerelease tags mark the release as such, and an existing
release is reused on a tag re-push. Still nothing published beyond
this repo's release page.
The audit of the lock against Debian's archive (qa.debian.org
madison, sid/trixie/bookworm) showed 255 of 291 crates packaged, the
36 missing ones all Windows/wasm/apple/redox-only. Making the tree
build against the Debian-packaged crates — the difference between a
vendored source package and a thin debcargo one — needed:
- sha1/sha2/md-5 bumped to the digest 0.11 line (sid moved, 0.10 is
trixie-only): finalize() now returns hybrid-array Output without
LowerHex (hex::encode it) and no io::Write impl (sha256_file reads
chunks into update()). The digest outputs are unchanged; the full
suite including the checksums tests passes.
- crossterm 0.28 -> 0.29: Debian skipped 0.28 entirely.
- reqwest 0.12 -> 0.13: the blocking/json/stream usages carried over
unchanged.
- suppaftp behind an opt-out 'ftp' feature: the one dependency absent
from the archive, used only by the plain-FTP upload fallback of
'pkh put' — the Debian build drops it and the SFTP failure path
reports the transport as unavailable instead.
- every patch-level requirement pin relaxed to the minor level
(4.5.51 -> 4.5, 1.0.228 -> 1.0, ...) — semver-compatible, and what
lets trixie's slightly older crates satisfy the manifest.
Result: 30 of 32 direct dependencies resolve from sid as-is; the
remaining two are indicatif 0.18 (sitting in experimental, migrating)
and the feature-gated suppaftp. No vendoring on sid.
The release-candidate version lands in 'pkh --version' and in the
build records' manifests, so an RC-tested tree is distinguishable
from the final release.
The size cap was the wrong shape: an --incremental user asked for a
carried tree, and the invariant that matters is which trees survive —
the newest successful one (the next run's automatic fork base) and the
newest one (what a failed run is re-forked from explicitly). The cap
is gone, with the tree measurement and manifest field it needed; the
retention planner now works on the records' outcomes alone and fills a
budget: staged_trees staged trees per target, the newer ones first,
the pair always among them. When the budget cannot hold the pair (one
staged tree configured, the newest build failed and an older
successful tree present) only one of the two is successful and that
one is kept; with no successful tree at all, the most recent one is.
The budget is a setting of the nested config file:
deb:
incremental:
staged_trees: 2
(default 2; 0 clamps to 1 with a warning — the newest successful tree
always survives). The fork-base selection no longer truncates the
candidate list to the budget before ranking it: with a budget of one
and a failed newest tree, that truncation hid the successful record
and forked the failed one instead.
The retention kept the newest five staged trees per target, so the
staged trees of a kernel-scale target alone could hold three-digit
GiB. The policy is now the pair that earns its disk: the newest
staged tree (what a failed run is re-forked from explicitly) and the
newest successful one (what the next --incremental run forks
automatically) — one record when they coincide; every other staged
tree degrades to metadata-only at the next build's retention. The
retention decision moved into a pure planner over (success, size) so
the policy is table-testable.
The size cap no longer drops a tree outright: an --incremental run's
own tree is always kept by its own run — the user asked for a carried
tree, and at least one always exists — and the cap only thins the kept
pair while another member covers the resume, the newest successful
tree being the last one standing. Oversized trees left by older pkh
versions or interrupted builds come down at the next build's
retention, once their lock is free.
The cap moves to the first user settings file, config.yml in
~/.config/pkh (YAML, like the data files; missing file or keys mean
defaults, a corrupt file is backed up, PKH_STAGED_MAXSIZE overrides),
replacing the JSON sketch from the previous commit. Default 20G, 0
disables the size rule. Every retained tree's size is measured once at
retention and recorded in the manifest, so sweeps never re-walk the
trees.
The per-target retention kept the newest five staged trees however
large they are: five carried kernel trees are three-digit GiB, most of
it compiled objects whose resume value does not scale with their size.
A staged tree larger than the retention cap (a user setting, see
below) is now dropped at the end of its own build — the record
degrades to metadata-only and the next build re-stages from scratch —
and the retention sweep also degrades older oversized trees, covering
records kept by older pkh versions and interrupted builds. The cap of
0 disables the size rule; the count cap is unchanged.
The cap is the first setting of a user configuration file,
~/.config/pkh/config.json (the directory already holds contexts.json):
missing file and missing keys mean defaults, a corrupt file is backed
up and the defaults apply, and PKH_STAGED_MAXSIZE overrides the file.
The default is 20G, the scale where a carried kernel tree stops
earning its disk.
Every retained tree's size is measured once at retention (the same
concurrent, mount-crossing-safe walker the prune report uses) and
stored in the record manifest, so sweeps reuse it instead of re-walking
the trees on every build.
resolve_pkh preferred target/release/pkh however old it was: after a
commit without a release rebuild, the harness measured the previous
code — the 54-package run recorded before this change exercised the
pre-nested-namespace builds and recorded no phase profiles, the
profile feature itself being newer than that binary. A repo binary is
now rebuilt when it is older than HEAD or than the newest source edit;
an explicit --pkh still pins whatever binary it names.
Rootless builds handed the staged tree to the build user with a
recursive chown; through the overlay staging every lower-layer file
the chown touched copy-ups — the full data movement of the tree, 19%
of a kernel build's wall clock.
The rules steps are now flagged and the unshare driver enters the
chroot by pivot_root instead of chroot(2), then creates a nested user
namespace mapping the context's root onto a fixed build id: the
chroot's files are the invoking user's host-side and the context maps
that user to uid 0, so the staged tree is owned by the build identity
by construction — no chown, no data movement. The kernel refuses user
namespace creation to chrooted processes, hence the pivot, and the
bind is recursive and taken inside the namespace: a plain bind of the
chroot's host-made submounts fails EINVAL on locked children, while
the rbind carries them across the pivot, and a fresh /proc is mounted
before the old root is shed (umount resolves through /proc/mounts).
fakeroot's chown wrappers attempt the real chown and swallow only
EPERM; as the mapped identity, root targets are unmapped and fail
EINVAL, so mapped builds export FAKEROOTDONTTRYCHOWN — fakeroot's own
knob — and the fake ownership is still recorded. Files the build
writes come out as the invoking user host-side, which also fixes the
silently missing .buildinfo/.changes of rootless builds: the host-side
rewrite of debian/files hit a subuid-owned file with EACCES.
The wrapper is probed per build; when nested namespaces are
unavailable (old util-linux, userns disabled), the previous setpriv
plus selective parallel chown handover runs unchanged. A setpriv-era
staged tree is handed to the invoking user once, host-side and
metadata-only, before the overlay mounts it — afterwards overlayfs
serves stale owners from its attribute cache.
The handover chowned the whole build root with a serial 'chown -R'.
Through the overlay staging every lower-layer file it touched is
copy-up'd — the full data movement of the staged tree — which the
profile of a kernel build showed as 15m53s in 'Preparing the build
user', 19% of the whole run.
The handover is now a 'find ! -uid | xargs -P chown' pipeline: only
entries the build user does not own yet are chowned (staging and quilt
re-application flip entries back to root; a resumed tree is already
owned and the phase collapses to a metadata-only walk), and the
copy-ups of independent files parallelize across the chroot's cores.
On a 200k-file tree the handover drops from 9.9s to 0.9s.
The ccache handover ran only when the build user was fresh, which is
every build in practice (the chroot is per-run scratch); it now rides
the same selective pipeline on every run, still best-effort — a failed
handover costs cache hits, never the build.
The pull downloads reported through the plain count port only because
byte_progress landed on main after the branch was cut. PullView now
implements it with a byte-counter template in the transfer-style
family (same shape, {bytes}/{total_bytes} counters, human-formatted by
indicatif), downloads emit through it, and the trait doc lists the
event among the ones the pull flow emits — un-duplicating the merged
'message, progress and progress' sentence.
* cleanup/errors-deps:
apt/release: type the file-verification and armor errors
new: type the scaffold validator errors
debian: type the errors of the format primitives
deps: add thiserror
pull,package_info,ui,cli: report the pull flow via BuildView
deps: trim the tokio feature set to the APIs in use
deps: drop the lazy_static dependency
data,apt/release,distro_info,new: replace lazy_static with LazyLock
deps: drop the unused cmd_lib dependency
# Conflicts:
# src/ui.rs
verify_file returns FileVerifyError (no Release file, unlisted path,
checksum mismatch, size mismatch — the structured pieces carried, so
the by-hash retry in package_info can match on the failure instead of
parsing text) and the in-process armor/base64 decoders return
ArmorError; the gpgv staging and keyring-download paths keep their
diagnostic strings, rendered at the boundary. The existing
VerifyError(String) keeps its name and downcast role for suite
signature verification. Display texts are byte-identical to the old
format! strings.
The leaf validators of the wizard (source name, command, upstream
version, maintainer, homepage, Depends, orig path, template id, file
collisions) now return a shared options::ValidationError carrying the
structured pieces (the rejected string, the wrapped DebianVersion or
Deps parse failure) instead of pre-formatted Strings; the wizard
resolve flow keeps its aggregate String errors and renders at the
boundary, and the Prompter validator closures adapt with to_string()
since the port's Validator deliberately exchanges re-ask explanations
as plain strings. Display texts are unchanged, as are the tests that
assert on them.
Give every parser of the shared format layer a proper error enum
(thiserror): VersionParseError, ArchError, deps::ParseError (wrapping
the version and arch failures as #[source] so the context chain
survives), ControlParseError, ChecksumParseError, FilesParseError and
TestsControlParseError, each carrying its structured pieces (the
offending string, the stanza number, the field name) instead of a
pre-formatted String.
Display texts are byte-identical to the old format! strings, so
user-facing output and the message-asserting tests are unchanged; the
few tests asserting on error text now render the error first. Flow
boundaries that still take String (build::env, quirks, test::expand)
render with to_string() at the edge — typing those flows is a later
step. ControlInfo's FromStr impl follows parse_content's error type.
Retire the ProgressCallback option-of-closure, the one pre-port
reporting idiom left in the tree: pull and package_info::lookup now
take &dyn BuildView like every other core flow, and the announcements
map onto message (unsized work), progress (item counts: series probes,
git objects, extraction) and progress again for byte downloads, which
the old callback carried in the same 4-argument shape.
The terminal side swaps the (ProgressBar, closure) pair for PullView, a
BuildView implementation owning one bar of the shared MultiProgress
with the same spinner/transfer style switching; it releases the line on
drop, which the CLI triggers explicitly on both outcomes since
exit(1) skips destructors. Headless callers (tests, internal lookups)
pass &report::Quiet. git clone sideband parsing is skipped entirely
when the view is not enabled, matching the old None-callback shortcut.
The runtime (multi-thread, for Runtime::new + block_on and task
spawning), time (retries, timeouts), sync (channels) and macros
(#[tokio::test]) cover every tokio:: path in the tree; 'full' also
pulled fs, io, net and process support nothing uses. Cargo.lock loses
the corresponding mio/signal deps if nothing else needs them.
Unify the lazily-initialized statics on std::sync::LazyLock, which the
newer code already uses: the embed_data! macro expands to a plain
static now, which also removes the visibility-forwarding workaround the
lazy_static! matcher forced on it (the @expand arm wrapping vis in
parentheses), and the 'static ref' spelling is kept at the call sites.
distro_info's HTTP client, the apt keyring cache and the new-template
registry follow the same pattern.
Confirming a 500 GiB prune went silent under "Removing:" for minutes:
drive an indicatif bar from the plan's sizes through a new
apply_with_progress observer, in the deb live view's layout — the
running count behind the spinner on the first line, the byte bar with
its counters on the second. The removed path is not rendered: it
fights the bar on a narrow terminal, and the failed-removal errors
name the paths that matter. --force plans no sizes and gets the count
line only.
The dry-run size spinner lacked the "> " prompt prefix the other
one-line waits carry; wait_spinner renders it too now.
Forking an incremental base copies its whole staged tree — bounded,
and without reflinks a matter of minutes — but the copy emitted no
build events: the live view showed the target line with nothing below
it, and an interrupt there printed raw cp errors with no hint of what
was running (the fork happens before the first bootstrap phase owns
the view).
Announce the copy through a new BuildView::byte_progress port. The
fork sizes the source up front (the copy's total), re-measures the
growing destination from a scoped thread while cp runs (cp has no
progress output to tap), and reports the completed copy when it
returns; unmeasurable or empty sources copy unreported. The terminal
view renders it like the other determinate bars, with byte-unit
counters.
The incremental fork's staged-tree copy wants to report byte progress
against the same measurements the prune report is built from: move the
concurrent size walk and the human-readable byte formatting out of
prune into utils::size and rewire prune to them. Behavior is unchanged:
same-device walks, unreadable entries count as zero.
The shared apt download cache is bind-mounted from outside the
chroot, so when an apt of ANOTHER build holds the archives lock from
its own pid namespace, a concurrent build's apt sees the holder as
"process 0", cannot wait for it, and fails the build immediately
(observed as an e2e samba essentials install racing a kernel build's
build-deps transaction on the stonking-amd64 cache).
Set DPkg::Lock::Timeout to ten minutes in the shared-cache apt conf:
apt then waits for the holder instead of erroring. pkh itself
already serializes builds per cache with a host-side flock; the
timeout covers the window where a killed run's orphaned apt is still
winding down.
- -f/--force skips the size pass and the confirmation prompt: remove
immediately, reporting counts but no sizes. The fastest cleanup
path; --dry-run still previews.
- The default path shows the grouped digest and asks before removing
(the raw-mode yes/no prompt; non-interactive sessions keep their
headless path, so scripts reach for --force).
- Sizes are computed under a spinner ("Computing sizes of
directories to prune..."), and the digest colors the counts (bold)
and the sizes (cyan) on terminals only.
The dry run still spammed one info line per artifact before the
grouped digest, and the size walk was serial: tens of seconds of pure
readdir/stat over the records' hundreds of thousands of files.
The per-path lines drop to debug (the grouped report is the
user-facing output), and the size pass now fans out over a shared
work queue with scoped threads — directory-level parallelism, same
du -x mount-crossing rule. 16s to 6.4s wall on a 440 GiB sessions
tree, totals unchanged.
The prune output spammed one line per path. The report now groups
entries by artifact kind with their total size, and the size is
measured before removal whatever the mode, so --dry-run prints a
digest ("24 residual build directories for 1.2 GiB") and both modes
end with the freed total. The size walk never crosses a mount point:
residual chroots hold bind-mounted /proc trees whose virtual files
(/proc/kcore alone reports 128 TiB) would poison the total.
--all is also honest now: it removes recent build records and staged
trees (that is where the space is), and the flag description says so.
One behavior fix rides along: a record or lock a live build holds was
previously prunable with --all, which would wreck a running build;
held locks are now out of bounds whatever the policy.
A rootless build's upperdir entries are copy-up'd by whoever wrote
them during the build: the unprivileged build user, which host-side
is a subuid of the invoking user. The host-side sync of a forked
record removes the stale upper copy of every host-modified file, and
the plain fs removals failed with EPERM on those build-user-owned
entries: the kernel incremental run died with a bare
'Permission denied (os error 13)' and no context, ten minutes into
the run.
Route the sync removals through the privileged path (the same sudo
fallback the record teardowns use) and wrap the sync error at the
call site, so a failure names the operation instead of surfacing a
bare errno string.
The specs describe shipped features — incremental builds, the live
view, overlayfs staging, pkh new, pkh test, pkh lint phase 1 — and
the satellite-tools plan is implemented in full (no shell-outs to
dpkg-architecture, dpkg-checkbuilddeps or dpkg-parsechangelog
remain). The two unshipped proposals (the lint porting track and the
data/configuration architecture) stay in whatever notes the project
keeps outside the tree; the README remains the user-facing
documentation.
Plain-speech pass: drop the em-dash asides and mid-sentence colons,
reword the boldface digest constructions ("What --incremental buys
is ..."), and replace the fluffy intro ("aims at wrapping ... while
being smarter at integrating all workflows") with what pkh actually
does. No content changes.
The profile's build-dependency phase spanned everything between the
apt passes and the build step, and for a kernel incremental build
that hid ~13 minutes of housekeeping: the recursive chown handing
the staged tree to the build user, and the resume_clear quirk
dropping the previous attempt's stamp caches. Both now enter the
profiler (ClearingResumeCaches only when the quirk has paths and a
tree is carried, PreparingBuildUser only for rootless builds), so
the status bar and pkh deb profile show where the time actually
goes — on a large carried tree the chown dominates, which is the
honest answer to a 6m30s DEPS column next to a 19m19s phase.
A rules chown failing for the unprivileged build user is expected
and harmless (see the revert of the build-step fakeroot wrap), but a
sysroot tree can print thousands of 'chown: changing ownership of
...: Operation not permitted' lines. MakeClassifier now shows one
warning digest for the flood and hides the rest; every other line,
make errors included, keeps its classification. The full tee in the
record's build.log is untouched.