Compare commits

..
57 Commits
Author SHA1 Message Date
vhaudiquet 6df490cf9a ci: install build-essential in the publish job
CI / build (push) Successful in 3m3s
CI / test (push) Skipped
CI / snap (push) Successful in 6m9s
CI / publish (push) Successful in 2m11s
cargo publish verifies the packaged tarball with a full build before
uploading, and that build needs a C linker for build-script crates
(proc-macro2 was the first to fail). The job's container setup step
omitted build-essential, unlike the build and test jobs, so the
verification failed with 'linker cc not found' on the v0.1.0 tag.
2026-09-24 21:41:15 +02:00
vhaudiquet f5b7704647 data: add an agent SKILL.md teaching pkh usage
CI / build (push) Successful in 3m1s
CI / test (push) Skipped
CI / snap (push) Successful in 6m1s
CI / publish (push) Failing after 53s
Add data/skill/SKILL.md in the agent-skills open format: a skill
directory that agents (Claude Code, Codex, OpenCode, ...) discover and
load on demand. It documents the shared option surface, the pull,
chlog, build, deb, lint, put workflow and the flags that keep runs
non-interactive; the command reference was generated from the live
--help output of every subcommand.

It lives under data/ so a later module can embed it and ship it with
the binary, for example a 'pkh skill' installer writing it into the
agent skill directories.
2026-09-22 23:55:56 +02:00
vhaudiquet 4c1edc7dcd docs: update example gif
CI / build (push) Successful in 3m13s
CI / test (push) Skipped
CI / publish (push) Skipped
CI / snap (push) Successful in 6m7s
2026-09-22 21:35:35 +02:00
vhaudiquet 9a66f8f7df deb: retry the interrupt chroot removal while children die off
dpkg defers SIGINT until it reaches a safe state, so it can still be
writing into the chroot when the watchdog's rm -rf starts racing
through it, failing with "directory not empty" and leaving the tree
half-removed. Retry the removal for a few seconds while the
interrupted children finish dying off; a genuinely stuck tree still
ends in the pkh prune message.
2026-09-22 19:57:01 +02:00
vhaudiquet 7a6337e1cb deb: build the tree the caller pointed at before the name search
CI / build (push) Successful in 3m16s
CI / test (push) Skipped
CI / publish (push) Skipped
CI / snap (push) Failing after 7m17s
build_binary_package stages the parent of the requested cwd, then
re-derived the package directory inside the staging area from
package/version name patterns plus the calling process's working
directory. That only works by accident for interactive users sitting
in the package directory: an embedded caller whose tree lives at
<job>/tree matches no pattern, and the process cwd means nothing to
a library consumer — the bc build above failed here even though the
tree was staged correctly.

The pointed-at tree is authoritative anyway: its changelog defined
the package, version and series for this build. Resolve its staged
copy outright when it carries a debian/ tree, keep the pattern
search (with the quirks overrides) as a fallback, and hand the
resolved directory to local::build instead of searching again.
2026-09-22 14:31:25 +02:00
vhaudiquet 9186bbbe51 deb: classify the staged listing through the context
The fallback listing of find_package_directory called Path::is_dir
on entries returned by list_files — a host-side stat. For an unshare
context (every local build) those paths are rooted inside the chroot
and do not exist at the same host path, so every entry came out a
non-directory, the 'Found directories' list silently stayed empty
and the failure degraded to the list-less 'Could not find package
directory' variant, hiding the actual layout (seen building bc from
an ubuntu/devel checkout whose staged tree was named 'tree').

Classify entries through the new ContextDriver::is_dir, and log
every entry instead of only those matching the broken host stat.
2026-09-22 14:31:10 +02:00
vhaudiquet 681fa3d687 context: add ContextDriver::is_dir
list_files returns context-relative paths (rooted inside the chroot
for an unshare context, on the remote for ssh): whether an entry is
a directory can only be decided through the context, never with a
host-side stat. Give every driver a proper is_dir rather than
approximating it with exists, so callers can keep directories and
files apart — the deb package-directory search lists directories
only.
2026-09-22 14:29:12 +02:00
vhaudiquet cc5bbd2297 chlog: drop a trailing buildN before appending ubuntu1
An Ubuntu upload of a package sitting at X-2build1 produced
X-2build1ubuntu1: the blind append misrepresents the lineage and,
sorting below the proper X-2ubuntu1, could never supersede it. A real
change on top of a rebuild replaces the marker instead, so the
trailing buildN is now stripped before the ubuntu counter is appended
or incremented: X-2build1 becomes X-2ubuntu1, X-2ubuntu1build1
becomes X-2ubuntu2.
2026-09-22 13:00:40 +02:00
vhaudiquet 5a1c1672cd cli: intercept Ctrl+C for deb, build and put
The SIGINT handler only records the interruption and wakes a watchdog
through a self-pipe (async-signal-safe); the watchdog runs the whole
shutdown in thread context — the live view's reporter first, then the
notice and the log hint, then the cleanup hooks, then exit 130. Flows
park in wait_for_shutdown instead of racing it with their own exit,
and an end-to-end test drives the sequence by re-spawning the test
binary and raising SIGINT at itself.
2026-09-22 10:44:05 +02:00
vhaudiquet adde0ee977 deb,ui: tear the live view down through the interrupt core
The ephemeral guard registers its chroot removal as an interrupt
cleanup hook and, once interrupted, stands down from its own teardown
so the two cannot race umount/rm; bootstrap bails out of tarball
extraction and the lockfile wait, keeping the hook registered on the
bootstrap error path so the watchdog can remove the partial tree. The
live view registers an interrupt reporter that suspends the widget and
returns the log-file hint, silences the tty rendering of the ^C
keypress (the echoed "^C" can wrap near the right edge and shift the
teardown erase by a row, leaving the first widget line on screen) and
kills the shared draw target so late log records cannot repaint the
cleared bars. Failure summaries stay quiet when interrupted: the
captured errors are just the killed children's death throes, and the
dose-builddebcheck diagnosis is skipped for a dependency failure the
user interrupted themselves.
2026-09-22 10:43:58 +02:00
vhaudiquet 84405a6762 interrupt: add the passive interrupt core
The library holds only the state its own types need when a Ctrl+C
arrives: the interrupted flag flows check to stand down, the cleanup
hook registry for resources that must not outlive the process (the
ephemeral build chroot), and the live view's reporter slot. The
signal handling itself is CLI wiring and lands separately: nothing
here installs handlers, prints or exits, so a library consumer
embedding these types keeps its own signal disposition.
2026-09-22 10:43:49 +02:00
vhaudiquet 8250a0e3b1 ci: publish the crate to crates.io on a v* tag
CI / build (push) Successful in 3m15s
CI / test (push) Skipped
CI / publish (push) Skipped
CI / snap (push) Failing after 7m34s
Trusted publishing is GitHub-Actions-only, so authentication goes
through a crates.io API token stored as the CARGO_REGISTRY_TOKEN
secret, scoped to the pkh crate. The job gates on the build job and
fails loudly when the tag does not match the version in Cargo.toml,
since cargo publish ships the declared version regardless of the
tag name.
2026-09-21 23:14:27 +02:00
vhaudiquet edfd7ed5ed snap: declare the dual license and ship the license texts
CI / build (push) Successful in 3m11s
CI / test (push) Skipped
CI / snap (push) Failing after 7m18s
The store listing carries the crate's MIT OR GPL-2.0-only
expression, and both texts ride along in the snap under
/usr/share/doc/pkh: the MIT grant requires the notice to
accompany copies, and GPL-2 requires the license text with
distribution.
2026-09-21 22:41:09 +02:00
vhaudiquet 6545d327e4 docs: add example gif
CI / build (push) Successful in 3m13s
CI / test (push) Skipped
CI / snap (push) Failing after 7m23s
2026-09-21 21:33:52 +02:00
vhaudiquet b118a54bff deps: describe the crate and dual-license it MIT or GPL-2.0-only
CI / build (push) Successful in 3m2s
CI / test (push) Skipped
CI / snap (push) Failing after 7m15s
Publishing to crates.io requires a description and a license; the
repository and readme give the crates.io page the right links. The
license texts ship as LICENSE-MIT and LICENSE-GPL.

Dual licensing is valid while no lintian-derived code is in tree:
shelling out to lintian is mere aggregation. When derived lint
collections are implemented they must land in a separate GPL-2-only
crate, since their copyright belongs to lintian's authors and
cannot carry an MIT grant; the core crate stays dual.
2026-09-21 20:46:46 +02:00
vhaudiquet 43f8a9e275 docs: remove roadmap, consolidate README
CI / build (push) Successful in 3m2s
CI / test (push) Skipped
CI / snap (push) Successful in 6m9s
2026-09-21 18:20:26 +02:00
vhaudiquet ff41edbd47 ui: create the rolling pane lazily, on its first line
DebUi seeded both widgets with a "(starting...)" placeholder,
replaced as soon as real content arrived during a build. pkh put
reports through the same view but runs no subprocess, so nothing
ever fed the pane: its seed line stayed on screen for the whole
upload, stacked under the per-file byte progress.

Drop the seeds and add the pane bar to the terminal only when the
first classified line arrives; a phase change (and the final
suspend) takes it off again. Flows without subprocess output now
render the status bar alone.
2026-09-21 15:20:17 +02:00
vhaudiquet 9e0b6a37a6 put: degrade to the anonymous FTP queue when the SSH transport fails
CI / test (push) Skipped
CI / build (push) Successful in 3m0s
CI / snap (push) Successful in 6m6s
pkh put only spoke SFTP to the PPA queue, so a failure of the SSH
transport itself (TCP, banner exchange) failed the upload even though
dput happily pushes the same files: its plain ppa: profile goes over
the anonymous FTP queue of ppa.launchpad.net, the same destination
over another port.

Classify the SSH connection failures: Transport (the connection never
came up: resolution, TCP, banner or key exchange) degrades to that FTP
queue — the upload order (payload first, .changes last), the
reverse-order DELE cleanup of a failed upload and the per-chunk
progress reporting all mirror the SFTP path, sharing cleanup_list.
Refused failures (host key not accepted, no matching authentication)
stay errors: silently switching transport would bypass the refusal.

The FTP client is suppaftp's blocking stream, with the time bounds it
does not carry by itself: the control channel's reads and writes, the
data channel's writes and connect (through a custom passive stream
builder), and the NAT workaround for PASV replies announcing an
unroutable address. The queue endpoints (host, port) join
data/launchpad.yml next to the SFTP ones, and the FTP transport is
covered by unit tests against an in-process fake queue plus a live
control-channel handshake with the real server (ignored, network).
2026-09-21 14:34:04 +02:00
vhaudiquet ac19fd9d65 deps: add suppaftp and parking_lot
suppaftp 12 is the FTP client behind the put FTP fallback transport:
the maintained continuation of rust-ftp (4.2M downloads, releases this
month), used with its default features only — a plain blocking FTP
stream, no TLS, no async. It brings just lazy-regex into the tree;
chrono is shared.

parking_lot replaces std sync mutex unwrapping in test code, per the
project rule.
2026-09-21 14:34:04 +02:00
vhaudiquet 02e1f739c3 chlog,cli,docs: offer suite-aliased series in the chlog selector
The chlog series selector only appeared when the changelog's current
distribution resolved to a known series; a Debian package targeting
'unstable' (or 'stable', 'testing', ...) fell through to keeping the
current series, silently, with no menu.

Resolve the changelog distribution through the suite aliases first
(unstable identifies the same series as sid, which resolves to the
Debian series list). The selector offers an aliased series as
'<suite> (<series>)' — 'unstable (sid)' — preselected, but selects
the suite name: what a changelog distribution field expects, instead
of the codename. Every other label and free-typed input selects
itself, unchanged.

Reflect the selector in the README roadmap checklist.
2026-09-21 11:52:39 +02:00
vhaudiquet 8c6f6f4028 distro_info: match changelog suite names with their series
Debian packages conventionally target 'unstable' in their
debian/changelog distribution field, but the series data (the
distro-info CSVs) only knows codenames: the suite is the alias
'unstable' of the series 'sid', a mapping the debian-distro-info tool
resolves internally without exposing it in its data.

Add a per-dist suite_aliases reference-data key (debian: unstable ->
sid), with two helpers on top: resolve_suite_alias, identifying a
changelog suite name with its series codename and the dist that
codename belongs to, and series_suite_alias, the inverse direction.
The two names identify the same series.
2026-09-21 11:52:39 +02:00
vhaudiquet 37e0b5c978 pull: fetch every component tarball of multi-orig packages
Sources listed with "3.0 (quilt)" extra components (node-jest, php-*,
...) carry one tarball per bundled module next to the main orig, named
<package>_<uver>.orig-<component>.tar.<ext>. fetch_orig_tarball picked
the single file matching ".orig.tar." — which cannot even match the
component naming — so a git pull only fetched the main orig. The later
dpkg-source -b quilt verification then failed with "can't find file to
patch" on the first patch touching a component directory.

Select the files with the existing build::changes::is_orig_tarball
helper (mirroring dpkg's \.orig(-.+)?\.tar\. strip pattern) and fetch
all of them, pristine-tar checkout first with a checksummed archive
download fallback, per tarball.

The end-to-end test now asserts every stanza-listed orig lands in the
package dir instead of just any *.orig.tar.* file, and gains a
node-jest (trixie, 24 components) regression case.

Verified live: pkh pull node-jest -d debian fetches all 15 origs of the
sid ds7 repack, and dpkg-source -b builds the debian.tar.xz and dsc
without touching the series.
2026-09-21 11:28:55 +02:00
vhaudiquet 6c0b200241 deps: commit the Cargo.lock
A binary crate should pin its dependency graph: without the lockfile
in git, source and snap builds float transitive versions, so a
0.1.0 artifact rebuilt later would not be the same binary.
2026-09-21 01:36:40 +02:00
vhaudiquet 4edf331444 docs: refresh command list, workflow example, roadmap and install
The command block is now the actual pkh --help output (new, lint and
prune were missing). The example workflow used pkh commit, a
subcommand that does not exist; commits go through git until
chlog/pkh commit land. The roadmap now reflects what is implemented
(pull -v, deb --mode local, lint, prune, new) and an installation
section documents the source build and its system dependencies.
2026-09-21 01:36:39 +02:00
vhaudiquet fa121f08ec cli: expose --version on the root command
The flag was disabled in the initial commit, leaving the binary with
no way to report its number — wrong for a release. clap scopes the
automatic version flag to the root command (-V/--version), so the
per-subcommand -v target-version options of pull and chlog are
unaffected.
2026-09-21 01:36:39 +02:00
vhaudiquet ac83a939e3 snap: confine classically and carry the full packaging toolchain
CI / build (push) Successful in 2m50s
CI / test (push) Skipped
CI / snap (push) Successful in 5m55s
A devmode snap is a smoke test, not a distribution channel: pkh
drives the whole host packaging stack (unshare chroots, overlay
mounts, dpkg/quilt/lintian across arbitrary paths), which only
classic confinement can express.

The snap now bundles every host-side tool pkh execs (git, gnupg,
dpkg-dev, quilt, pristine-tar, mmdebstrap, lintian, fakeroot,
util-linux, mount, schroot, openssh, tar/xz/bzip2), with apt and
dpkg deliberately left to the host: a core24 apt managing a newer
host's package database is exactly the skew classic snaps must
avoid. Tools running only inside the build chroot stay out; pkh
provisions those itself.

Release metadata comes from Cargo.toml instead of the git hash, and
grade is stable, so a build of any commit packs as the declared
version.

Classic-mode correctness: noble's mount/umount are staged from the
split mount package, fakeroot is exposed via symlink since
update-alternatives does not run at staging, and every bundled ELF
is patched to the core24 loader with a DT_RPATH resolving the base
and $ORIGIN. Without this the host loader would pin the snap to
hosts with a matching glibc, and the host ld.so.cache would mix
host libraries with base ones.
2026-09-21 01:22:10 +02:00
vhaudiquet 1145ca55eb docs: drop the pkh context roadmap items
The subcommand is gone from the CLI; context survives only as the
internal build backend, so there is no user-facing surface left to
track on the roadmap.
2026-09-21 00:10:45 +02:00
vhaudiquet b99f945b98 cli: remove the pkh context subcommand
The context management interface never worked reliably, and keeping
it exposed presents a feature that is not ready. Contexts remain in
the library as the execution backend for pkh deb (unshare chroots
and friends); only the CLI surface goes.
2026-09-21 00:10:45 +02:00
vhaudiquet ae5b0042e4 context: name temp dirs atomically, not probe-then-create
CI / build (push) Successful in 2m58s
CI / test (push) Skipped
CI / snap (push) Successful in 4m39s
create_temp_dir probed for a free pkh-<seconds> name and then created
the directory, so two contexts arriving together could both observe a
free name and unpack into the same directory — observed as two e2e
tests started within the same second failing on 'File exists when
hard linking' during the chroot tarball unpack.

Name with sub-second precision and create atomically: a losing race
gets AlreadyExists and falls through to the next attempt, which
removes the probe window instead of narrowing it. The schroot and
ssh drivers already use mktemp -d and need no change.
2026-09-20 19:47:24 +02:00
vhaudiquet 5b0cc08d8f deb,quirks: scope deb entries per series with an entry list
One deb block per package cannot express two series needing different
rules. Make pull and deb lists of entries, each with its own series
scope: every matching entry applies, in file order, so a stonking
entry can carry its own dependency rules next to the resolute one.
Thread the series through find_package_directory for the
package_directory lookup, which keeps its deb-then-pull fallback.
2026-09-20 19:40:19 +02:00
vhaudiquet ff1f8c7ccd deb,quirks: generic dependency rules for build-dep workarounds
CI / build (push) Successful in 3m3s
CI / test (push) Skipped
CI / snap (push) Successful in 4m38s
The resolute linux and linux-riscv controls declare llvm-21-dev
unqualified while their other llvm pieces are :native, so the dpkg
cross rules resolve it against the host architecture — whose
dependency closure needs python3:riscv64, conflicting with the
python3 the control itself declares :native. No resolver can install
that set; sbuild fails on it identically.

Give the deb quirks a dependencies rule set rather than a one-off
native-qualification knob: replace rewrites a declared dependency by
name with a full dependency string (qualifier, version and
restrictions included), inject adds dependencies resolved as if
declared, drop ignores declared ones. Entries are scoped by series so
they can be dropped when the upstream packaging catches up, and rule
names that match nothing are warned about so stale quirks surface.

Ship the llvm-21-dev entry for the resolute kernels: resolve it as
llvm-21-dev:native, keeping the declared <!stage1> restriction.
2026-09-20 19:10:06 +02:00
vhaudiquet 31fe6dc524 context: run commands in the C locale by default
Subprocesses inherit the session environment, so a translated host
locale leaked into builds: perl-based packaging tools
(dpkg-architecture, dpkg-parsechangelog, quilt, ...) warned about
missing locale settings, change their output with the environment,
and dpkg-buildpackage treats some of that output as data.

Default every command to LANG=C and LC_ALL=C; a caller can still
override explicitly through envs().
2026-09-20 19:06:04 +02:00
vhaudiquet 5592d5a6e4 test: cross-build linux-riscv from resolute, noble known-broken
The e2e pulled linux-riscv from questing, which went EOL in July
2026 and has since been removed from the mirrors entirely — not even
old-releases carries a Release file for it, so the chroot bootstrap
cannot resolve the series and the e2e can no longer run.

Point it at resolute (26.04), the current LTS: linux-riscv is
packaged there like in every supported series, and an LTS stays
pullable for years where an interim series turns the test stale
within months.

Keep a noble fixture next to it, also not run by default: noble-era
controls declare their build tools unqualified (the :native idiom
landed in later series), so exact dpkg semantics demand
host-architecture instances of them (python3:riscv64, gcc-13:riscv64,
...), and the two-architecture install set is unsolvable — t64
libraries conflict with their own foreign-arch variant, and the
riscv64 toolchain instances drag depends chains that do not resolve
from the chroot sources.
2026-09-20 19:05:59 +02:00
vhaudiquet bb719e7c80 apt/keyring: keep the keyring cache readable
A fresh chroot download failed with mmdebstrap unable to copy the
keyrings: its unshare-mode hooks run under an identity that cannot
read the invoking user's private directories, so the 0700 cache
directory the module created broke the trusted.gpg.d setup hook with
'Permission denied' (builds reusing an already-cached tarball never
hit the path).

Create the cache 0755 and chmod the keyrings 0644 instead. The
planting guard does not weaken: cached keyrings are trusted as-is,
and validate_keyring_dir keeps refusing directories that are not
owned by the current user or are writable by group or others — read
access for the bootstrap tool is not a planting vector. Legacy 0700
cache directories are validated as before, then widened.
2026-09-20 18:14:28 +02:00
vhaudiquet af9845c480 deb: install build-deps per dpkg cross semantics
Replace the apt-get build-dep passes with a native resolver: the
Build-* clauses are evaluated against the context's package state by
crate::debian::deps (the dpkg-checkbuilddeps equivalent), and the
unsatisfied ones install through explicitly architecture-qualified
names.

apt's --host-architecture build-dep resolution is coarser than the
checker it feeds: unqualified Multi-Arch: same libraries only ever
get the host-architecture variant, and there is no way to also
satisfy the build-architecture needs of a cross build without a
native re-resolution pass — which installs co-install partners that
break unpacking for -dev packages whose variants conflict on
arch-differing files (curl-config; the libcurl4-gnutls-dev
regression). The resolver now applies the same dpkg rules as the
checker, preferring the runnable build-architecture variant of
Multi-Arch: foreign tools, and passes virtual names through to apt.

Build-Conflicts are now checked before anything installs, which the
build-dep passes never did, and a failed dose3 diagnosis no longer
masks the resolver error (a latent flaw of the passes: binary-only
builds have no .dsc for dose-builddebcheck to read).

An end-to-end test pins the expected failure: a cross build whose
build-dependencies cannot satisfy dpkg's cross semantics aborts,
naming the dependency.
2026-09-20 18:04:13 +02:00
vhaudiquet 02cb5306f3 debian: test the cross build-dep lookup against dpkg
The Facts lookup already implemented Dpkg::Deps::KnownFacts
_find_package, but nothing pinned its cross-compilation behaviour:
with host != build, an unqualified dependency resolves against the
HOST architecture instance (or any instance of a Multi-Arch: foreign
package, or an Architecture: all one) and never against the
build-architecture instance of a Multi-Arch: no/same package.

Cover the matrix with unit tests and, like the source-build checker,
with a differential sweep against real dpkg-checkbuilddeps on a
synthetic admindir with -a <host>: the checker and the real tool
agree on every fixture, including the quirks (:native aborting on
Multi-Arch: foreign instances, first-match version binding).
2026-09-20 18:04:05 +02:00
vhaudiquet 5c026a7050 docs: add AGENTS.md with the tree conventions
CI / build (push) Successful in 3m12s
CI / test (push) Skipped
CI / snap (push) Successful in 4m32s
Codify what the history already does so agents and new contributors do
not have to reverse-engineer it: the fmt/clippy gates mirroring CI's
-Dwarnings, the <scope>: <summary> commit format with the per-module
scope table, and the library-side invariants (missing_docs, report
ports, data/*.yml embeds, README parity for user-facing changes).
2026-09-20 16:30:15 +02:00
vhaudiquet 9238aa961f chlog: fall back to the changelog history when no version tag exists
CI / build (push) Successful in 2m56s
CI / test (push) Skipped
CI / snap (push) Successful in 4m30s
get_commits_since_version silently returned an empty change list when
the previous version carried no tag. Walk the history back to the last
commit that modified the changelog itself and use it as the boundary
instead, so entries stay correct in repositories that commit their
changelogs without tagging them.

Tag detection remains the preferred path. Before falling back to the
changelog commit, the version recorded by the committed changelog is
probed for a tag: an uncommitted newer entry on top (e.g. UNRELEASED
from a previous run) does not hide the previous version's tag.
2026-09-20 00:38:15 +02:00
vhaudiquet d1056fbbbf ui: ellipsize fake-terminal pane lines wider than the terminal
Overflowing log lines in the pkh deb / pkh build rolling pane used to
wrap onto a second line, corrupting the pane layout. Truncate them to
the terminal width (minus the pane prefix) by display width and append
an ellipsis instead; lines are left whole when the terminal size is
unknown.
2026-09-20 00:15:37 +02:00
vhaudiquet a0e74073bf lint: add pkh lint, wrapping lintian for parity plus pkh-native checks
CI / build (push) Successful in 2m55s
CI / test (push) Skipped
CI / snap (push) Successful in 4m32s
pkh covered the package lifecycle but never validated the packaging
itself: broken control stanzas, unparsable changelog versions or
uncommitted debian/ edits only surfaced at build or upload time. pkh
lint lints a source tree with day-one lintian parity plus a native Rust
engine for the checks lintian cannot have.

The wrapper reuses the pkh build output next to the tree when it matches
the current changelog entry and no tree content is newer (mtime walk,
skipping .git/.pc), else packs fresh with dpkg-source -b using weak gzip
compression (the artifact is ephemeral; xz dominated the run at 9.8 s
versus 2.7 s on a 111 MB tree) and symlinks quilt orig tarballs from the
tree's parent, which dpkg-source searches in cwd. Findings are parsed
from the installed lintian into a unified report, deduplicated by tag
name against the native engine, and rendered lintian-shaped
(<L>: <pkg> <type>: <tag> <details>) as text or JSON, colorized at
render time (--color auto/always/never). Exit codes follow lintian's
contract (0 clean, 1 findings at/above --fail-on, 2 runtime error);
lintian's own exit code is ignored because it uses 2 both for findings
and for runtime errors. -d/--dist maps to lintian --profile so the
target distro's rules apply even on a foreign host.

The native engine hosts the first workflow check lintian cannot know:
pkh-debian-changes-not-committed flags debian/ content that is not
committed to git, since the pkh flow builds and uploads the tree as-is.
Checks register in a static registry validated by a unit test, and the
wrapper's parser is pinned by golden tests captured from lintian 2.129
output. Strategies for lintian's Ubuntu blind spots (its vendor data
there is one file plus 14 disabled tags) are specced in
plans/pkh-lint.md, deliberately not implemented yet.
2026-09-19 23:45:31 +02:00
vhaudiquet 4f5246ccd3 chlog: number Ubuntu backports with the per-release SRU scheme
CI / build (push) Successful in 2m56s
CI / test (push) Skipped
CI / snap (push) Successful in 4m35s
--backport was Debian-only: ~bpo is backports.debian.org's scheme and
its number the Debian release, so Ubuntu targets were rejected outright.
Ubuntu backports have their own documented scheme (Ubuntu version-
strings): the development release's version with a per-release ~YY.MM.1
appended, sorting before it (3.1-1ubuntu2 backported to 22.04 becomes
3.1-1ubuntu2~22.04.1; native 3.1 becomes 3.1~22.04.1) and independent of
the version the target release carries. The .N increments for
subsequent per-release SRU uploads.

backport_series_number becomes backport_suffix_for_series: the release
number comes from the new generic get_series_release_number (version
column of the target series' own distro-info data, leading token kept —
"12" for bookworm, "26.04" out of resolute's "26.04 LTS"; empty column
as on sid/experimental means None), and the suffix is picked per vendor:
~bpoNN+ for Debian (plain integer releases only), ~YY.MM. for Ubuntu.
Unnumbered series still error before anything is written.

Also serialize the changelog tests that mutate the process-global
DEBFULLNAME/DEBEMAIL variables behind a tokio Mutex: run in parallel
they raced each other's identity reads, which started failing
intermittently as generate_entry tests accumulated.
2026-09-19 21:32:02 +02:00
vhaudiquet dd2438a72c chlog: number regular uploads after the target series' vendor
A flagless entry (no --backport/--nmu/--rebuild) targeting an Ubuntu
series was numbered the Debian way: 1.0-1 became 1.0-2 with distribution
noble, and getting the conventional 1.0-1ubuntu1 required hand-editing
the version. The bump now derives from the vendor of the target series
(distro-info): Ubuntu series get the ubuntu suffix convention (1.0-1
becomes 1.0-1ubuntu1, and re-bumping an already-Ubuntu changelog
increments the counter instead of the revision), Debian series keep the
plain revision bump, and series that cannot be resolved to a vendor
(UNRELEASED, unknown) fall back to it too. There is no reverse sync, so
the Ubuntu-to-Debian direction needs no special casing.

Runs with an explicit --series now consult distro-info once, where they
previously queried it not at all; the interactive flow already did for
the series selector. EntryKind::Ubuntu remains the library-level way to
force the numbering regardless of the series.
2026-09-19 20:58:44 +02:00
vhaudiquet 012df20961 chlog: number entries as backport, NMU or no-change rebuild
The version flags were never reachable: --backport was declared but not
read, and compute_new_version's NMU/rebuild numbering sat behind a TODO
asking for CLI wiring (the old positional-bool signature always received
false). generate_entry now takes an EntryKind selected by three mutually
exclusive flags, and is async because the backport numbering derives the
Debian release number of the target series from distro-info:

- --backport: 1.0-1 becomes 1.0-1~bpo12+1 (12 = release number of the
  target series, backport suite names accepted too). Re-running on an
  already-numbered version bumps the counter; series without a numeric
  Debian release (sid, Ubuntu series, UNRELEASED) are rejected before
  anything is written.
- --nmu: 1.0-1 becomes 1.0-1.1 (native 1.0 becomes 1.0+nmu1).
- --rebuild: 1.0-1 becomes 1.0-1build1.

An explicit --version overrides all three. compute_new_version went
from four positional bools to a private Bump enum; backport numbering
reuses increment_suffix with a '~bpoNN+' suffix. The CLI prints the
computed new version before opening the editor.
2026-09-19 20:56:03 +02:00
vhaudiquet 47bb7c608e deb: resolve cross pkg-config against the target multiarch
CI / build (push) Successful in 2m54s
CI / test (push) Skipped
CI / snap (push) Successful in 4m32s
In-tree tools locate their libraries with the *host* pkg-config during
cross builds (the kernel's tools/build feature checks derive their
cflags/ldflags from 'pkg-config --cflags/--libs'), whose search path
only covers the build architecture's pkgconfig dirs. Cross builds of
linux-riscv died in rtla's Makefile.config: libtraceevent/libtracefs
were reported missing although the riscv64 -dev packages were
installed, because pkg-config never saw their .pc files.

The host-arch -dev packages that used to make those checks pass came
from the unscoped arch-indep build-dep pass, whose native
re-resolution b34e86d correctly scoped to the host arch — removing the
accidental co-install along with the bug it papered over. Export
PKG_CONFIG_LIBDIR for the target multiarch instead, so the checks
resolve target-arch libraries directly: no native build-dep bloat and
no wrong-arch linking.
2026-09-19 13:07:46 +02:00
vhaudiquet 6caedce61a report: reproduce the pre-refactor CLI output through the ports
Instead of carrying raw UI in core, the ports now represent everything
the CLI used to do inline:

- Prompter::present shows context outside of a question (the wizard
  summary screen, the vendoring notice spacing); TerminalPrompter
  prints it on stdout exactly like the println!s it replaces, server
  embeds forward it as a display event.
- generate_entry returns the generated entry (package, versions,
  series, path) instead of printing; the CLI renders the same lines.
- BuildTarget carries a flow-composed display line and a tee_log flag:
  the terminal adapter renders it verbatim ("Building source package
  ...", "Building ... for series/arch", "Uploading ... to ...") and
  uploads open no build log.
- The unmet build-dependency diagnostics are rendered by the CLI from
  the typed error, in the original order (details, then summary).
- --verbose constructs no live view at all (an idle widget used to
  linger), and the re-vendor offer only logs when it is actually
  asked, so headless runs print the error exactly once.
2026-09-19 00:52:31 +02:00
vhaudiquet bd8f814a53 Revert "package_info: drop the ANSI color from the not-found warning"
This reverts commit 4fae02bc85.
2026-09-19 00:39:29 +02:00
vhaudiquet 4fae02bc85 package_info: drop the ANSI color from the not-found warning
The location embedded in the log record was styled with crossterm: a
remote consumer of pkh's log records would receive ANSI codes inside
the message text. Plain text is the logger's business to style.
2026-09-18 20:55:56 +02:00
vhaudiquet c2dae4f3f9 lift main.rs business logic into the library
The chlog target-series resolution becomes changelog::series_candidates
(UNRELEASED pinning, development-series default and fallbacks modeled
by SeriesCandidates), PPA references get package_info::split_ppa
(shared by pull and deb, now also rejecting empty parts), and SSH
endpoints get context::ContextConfig::from_endpoint — so a library
consumer can resolve series, validate PPAs and build context
configurations without reimplementing the CLI's rules. All three carry
unit tests; main.rs keeps only parsing of flags and error handling.
2026-09-18 20:51:36 +02:00
vhaudiquet 0421a91e01 changelog: report through the log instead of printing to stdout
generate_entry's status messages become log lines, and the unmet
build-dependency diagnostics travel inside the UnmetBuildDependencies
error (its Display carries the full report) instead of being printed
to stderr by the library: the caller renders both like any other
outcome.
2026-09-18 20:44:05 +02:00
vhaudiquet 54cb04ba27 put: report through the view and ask the host-key question through the Prompter
put() loses its MultiProgress parameter: the summary, pre-flight and
connection spinners become view messages, the per-file SFTP transfer
reports determinate progress through view.progress (upload_file takes
a byte-count callback instead of an indicatif bar), and the display is
released through view.suspend on every exit path. The hardcoded
trust-on-first-use prompt in the SSH host-key verification becomes the
Prompter::accept_host_key port (fail-closed by default; the terminal
prompter prints the authenticity banner and confirms), so a remote
frontend can surface its own host-key dialog.
2026-09-18 20:42:17 +02:00
vhaudiquet bb76e41908 new: drive the wizard and verification offers through the Prompter port
The interactive half of pkh new no longer touches the terminal prompt
module directly: run() takes a Prompter, picks the wizard or the plain
resolve path through interactive(), and every select/text/confirm
question (including the verification offers) goes through the port.
Cancellations propagate as Err, preserving Ctrl+C-aborts; the summary
and vendoring-notice prints become log lines. A builder-server embed
can now drive the whole scaffold wizard over its own wire format by
implementing Prompter.
2026-09-18 20:38:05 +02:00
vhaudiquet d64e472845 report: grow the Prompter port and move display_path out of the ui module
Prompter gains interactive(), select() and text() (with the Validator
type), and confirm() now propagates cancellation as Err so flows abort
instead of silently taking a default when the user hits Ctrl+C. The
terminal prompter implements the full port; the port also re-exports
the path display helper, which is pure presentation formatting used by
events and messages rather than terminal code.
2026-09-18 20:34:33 +02:00
vhaudiquet 052c02cdc3 deb: drive build_binary_package through the BuildView port
The binary build joins the source build on the reporting ports:
build_binary_package takes a DebBuildOptions struct (replacing eleven
positional arguments), reports target, phases, progress and the
outcome through the environment-agnostic BuildView, and the Phase enum
with its default classifiers moves from the terminal widget into the
deb module (announced through the enter_phase helper). DebUi loses its
inherent event methods and only implements the port; tee logging and
the SIGINT behavior are unchanged.

No behavior change for the CLI; headless consumers pass report::Quiet.
2026-09-18 20:22:45 +02:00
vhaudiquet 27b1083b15 report: add BuildView/Prompter ports and drive pkh build through them
Core flows no longer reach into the terminal UI: build_source_package
takes a BuildSourceOptions struct (source tree, domain options, view,
prompter) and reports phases, messages and outcomes through the
environment-agnostic ports in the new report module. The classifiers
move from ui/logfmt to the core logfmt module, DebUi becomes a
BuildView adapter, the re-vendor retry asks the prompter instead of
checking for a TTY, and artifact/success printing moves to the CLI.

Headless consumers pass report::Quiet; an embedding (e.g. a builder
server forwarding events to a web frontend) implements BuildView and
maps the plain-data events onto its own wire format.
2026-09-18 20:02:56 +02:00
vhaudiquet a7d2cfdc6e ci: pin upload-artifact to v3
CI / build (push) Successful in 2m57s
CI / test (push) Skipped
CI / snap (push) Successful in 4m29s
v4's GHES check refuses to run on any non-github.com server, so the
snap artifact upload always failed on gitea; v3 uses the artifact API
gitea implements.
2026-09-18 18:07:54 +02:00
vhaudiquet ff635b305b ci: install nodejs in the snap job so JS-based actions can run
CI / build (push) Successful in 2m55s
CI / test (push) Skipped
CI / snap (push) Failing after 4m19s
2026-09-18 16:33:16 +02:00
vhaudiquet 781ed204c2 clippy: write the changelog body loop as while let (rust 1.98)
CI / build (push) Successful in 2m53s
CI / test (push) Skipped
CI / snap (push) Failing after 9s
2026-09-18 15:43:12 +02:00
59 changed files with 11341 additions and 1691 deletions
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 799 KiB

+43 -2
View File
@@ -3,6 +3,7 @@ name: CI
on: on:
push: push:
branches: [ "main", "ci-test" ] branches: [ "main", "ci-test" ]
tags: [ "v*" ]
pull_request: pull_request:
branches: [ "main" ] branches: [ "main" ]
@@ -102,7 +103,7 @@ jobs:
- name: Install build prerequisites - name: Install build prerequisites
run: | run: |
apt-get update -q apt-get update -q
apt-get install -y -q --no-install-recommends git curl apt-get install -y -q --no-install-recommends git curl nodejs
- uses: actions/checkout@v6 - uses: actions/checkout@v6
- name: Build snap - name: Build snap
run: | run: |
@@ -114,8 +115,48 @@ jobs:
export PATH="/usr/libexec/snapcraft:$HOME/.cargo/bin:$PATH" export PATH="/usr/libexec/snapcraft:$HOME/.cargo/bin:$PATH"
snapcraft pack --destructive-mode snapcraft pack --destructive-mode
- name: Upload snap artifact - name: Upload snap artifact
uses: actions/upload-artifact@v4 # v4 refuses to run outside github.com (GHESNotSupportedError); Gitea
# implements the artifact API used by v3, so v3 is the supported choice.
uses: actions/upload-artifact@v3
with: with:
name: snap name: snap
path: ./*.snap path: ./*.snap
if-no-files-found: error if-no-files-found: error
publish:
# Publishes the crate to crates.io on a v* tag. Trusted publishing
# (OIDC) is GitHub-Actions-only, so authentication goes through a
# crates.io API token stored as the CARGO_REGISTRY_TOKEN secret,
# scoped to the pkh crate.
if: startsWith(github.ref, 'refs/tags/v')
needs: build
runs-on: ubuntu-latest
container:
image: ubuntu:26.04
options: --privileged --cap-add SYS_ADMIN --security-opt apparmor:unconfined
steps:
- name: Set up container image
run: |
apt-get update
apt-get install -y nodejs sudo curl wget ca-certificates build-essential
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
- name: Install build dependencies
run: |
sudo apt-get update
sudo apt-get install -y pkg-config libssl-dev libgpg-error-dev libgpgme-dev
- name: Check the tag matches the crate version
# cargo publish ships whatever version Cargo.toml declares,
# regardless of the tag: a mismatch must fail loudly instead of
# publishing the wrong version under the release tag.
run: |
crate_version="$(awk -F'"' '/^version =/{print $2; exit}' Cargo.toml)"
tag_version="${GITHUB_REF_NAME#v}"
if [ "$crate_version" != "$tag_version" ]; then
echo "tag $GITHUB_REF_NAME does not match crate version $crate_version" >&2
exit 1
fi
- name: Publish
run: cargo publish
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
+7 -1
View File
@@ -1,2 +1,8 @@
*.lock
target target
# Local snapcraft builds
.craft
parts
prim
stage
*.snap
+108
View File
@@ -0,0 +1,108 @@
# AGENTS.md
Conventions for working in this tree. They apply to every commit; the
whole history follows them.
## Before every commit
Run, in order, and make sure they are clean before committing:
```sh
cargo fmt --all
cargo clippy --all-targets --all-features # zero warnings
```
CI builds and lints with `RUSTFLAGS: -Dwarnings`, so a `cargo build`
warning fails the gate too. `cargo fmt` may amend files you did not
touch — include those changes in the commit (or in a separate `fmt:`
commit) rather than leaving the tree dirty.
The test suite is heavy (chroots, ssh, network): always run the
`#[cfg(test)]` modules of what you touch, and the full suite when
changing shared plumbing (`report`, `logfmt`, `test_support`,
`debian/`). Tests marked `#[ignore]` shell out or hit the network and
are for deliberate ad-hoc runs (`cargo test -- --ignored`), not for the
pre-commit pass. Building needs the gpgme/openssl system packages
(`pkg-config libssl-dev libgpg-error-dev libgpgme-dev`).
## Commit messages
There are no conventional-commit types; the format is a component scope
and a summary:
```
<scope>: <short summary>
```
Rules:
- The scope is the component touched — the module under `src/` (file or
directory), named after the user-facing subcommand when that differs:
- `pull` — source package download (`src/pull.rs`)
- `chlog` — changelog entry generation (`src/changelog.rs`)
- `build` — source package builds, .dsc (`src/build/`)
- `deb` — binary package builds, .deb (`src/deb/`)
- `put` — PPA/archive upload (`src/put/`)
- `new` — package scaffolding (`src/new/`)
- `lint` — tree linting (`src/lint/`)
- `prune`, `package_info` — remaining subcommand modules
- `context` — build contexts: local, ssh, chroot/schroot, unshare
(`src/context/`)
- `interrupt` — Ctrl+C interception and interrupt-time cleanup
(`src/interrupt.rs`)
- `debian` — Debian format primitives: control, versions, checksums,
arch (`src/debian/`)
- `apt`, `launchpad`, `distro_info`, `quirks` — archive/distro
integration
- `report` — BuildView/Prompter ports and the views implementing them
- `ui`, `logfmt` — terminal rendering and output classification
- `data` — the `data/*.yml` embed convention itself; content changes
to a data file belong to the commit of the module consuming it
- `cli` — the binary, argument wiring (`src/main.rs`)
- `test` — test-only changes (shared plumbing: `src/test_support.rs`)
- `deps` — dependency additions/bumps (manifests, lockfile)
- `fmt`, `clippy` — rustfmt/clippy fixups
- `ci`, `snap`, `docs` — workflows, snap packaging, README
- Use the submodule path when the change is confined to one
(`apt/keyring`, `debian/version`).
- A commit touching several components should be split into one commit
per component when practical; otherwise comma-join the scopes without
spaces (`pull,deb`).
- Summary: imperative mood, lowercase first letter (proper nouns keep
theirs: Ubuntu, SRU, lintian), no trailing period, max ~72 characters.
- Body (expected for anything nontrivial): separated by a blank line,
wrapped at 72 columns; explain why, and the design when the approach
was a choice among alternatives. Reference issues as `#123`.
- Reverts use git's default `Revert "<original subject>"`.
### Examples
```
chlog: fall back to the changelog history when no version tag exists
ui: ellipsize fake-terminal pane lines wider than the terminal
lint: add pkh lint, wrapping lintian for parity plus pkh-native checks
chlog: number Ubuntu backports with the per-release SRU scheme
deb: resolve cross pkg-config against the target multiarch
pull,deb: add top-level --pocket option
debian/version: dpkg-compatible version comparison
deps: bump git2 to 0.21
fmt: apply rustfmt
docs: refresh the README roadmap for 1.0
```
## Code
- The crate denies missing docs (`#![deny(missing_docs)]` in
`src/lib.rs`): every public item carries a doc comment, and the module
list there is the layout map — keep it in sync when adding a module.
- Subcommand business logic lives in the library and reports through the
`report` ports (`BuildView`, `Prompter`) instead of printing;
`src/main.rs` is argument wiring only. Subprocess output
classification is pure logic in `logfmt`, testable without a pty.
- Static reference data (series tables, keyserver URLs, licenses,
forges, templates) lives in `data/*.yml`, embedded with the
`embed_data!` macro — not in hardcoded tables.
- Comments state constraints the code cannot show; no narration.
- Anything user-facing (subcommands, flags, option defaults) is
reflected in `README.md` — including its roadmap checklists — before
commit.
Generated
+3068
View File
File diff suppressed because it is too large Load Diff
+7
View File
@@ -3,6 +3,10 @@ name = "pkh"
version = "0.1.0" version = "0.1.0"
edition = "2024" edition = "2024"
authors = ["vhaudiquet"] authors = ["vhaudiquet"]
description = "pkh is a packaging helper for Debian/Ubuntu packages"
license = "MIT OR GPL-2.0-only"
repository = "https://git.vhaudiquet.fr/vhaudiquet/pkh"
readme = "README.md"
[dependencies] [dependencies]
clap = { version = "4.5.51", features = ["cargo"] } clap = { version = "4.5.51", features = ["cargo"] }
@@ -34,6 +38,9 @@ ssh2 = "0.9.5"
gpgme = "0.11" gpgme = "0.11"
serde_yaml = "0.9" serde_yaml = "0.9"
lazy_static = "1.4.0" lazy_static = "1.4.0"
unicode-width = "0.2"
parking_lot = "0.12"
suppaftp = "12"
[dev-dependencies] [dev-dependencies]
test-log = "0.2.19" test-log = "0.2.19"
+338
View File
@@ -0,0 +1,338 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
<https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Moe Ghoul>, 1 April 1989
Moe Ghoul, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025-2026 Valentin Haudiquet
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+38 -56
View File
@@ -2,6 +2,24 @@
`pkh` is a packaging helper for Debian/Ubuntu packages. `pkh` is a packaging helper for Debian/Ubuntu packages.
![](.github/pkh.gif)
## Installation
No distribution channel is published yet; build from source:
```
sudo apt install pkg-config libssl-dev libgpg-error-dev libgpgme-dev
git clone https://git.vhaudiquet.fr/vhaudiquet/pkh.git
cd pkh
cargo install --path .
```
At runtime pkh shells out to the Debian packaging toolchain (git,
dpkg-dev, quilt, mmdebstrap, lintian, pristine-tar, ...): install the
ones your workflows use, or build the classic snap from
`snap/snapcraft.yaml` (`snapcraft pack`), which carries them.
## Usage and features ## Usage and features
### Basic concepts ### Basic concepts
@@ -25,12 +43,19 @@ Options:
Commands and workflows include: Commands and workflows include:
``` ```
Commands: Commands:
new Scaffold a new Debian source package (buildable right away)
pull Pull a source package from the archive or git pull Pull a source package from the archive or git
chlog Auto-generate changelog entry, editing it, committing it afterwards chlog Auto-generate changelog entry, editing it, committing it afterwards
build Build the source package (into a .dsc) build Build the source package (into a .dsc)
deb Build the source package into binary package (.deb)
put Upload the built source package to a PPA put Upload the built source package to a PPA
deb Build the source package into binary package (.deb)
lint Lint the package (lintian wrapper + pkh-native checks)
prune Prune residual pkh build artifacts and caches
help Print this message or the help of the given subcommand(s) help Print this message or the help of the given subcommand(s)
Options:
-h, --help Print help
-V, --version Print version
``` ```
### Examples ### Examples
@@ -66,68 +91,25 @@ That is a lot of different tools and operations. With pkh, the same workflow:
pkh pull hello # needs -d ubuntu if you are not running Ubuntu pkh pull hello # needs -d ubuntu if you are not running Ubuntu
# Apply the patch to the package # Apply the patch to the package
... ...
pkh commit -m "Applied patch xxx" git add debian/patches/xxx.patch
git commit -m "Applied patch xxx"
pkh chlog pkh chlog
git add debian/changelog
git commit -m "d/changelog"
# Test that the package builds # Test that the package builds
pkh build pkh build
pkh deb pkh deb
# Upload the package to a ppa # Upload the package to a ppa
pkh put --ppa user/hello_xxx pkh put --ppa user/hello_xxx
# Push previously commited changes # Push the commits to your fork
git push xxx user-fork git push xxx user-fork
``` ```
## Roadmap: features needed for 1.0 ## Future improvement ideas
Basically, wrapping the basic debian workflows.
Missing features:
- [ ] `pkh pull`
- [x] Obtain package sources from git
- [x] Obtain package sources from the archive (fallback)
- [x] Obtain package source from PPA (--ppa)
- [ ] Obtain a specific version of the package
- [x] Fetch the correct git branch for series on Ubuntu
- [ ] Try to fetch the correct git branch for series on Debian, or fallback to the archive
- [ ] `pkh chlog`
- [x] Auto-generate changelog entry
- [ ] Extra flags: backport, non-maintainer upload, no change rebuild, ...
- [ ] Commit changelog entry
- [ ] `pkh build`
- [x] Build the source package
- [ ] `pkh deb`
- [x] Build the binary package
- [x] Build for a specific architecture
- [ ] Three build modes:
- [ ] Build locally (discouraged)
- [x] Build using unshare chroot, with binary emulation (default)
- [x] Cross-compilation
- [ ] Async build
- [ ] `pkh status`
- [ ] Show build status
- [ ] `pkh put`
- [x] Upload the source package to a PPA (native SFTP, no `dput` dependency)
- [ ] Upload the source package to the archive
- [ ] `pkh commit`
- [ ] Commit the changes to git
- [ ] `pkh lint`
- [ ] Lint the package
- [ ] `pkh test`
- [ ] Run autopkgtest
- [ ] Provide options: local (discouraged), chroot, VM?, ppa
- [ ] Async test
## Nice-to-have features
- [ ] 'pkh pull'
- [ ] Cache the Sources.gz files, to improve speed
- [ ] Work in an already downloaded package, to git pull and re-fetch orig tar gz
- [ ] 'pkh context'
- [x] Select, add, remove, list contexts
- [x] Context-scoped command execution
- [ ] Context-scoped deb and test commands
- [ ] Per-architecture contexts
- [ ] Per-series contexts
- [x] ssh contexts
- [ ] docker, lxc contexts?
- [ ] context push, context pop: context stack
- pull: try to fetch the correct git branch for series on Debian
- deb: asynchronous build, detachable and monitorable
- put: allow uploads to Debian or Ubuntu archives
- test: add 'pkh test' to run autopkgtests
- pull: cache Sources.gz files to improve speed
- pull: 'pkh pull' in a package tree should git pull and re-fetch orig tgz
+10
View File
@@ -28,6 +28,12 @@
## series (`<series>-updates`, ...). Deliberately not the ## series (`<series>-updates`, ...). Deliberately not the
## `pockets` key: that one is the *search order* of pull, ## `pockets` key: that one is the *search order* of pull,
## where backports must not fold in. ## where backports must not fold in.
## suite_aliases: the changelog suite names that alias a series
## codename: Debian packages conventionally target
## 'unstable' where the series data carries 'sid'.
## Mapped suite name -> series codename; the two
## names identify the same series, and the selector
## offers the aliased entry as '<suite> (<series>)'.
## build_profiles: the vendor's default DEB_BUILD_PROFILES (Ubuntu ## build_profiles: the vendor's default DEB_BUILD_PROFILES (Ubuntu
## activates derivative.ubuntu noudeb, Debian none), ## activates derivative.ubuntu noudeb, Debian none),
## mirroring what Dpkg::BuildProfiles resolves when the ## mirroring what Dpkg::BuildProfiles resolves when the
@@ -47,6 +53,10 @@ dist:
- updates - updates
- security - security
- proposed-updates - proposed-updates
# Debian changelogs conventionally target 'unstable'; the series data
# knows the same series as 'sid'.
suite_aliases:
unstable: sid
sections: sections:
# Valid Section values for debian/control: the Debian policy section # Valid Section values for debian/control: the Debian policy section
# list unioned with the sections observed in the live Ubuntu archive. # list unioned with the sections observed in the live Ubuntu archive.
+7
View File
@@ -13,6 +13,9 @@
## ssh_*: the PPA upload queue, as expanded by dput-ng's ## ssh_*: the PPA upload queue, as expanded by dput-ng's
## ppa:user/ppa profile (ppa.launchpad.net:22, incoming ## ppa:user/ppa profile (ppa.launchpad.net:22, incoming
## ~<user>/<ppa>) ## ~<user>/<ppa>)
## ftp_*: the same upload queue over anonymous FTP, dput-ng's
## plain ppa: profile: the transport pkh degrades to
## when the SSH connection itself never comes up
## content_host_template: ppa.launchpadcontent.net serves PPA apt ## content_host_template: ppa.launchpadcontent.net serves PPA apt
## repositories since the 2022 move off ppa.launchpad.net ## repositories since the 2022 move off ppa.launchpad.net
## git_web_template: Launchpad's CGit mirrors of Ubuntu source packages ## git_web_template: Launchpad's CGit mirrors of Ubuntu source packages
@@ -21,6 +24,10 @@
api_base: https://api.launchpad.net/1.0 api_base: https://api.launchpad.net/1.0
ssh_host: ppa.launchpad.net ssh_host: ppa.launchpad.net
ssh_port: 22 ssh_port: 22
## The anonymous FTP upload queue dput-ng's plain ppa: profile uses:
## pkh degrades to it when the SSH connection itself never comes up.
ftp_host: ppa.launchpad.net
ftp_port: 21
incoming_template: "~{owner}/{ppa}" incoming_template: "~{owner}/{ppa}"
content_host_template: https://ppa.launchpadcontent.net/{owner}/{ppa}/ubuntu content_host_template: https://ppa.launchpadcontent.net/{owner}/{ppa}/ubuntu
git_web_template: https://git.launchpad.net/ubuntu/+source/{package} git_web_template: https://git.launchpad.net/ubuntu/+source/{package}
+38 -6
View File
@@ -1,15 +1,47 @@
# Quirks configuration for package-specific workarounds # Quirks configuration for package-specific workarounds
# This file defines package-specific quirks that are applied during pull and deb operations # This file defines package-specific quirks that are applied during pull and deb operations
#
# `pull` and `deb` hold one entry per scope: several entries can carry
# different `series` lists, and every matching entry applies in file
# order. Entries can be scoped with `series`: an empty list applies to
# every series, otherwise only the listed ones. Packaging workarounds
# should carry the series they were verified against so they can be
# dropped once the upstream packaging catches up.
quirks: quirks:
# The resolute kernels declare `llvm-21-dev` unqualified while their
# other llvm pieces are `:native`; the dpkg cross rules then resolve it
# against the host architecture, whose dependency closure conflicts with
# the `:native` python3. Resolve it against the build architecture
# until the control is fixed upstream.
linux:
deb:
- series: [resolute]
dependencies:
replace:
llvm-21-dev: llvm-21-dev:native <!stage1>
linux-riscv:
deb:
- series: [resolute]
dependencies:
replace:
llvm-21-dev: llvm-21-dev:native <!stage1>
# Add more packages and their quirks as needed # Add more packages and their quirks as needed
# example-package: # example-package:
# pull: # pull:
# method: archive # - series: [noble]
# package_directory:
# - linux-main
# deb: # deb:
# extra_dependencies: # - series: [resolute]
# - another-dependency # dependencies:
# parameters: # replace:
# key: value # llvm-21-dev: llvm-21-dev:native <!stage1>
# - series: [stonking]
# dependencies:
# replace:
# llvm-22-dev: llvm-22-dev:native <!stage1>
# parameters:
# key: value
+130
View File
@@ -0,0 +1,130 @@
---
name: pkh
description: 'Drive pkh, a Debian/Ubuntu packaging helper: pull source packages, generate changelog entries, build .dsc/.deb, lint, and upload to a PPA. Use it whenever the task touches Debian or Ubuntu packaging: patching an existing package, preparing an SRU, backport or NMU, scaffolding a new .deb, rebuilding for a PPA, or uploading a source package. Trigger on "update the changelog", "package this", or a bare package name, even when the user never mentions Debian.'
---
# pkh
`pkh` wraps the Debian packaging toolchain (`dch`, `dpkg-buildpackage`,
`sbuild`, `dpkg-source`, `quilt`, `lintian`, PPA uploads) in one CLI.
The subcommands share one set of option names, so `-s` always targets
the series and `--ppa` always names the PPA. Each step also does more
than the raw tool it replaces: `pull` fetches the orig tarball with the
source, `chlog` commits the entry it writes, `deb` sets up a chroot and
installs the build dependencies.
Check the install with `pkh --version`. Each command lists its flags
with `pkh <command> --help`, so check there instead of guessing. pkh
shells out to host tools (git, dpkg-dev, quilt, mmdebstrap, lintian,
pristine-tar, schroot, ...). Install the ones your workflow uses, or
use the classic snap, which carries them.
## Shared options
| Option | Meaning |
|---|---|
| `-d, --dist <dist>` | Target distribution, `debian` or `ubuntu` |
| `-s, --series <series>` | Target series, for example `resolute` or `noble` |
| `-v, --version <version>` | Target package version |
| `-a, --arch <arch>` | Target architecture, for example `amd64` or `riscv64` |
| `-p, --pocket <pocket>` | Distribution pocket: `updates`, `security`, `proposed` |
| `--ppa <user/ppa>` | Act on the named PPA |
Defaults come from the host vendor, its development series, and its
architecture. When the target differs, pass the flags: packaging for
Ubuntu on a Debian host needs `-d ubuntu`, and a series or architecture
that differs from the host needs `-s` or `-a`.
## Patch an Ubuntu package
```
pkh pull hello # source and orig tarball; add -d ubuntu off an Ubuntu host
# edit the package, committing each patch to git
pkh chlog # generates the entry, opens it for editing, commits it
git add debian/changelog && git commit -m "d/changelog"
pkh build # source package, written next to the tree
pkh deb # binary build in a chroot with build deps installed
pkh lint # lintian plus pkh-native checks
pkh put --ppa user/hello_xxx # uploads the .changes file from the build
git push xxx user-fork # push the branch to your fork
```
Run `pkh chlog` and `pkh build` from the root of the source tree; they
act on the package in the current directory.
## Command reference
- `pkh new [name]` scaffolds a buildable source package. `--lang`
picks the build system (`rust`, `python`, `meson`, `cmake`,
`autotools`, `go`, `shell`, `makefile`); `--source <PATH>` packages
existing sources instead. `--upstream-version` and `--revision` set
the version. `--description`, `--homepage`, `--license <SPDX>`,
`--command`, `--maintainer "Name <email>"`, and `--depends` fill in
the package metadata, with the maintainer defaulting to
`DEBFULLNAME`/`DEBEMAIL` and then git config. `--quilt` and
`--native` choose the source format; `--orig-from
release|git|path|snapshot` and `--orig-path` control the orig
tarball. The changelog starts as `UNRELEASED`; `--release` targets
`--series` instead. `--defaults` answers every remaining question
with its default, which keeps the run non-interactive.
- `pkh pull <package>` fetches a source package from the archive or
git. `--archive` skips git. `--ppa user/ppa` and `--repository
<suite-url>` pull from a PPA or an external flat repository instead.
`-d`, `-s`, `-v`, and `-p` target an exact source.
- `pkh chlog` generates the changelog entry from the commits since the
last version tag, opens it for editing, and commits it. `--backport`,
`--nmu`, and `--rebuild` apply the matching numbering scheme
(`3.1-1ubuntu2~24.04.1`, `1.0-1.1`, `1.0-1build1`); `-v` sets an
explicit version instead.
- `pkh build` produces the .dsc. `--orig auto|always|never` controls
whether the upload includes the orig tarball; the default, `auto`,
includes it only when the upstream version changed.
- `pkh deb` builds the binary packages in an isolated context with the
build dependencies installed. `--ppa` (repeatable) adds dependency
sources, `--inject <package|.deb>` preinstalls a package, and `-j`
caps parallel jobs. `--cross` cross-compiles instead of using
qemu-binfmt, but most packages cannot cross-compile, so prefer qemu.
Leave `--mode` unset unless you need a specific build context.
- `pkh lint [path]` runs lintian plus the pkh-native checks. `--json`
emits a machine-readable report and `--list-tags` prints the native
tag catalog. `--fail-on` sets the severities that fail the run
(errors by default), `--suppress-tags` ignores tags, `--check` runs a
single native check, and `--info`, `--pedantic`, and
`--experimental` add detail. `--repack` packs the tree fresh instead
of reusing the existing build output.
- `pkh put [changes]` uploads a .changes file to `--ppa user/ppa`.
With no argument it uploads the .changes from this package's last
build, found next to the source tree. `--force` re-uploads a file
that was already uploaded.
- `pkh prune` removes build artifacts and caches. Run it with
`--dry-run` first to list them. `--all` also deletes the cached
chroot tarballs, which take long to download again, so use it when
you need the disk space.
## Notes for agent runs
- Pass `-d`, `-s`, and `-a` whenever the target differs from the host,
so runs are reproducible.
- Keep runs non-interactive. Pass explicit flags, use `pkh new
--defaults`, and set `EDITOR` before `pkh chlog` (`EDITOR=true` keeps
the generated text). Commands may ask short questions on the
terminal; flags avoid most prompts.
- Pass `RUST_LOG=debug` for pkh's own logs. `--verbose` on `pkh build`
and `pkh deb` prints raw tool output instead of the live view.
- pkh writes the build artifacts (.dsc, .changes, logs) next to the
source tree. `pkh put` finds them without arguments, and `pkh prune`
removes them again.
- Run `pkh lint` before `pkh put`. It exits nonzero when findings reach
the `--fail-on` level, which defaults to errors.
- pkh intercepts Ctrl+C, runs its cleanup hooks, and exits with status
130. `pkh prune` removes anything left over.
The upstream repository is https://git.vhaudiquet.fr/vhaudiquet/pkh.
Its README has longer workflow examples.
+83 -10
View File
@@ -2,12 +2,17 @@ name: pkh
base: core24 base: core24
summary: pkh is a packaging helper for Debian/Ubuntu packages summary: pkh is a packaging helper for Debian/Ubuntu packages
description: | description: |
pkh aims at wrapping the different debian tools and workflows pkh aims at wrapping the different debian tools and workflows
into one tool, that would have the same interface for everything, into one tool, that would have the same interface for everything,
while being smarter at integrating all workflows. while being smarter at integrating all workflows.
This snap uses classic confinement and carries the packaging
toolchain it drives (dpkg-dev, git, mmdebstrap, lintian, quilt, ...)
so it behaves the same on any Debian/Ubuntu host.
license: MIT OR GPL-2.0-only
adopt-info: pkh-part adopt-info: pkh-part
confinement: devmode confinement: classic
apps: apps:
pkh: pkh:
@@ -19,24 +24,92 @@ parts:
source: . source: .
override-pull: | override-pull: |
craftctl default craftctl default
craftctl set version=$(git rev-parse --short=11 HEAD) # Release metadata comes from the crate, not the git state: a build
craftctl set grade="devel" # of any commit must produce the version the crate declares.
craftctl set version="$(awk -F'"' '/^version =/{print $2; exit}' Cargo.toml)"
craftctl set grade="stable"
build-packages: build-packages:
- build-essential - build-essential
- file
- patchelf
- pkg-config - pkg-config
- libssl-dev - libssl-dev
- libgpg-error-dev - libgpg-error-dev
- libgpgme-dev - libgpgme-dev
# Host-side tools pkh execs directly. Tools that only run *inside*
# the build chroot (dose-builddebcheck, dpkg-cross) are provisioned
# there by pkh itself and must not be staged; likewise qemu-user-static
# is host binfmt configuration, not a bundled file.
#
# The apt and dpkg state-owning tools are deliberately excluded below:
# they must be the host's (classic mode makes them visible), since a
# core24 apt/dpkg managing a newer host's package database is exactly
# the version skew classic snaps must avoid. The source-package tools
# (dpkg-buildpackage, dpkg-source, ...) are bundled instead.
stage-packages: stage-packages:
- libgpgme11t64 - libgpgme11t64
- git - git
- curl - curl
- gnupg
- gpgv
- dpkg-dev
- quilt
- pristine-tar - pristine-tar
- mmdebstrap - mmdebstrap
- lintian
- fakeroot
- util-linux - util-linux
- dpkg-dev # mount/umount moved to their own package (split from util-linux)
- mount
- schroot
- openssh-client
- tar
- xz-utils
- bzip2
stage: stage:
- -usr/lib/x86_64-linux-gnu/libicuio.so.74.2 - -usr/bin/apt
- -usr/lib/x86_64-linux-gnu/libicutest.so.74.2 - -usr/bin/apt-cache
- -usr/lib/x86_64-linux-gnu/libicutu.so.74.2 - -usr/bin/apt-cdrom
- -usr/lib/x86_64-linux-gnu/libicui18n.so.74.2 - -usr/bin/apt-config
- -usr/bin/apt-get
- -usr/bin/apt-key
- -usr/bin/apt-mark
- -usr/lib/*/libapt-*
- -usr/lib/*/libicuio*
- -usr/lib/*/libicutest*
- -usr/lib/*/libicutu*
- -usr/lib/*/libicui18n*
# update-alternatives does not run at staging time: expose the sysv
# fakeroot under the plain name dpkg-buildpackage and pkh exec.
override-prime: |
craftctl default
ln -sfn fakeroot-sysv "${CRAFT_PRIME}/usr/bin/fakeroot"
# Ship the license texts with the binary: the MIT grant requires
# the notice to accompany copies, and GPL-2 requires the license
# text alongside distribution.
mkdir -p "${CRAFT_PRIME}/usr/share/doc/pkh"
cp "${CRAFT_PART_SRC}/LICENSE-MIT" "${CRAFT_PART_SRC}/LICENSE-GPL" \
"${CRAFT_PRIME}/usr/share/doc/pkh/"
# Classic-confined ELFs default to the host loader, which pins the
# snap to hosts shipping at least the build environment's glibc,
# and cannot see the libraries deduplicated against the base.
# Point every bundled ELF at the core24 loader and give it an
# rpath resolving base libraries from the mounted base and
# snap-local libraries from $ORIGIN — the classic linter's
# guidance, and what Canonical's own classic snaps do. DT_RPATH
# (--force-rpath) is required over the default DT_RUNPATH: the
# host ld.so.cache would otherwise resolve sonames to host
# libraries first, mixing host libm/libresolv with base libc.
# DT_RPATH also propagates transitively, covering dependencies of
# dependencies (libgpgme -> libassuan). Host tools spawned later
# (host apt-get, ...) run with a pristine environment since no
# LD_LIBRARY_PATH is exported.
find "${CRAFT_PRIME}" -type f -exec sh -c '
for f do
[ "$(od -An -N4 -tx1 "$f" | tr -d " \n")" = "7f454c46" ] || continue
patchelf --set-interpreter \
/snap/core24/current/lib64/ld-linux-x86-64.so.2 "$f" 2>/dev/null || true
patchelf --force-rpath --set-rpath \
"/snap/core24/current/lib/x86_64-linux-gnu:/snap/core24/current/usr/lib/x86_64-linux-gnu:\$ORIGIN:\$ORIGIN/../lib/x86_64-linux-gnu:\$ORIGIN/../usr/lib/x86_64-linux-gnu" \
"$f" 2>/dev/null || true
done' sh {} +
+21 -9
View File
@@ -89,24 +89,27 @@ pub async fn download_cache_keyrings(
keyring_dir.display() keyring_dir.display()
) )
})?; })?;
// Upgrade cache directories created by versions that made them
// private: mmdebstrap's unshare-mode hooks cannot read them.
} else { } else {
// Remote contexts (e.g. ssh) have no stat/metadata access through // Remote contexts (e.g. ssh) have no stat/metadata access through
// the context API, so the ownership guard cannot be performed; // the context API, so the ownership guard cannot be performed;
// keep the previous best-effort behavior of tightening the // keep the previous best-effort behavior of tightening the
// directory permissions instead (0700 instead of the former // directory permissions instead (no group/others write).
// world-writable a+rwx).
ctx.command("chmod").arg("700").arg(&keyring_dir).status()?;
} }
ctx.command("chmod").arg("755").arg(&keyring_dir).status()?;
} else { } else {
// Create the directory private to the invoking user (0700). This is // Create the directory readable but not writable by group/others.
// sufficient for mmdebstrap in unshare mode: it runs with the same // mmdebstrap's unshare-mode hooks run under an identity that cannot
// real uid (the user namespace only maps that uid to root, file // read the invoking user's private directories, so 0700 breaks the
// access still happens as the real uid), so no world-accessible // keyring copy into the chroot; the planting guard stays on the
// permissions are needed. // ownership and no-write checks of validate_keyring_dir (the
// skip-if-exists logic below trusts pre-existing keyrings, so the
// directory must never be writable by anyone else).
ctx.command("mkdir") ctx.command("mkdir")
.arg("-p") .arg("-p")
.arg("-m") .arg("-m")
.arg("700") .arg("755")
.arg(&keyring_dir) .arg(&keyring_dir)
.status()?; .status()?;
} }
@@ -178,6 +181,11 @@ pub async fn download_cache_keyrings(
binary_path.display() binary_path.display()
); );
} }
// Readable like the directory: mmdebstrap's hooks copy these into
// the chroot. Applies to legacy files too, which a restrictive
// umask may have left private, and a permissive one group-writable.
let _ = ctx.command("chmod").arg("644").arg(&binary_path).status();
} }
log::info!( log::info!(
@@ -348,6 +356,10 @@ mod tests {
assert!(validate_keyring_dir(1000, 0o750, 1000).is_ok()); assert!(validate_keyring_dir(1000, 0o750, 1000).is_ok());
assert!(validate_keyring_dir(1000, 0o1744, 1000).is_ok()); assert!(validate_keyring_dir(1000, 0o1744, 1000).is_ok());
assert!(validate_keyring_dir(0, 0o700, 0).is_ok()); assert!(validate_keyring_dir(0, 0o700, 0).is_ok());
// The world-readable modes the cache now uses: readable so that
// mmdebstrap's unshare-mode hooks can copy the keyrings, while the
// ownership and no-write checks keep the planting guard.
assert!(validate_keyring_dir(1000, 0o755, 1000).is_ok());
} }
#[test] #[test]
+126 -121
View File
@@ -14,7 +14,6 @@ pub mod env;
use std::collections::{BTreeMap, HashMap}; use std::collections::{BTreeMap, HashMap};
use std::error::Error; use std::error::Error;
use std::io::IsTerminal;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::process::{Command, Stdio}; use std::process::{Command, Stdio};
use std::sync::Arc; use std::sync::Arc;
@@ -24,8 +23,8 @@ use crate::context::{LineSink, Stream};
use crate::debian::{ use crate::debian::{
ChecksumEntry, ControlInfo, FileChecksums, FilesEntry, FilesList, parse_paragraphs, ChecksumEntry, ControlInfo, FileChecksums, FilesEntry, FilesList, parse_paragraphs,
}; };
use crate::ui::deb::DebUi; use crate::logfmt::{DpkgSourceClassifier, GenericClassifier};
use crate::ui::logfmt::{DpkgSourceClassifier, GenericClassifier}; use crate::report::{BuildTarget, BuildView, Prompter};
/// Whether the upload distributes the upstream orig tarballs (`--orig`), /// Whether the upload distributes the upstream orig tarballs (`--orig`),
/// mirroring the `dpkg-genchanges` source styles. /// mirroring the `dpkg-genchanges` source styles.
@@ -74,105 +73,118 @@ pub struct SourceBuildOutput {
pub signed: bool, pub signed: bool,
} }
impl SourceBuildOutput {
/// All produced artifacts in distribution order: dsc → tarballs →
/// buildinfo → changes.
pub fn artifacts(&self) -> Vec<PathBuf> {
let mut artifacts = Vec::with_capacity(3 + self.tarballs.len());
artifacts.push(self.dsc.clone());
artifacts.extend(self.tarballs.iter().cloned());
artifacts.push(self.buildinfo.clone());
artifacts.push(self.changes.clone());
artifacts
}
}
/// Parameters of one [`build_source_package`] call: where to build, the
/// domain options, and the reporting ports (view and prompter).
pub struct BuildSourceOptions<'a> {
/// Source tree to package. When unset, the process's current working
/// directory is used (resolved to an absolute path).
pub source: Option<PathBuf>,
/// Domain options (signing, orig-tarball inclusion, ...).
pub options: SourceBuildOptions,
/// Where build events (target, phases, messages, outcome) are reported.
pub view: &'a dyn BuildView,
/// Who answers the questions the flow may ask (e.g. the re-vendor
/// retry offer).
pub prompter: &'a dyn Prompter,
}
impl Default for BuildSourceOptions<'_> {
fn default() -> Self {
static QUIET: crate::report::Quiet = crate::report::Quiet;
BuildSourceOptions {
source: None,
options: SourceBuildOptions::default(),
view: &QUIET,
prompter: &QUIET,
}
}
}
/// Build a Debian source package (to a .dsc) using the native pipeline. /// Build a Debian source package (to a .dsc) using the native pipeline.
/// ///
/// When `ui` is set, subprocess output is captured into a live view (status /// Subprocess output is captured into the view (status line + rolling
/// bar + rolling pane) and tee'd to a log file; on failure the view prints a /// pane for the terminal adapter) and tee'd to a log file; on failure the
/// summary of the last captured errors. Without a UI, commands inherit the /// view prints a summary of the last captured errors. Headless callers use
/// terminal as before. /// [`crate::report::Quiet`], in which case commands still run with captured
/// output in test builds.
/// ///
/// A `dpkg-source -b` failure is classified (see /// A `dpkg-source -b` failure is classified (see
/// [`classify_dpkg_source_failure`]); when the vendored rust dependencies /// [`classify_dpkg_source_failure`]); when the vendored rust dependencies
/// diverged from the orig-vendor component and a terminal is attached, the /// diverged from the orig-vendor component, the flow asks the prompter
/// flow offers to re-vendor, recreate the component and retry the build /// whether to re-vendor, recreates the component and retries the build
/// exactly once. /// exactly once.
///
/// On success the produced artifacts are reported through the view and
/// returned.
pub fn build_source_package( pub fn build_source_package(
cwd: Option<&Path>, opts: BuildSourceOptions<'_>,
opts: SourceBuildOptions, ) -> Result<SourceBuildOutput, Box<dyn Error>> {
ui: Option<Arc<DebUi>>,
) -> Result<(), Box<dyn Error>> {
// Default to the process's current working directory, resolved to an // Default to the process's current working directory, resolved to an
// absolute path: the output directory is derived from `cwd.parent()` // absolute path: the output directory is derived from `cwd.parent()`
// downstream, which only yields a real directory for an absolute `cwd` // downstream, which only yields a real directory for an absolute `cwd`
// (the parent of "." is the empty path). // (the parent of "." is the empty path).
let cwd = match cwd { let cwd = match opts.source {
Some(p) => p.to_path_buf(), Some(ref p) => p.clone(),
None => std::env::current_dir() None => std::env::current_dir()
.map_err(|e| format!("cannot determine the current working directory: {e}"))?, .map_err(|e| format!("cannot determine the current working directory: {e}"))?,
}; };
let output = match run_source_build(&cwd, &opts, ui.clone()) { let output = match run_source_build(&cwd, &opts.options, opts.view) {
Ok(output) => output, Ok(output) => output,
Err(e) if e.downcast_ref::<VendorDriftError>().is_some() => { Err(e) if e.downcast_ref::<VendorDriftError>().is_some() => {
return retry_after_revendor(&cwd, ui, opts, e); return retry_after_revendor(&cwd, opts.view, opts.prompter, &opts.options, e);
} }
Err(e) => { Err(e) => {
if let Some(u) = &ui { opts.view.finish_failure();
u.finish_failure();
}
return Err(e); return Err(e);
} }
}; };
// Artifact listing in dpkg order: dsc → tarballs → buildinfo → changes. opts.view.finish_success(&output.artifacts());
let mut artifacts = Vec::with_capacity(3 + output.tarballs.len()); Ok(output)
artifacts.push(output.dsc.clone());
artifacts.extend(output.tarballs.iter().cloned());
artifacts.push(output.buildinfo.clone());
artifacts.push(output.changes.clone());
// The live view lists the artifacts itself when it renders; otherwise
// (verbose mode or non-TTY stdout) print them as plain lines.
let listed = match &ui {
Some(u) if u.is_enabled() => {
u.finish_success(&artifacts, u.elapsed());
true
}
_ => false,
};
if !listed {
for artifact in &artifacts {
println!(" {}", crate::ui::display_path(artifact));
}
}
if output.signed {
println!("Package built and signed successfully!");
} else {
println!("Package built successfully (unsigned).");
}
Ok(())
} }
/// The re-vendor retry hook for a [`VendorDriftError`]: on an interactive /// The re-vendor retry hook for a [`VendorDriftError`]: offer to re-run the
/// terminal, offer to re-run the vendoring step (the same helper the rust /// vendoring step through the prompter (the same helper the rust template
/// template uses at scaffold time), recreate the `orig-vendor` component /// uses at scaffold time), recreate the `orig-vendor` component from the
/// from the fresh `vendor/` tree and retry the source build exactly once. /// fresh `vendor/` tree and retry the source build exactly once. Without an
/// Without a terminal (or on a declined offer) the original error is /// accepting answer (headless prompters answer with the default, `false`)
/// returned untouched. /// the original error is returned untouched.
fn retry_after_revendor( fn retry_after_revendor(
cwd: &Path, cwd: &Path,
ui: Option<Arc<DebUi>>, view: &dyn BuildView,
opts: SourceBuildOptions, prompter: &dyn Prompter,
opts: &SourceBuildOptions,
original: Box<dyn Error>, original: Box<dyn Error>,
) -> Result<(), Box<dyn Error>> { ) -> Result<SourceBuildOutput, Box<dyn Error>> {
let interactive = std::io::stdin().is_terminal() && std::io::stdout().is_terminal(); // The offer is only made when someone can answer it: headless prompters
if !interactive { // answer with the default (false) without the error being logged here —
if let Some(u) = &ui { // the caller logs the returned error itself, exactly once.
u.finish_failure(); if !prompter.interactive() {
} view.finish_failure();
return Err(original); return Err(original);
} }
log::error!("{original}"); log::error!("{original}");
let retry = crate::ui::prompt::confirm( if !prompter
"Re-vendor the Cargo dependencies and retry the build?", .confirm(
false, "Re-vendor the Cargo dependencies and retry the build?",
) false,
.unwrap_or(false); )
if !retry { .unwrap_or(false)
if let Some(u) = &ui { {
u.finish_failure(); view.finish_failure();
}
return Err(original); return Err(original);
} }
@@ -189,9 +201,7 @@ fn retry_after_revendor(
std::fs::remove_file(&config)?; std::fs::remove_file(&config)?;
} }
if !crate::new::templates::rust::vendor_dependencies(cwd)? { if !crate::new::templates::rust::vendor_dependencies(cwd)? {
if let Some(u) = &ui { view.finish_failure();
u.finish_failure();
}
return Err( return Err(
"Re-vendoring did not complete: the tree is unchanged, fix the \ "Re-vendoring did not complete: the tree is unchanged, fix the \
vendoring by hand and build again." vendoring by hand and build again."
@@ -218,7 +228,7 @@ fn retry_after_revendor(
crate::new::orig::create_vendor_component(cwd, &entry.source, uversion)?; crate::new::orig::create_vendor_component(cwd, &entry.source, uversion)?;
// 3. One retry, with the options of the original attempt. // 3. One retry, with the options of the original attempt.
run_source_build(cwd, &opts, ui).map(|_| ()) run_source_build(cwd, opts, view)
} }
/// Run the full native source-build pipeline in `cwd`. /// Run the full native source-build pipeline in `cwd`.
@@ -237,14 +247,12 @@ fn retry_after_revendor(
pub fn run_source_build( pub fn run_source_build(
cwd: &Path, cwd: &Path,
opts: &SourceBuildOptions, opts: &SourceBuildOptions,
ui: Option<Arc<DebUi>>, view: &dyn BuildView,
) -> Result<SourceBuildOutput, Box<dyn Error>> { ) -> Result<SourceBuildOutput, Box<dyn Error>> {
// Without a live UI, test runs still capture command output into the // The view consumes the captured lines itself (live view + tee log);
// per-test log file instead of letting it inherit the terminal // without one, test runs still capture command output into the per-test
let sink: Option<Arc<dyn LineSink>> = ui // log file instead of letting it inherit the terminal
.as_ref() let sink: Option<Arc<dyn LineSink>> = view.sink().or_else(crate::test_support::subprocess_sink);
.map(|u| u.sink())
.or_else(crate::test_support::subprocess_sink);
// ------------------------------------------------------------------ // ------------------------------------------------------------------
// 1. Sanity checks // 1. Sanity checks
// ------------------------------------------------------------------ // ------------------------------------------------------------------
@@ -296,9 +304,19 @@ pub fn run_source_build(
let ctrl = ControlInfo::parse(&control_path)?; let ctrl = ControlInfo::parse(&control_path)?;
if let Some(u) = &ui { view.target(BuildTarget {
u.set_build_target(&entry.source, &entry.version.full(), &entry.distribution); package: &entry.source,
} version: &entry.version.full(),
target: &entry.distribution,
display: format!(
"Building source package {} ({}) for {}",
entry.source,
entry.version.full(),
entry.distribution
),
source_only: true,
tee_log: true,
});
let source_display = entry.source.clone(); let source_display = entry.source.clone();
@@ -357,9 +375,7 @@ pub fn run_source_build(
// ------------------------------------------------------------------ // ------------------------------------------------------------------
// 5. dpkg-source lifecycle: before-build + source build // 5. dpkg-source lifecycle: before-build + source build
// ------------------------------------------------------------------ // ------------------------------------------------------------------
if let Some(u) = &ui { view.phase("Applying patches", Box::new(DpkgSourceClassifier::new()));
u.phase_custom("Applying patches", Box::new(DpkgSourceClassifier::new()));
}
run_command( run_command(
cwd, cwd,
"dpkg-source", "dpkg-source",
@@ -372,9 +388,7 @@ pub fn run_source_build(
// dpkg-buildpackage skips it entirely for source-only builds unless // dpkg-buildpackage skips it entirely for source-only builds unless
// forced with -D; unsatisfied dependencies abort with exit status 3. // forced with -D; unsatisfied dependencies abort with exit status 3.
if opts.force_dep_check { if opts.force_dep_check {
if let Some(u) = &ui { view.message("Checking build dependencies");
u.progress_message("Checking build dependencies");
}
let check_opts = crate::debian::deps::CheckOpts { let check_opts = crate::debian::deps::CheckOpts {
host_arch: arch_vars host_arch: arch_vars
.get("DEB_HOST_ARCH") .get("DEB_HOST_ARCH")
@@ -389,19 +403,19 @@ pub fn run_source_build(
}; };
let report = crate::debian::deps::check_build_depends(&ctrl, &check_opts)?; let report = crate::debian::deps::check_build_depends(&ctrl, &check_opts)?;
if !report.is_ok() { if !report.is_ok() {
eprintln!("{}", report.message()); // The typed error carries the diagnostics (UnmetReport is
// public); the caller renders them and maps the type to exit
// status 3, like dpkg-buildpackage does.
return Err(Box::new(crate::debian::deps::UnmetBuildDependencies( return Err(Box::new(crate::debian::deps::UnmetBuildDependencies(
report, report,
))); )));
} }
} }
if let Some(u) = &ui { view.phase(
u.phase_custom( "Building source package",
"Building source package", Box::new(DpkgSourceClassifier::new()),
Box::new(DpkgSourceClassifier::new()), );
);
}
if let Err(failure) = run_command_capturing( if let Err(failure) = run_command_capturing(
cwd, cwd,
"dpkg-source", "dpkg-source",
@@ -427,9 +441,7 @@ pub fn run_source_build(
// ------------------------------------------------------------------ // ------------------------------------------------------------------
// 6. .buildinfo generation (native dpkg-genbuildinfo equivalent) // 6. .buildinfo generation (native dpkg-genbuildinfo equivalent)
// ------------------------------------------------------------------ // ------------------------------------------------------------------
if let Some(u) = &ui { view.message("Generating .buildinfo");
u.progress_message("Generating .buildinfo");
}
// What the .buildinfo itself records: like dpkg-genbuildinfo, only the // What the .buildinfo itself records: like dpkg-genbuildinfo, only the
// referenced .dsc — not the tarballs, and never the buildinfo itself. // referenced .dsc — not the tarballs, and never the buildinfo itself.
let mut buildinfo_checksums = FileChecksums::new(); let mut buildinfo_checksums = FileChecksums::new();
@@ -479,9 +491,7 @@ pub fn run_source_build(
// ------------------------------------------------------------------ // ------------------------------------------------------------------
// 7. .changes generation (native dpkg-genchanges equivalent) // 7. .changes generation (native dpkg-genchanges equivalent)
// ------------------------------------------------------------------ // ------------------------------------------------------------------
if let Some(u) = &ui { view.message("Generating .changes");
u.progress_message("Generating .changes");
}
// What the .changes distributes: the .dsc (with its recorded digests), // What the .changes distributes: the .dsc (with its recorded digests),
// the tarballs listed in it (below), and the .buildinfo (last, as // the tarballs listed in it (below), and the .buildinfo (last, as
// dpkg-genchanges does when it consumes debian/files). // dpkg-genchanges does when it consumes debian/files).
@@ -538,13 +548,11 @@ pub fn run_source_build(
if opts.orig_source == OrigSourceMode::Never && !has_debian_part { if opts.orig_source == OrigSourceMode::Never && !has_debian_part {
log::warn!("ignoring --orig never for a native Debian package"); log::warn!("ignoring --orig never for a native Debian package");
} }
if let Some(u) = &ui { view.message(if strip_origs {
u.progress_message(if strip_origs { "Not including original source code in upload"
"Not including original source code in upload" } else {
} else { "Including full source code in upload"
"Including full source code in upload" });
});
}
// Stripped orig tarballs (and their detached .asc signatures) are not // Stripped orig tarballs (and their detached .asc signatures) are not
// distributed at all: not hashed, not required on disk, like // distributed at all: not hashed, not required on disk, like
// dpkg-genchanges. // dpkg-genchanges.
@@ -634,9 +642,7 @@ pub fn run_source_build(
// ------------------------------------------------------------------ // ------------------------------------------------------------------
// 8. dpkg-source after-build (unapplies quilt patches it applied) // 8. dpkg-source after-build (unapplies quilt patches it applied)
// ------------------------------------------------------------------ // ------------------------------------------------------------------
if let Some(u) = &ui { view.phase("Restoring patches", Box::new(DpkgSourceClassifier::new()));
u.phase_custom("Restoring patches", Box::new(DpkgSourceClassifier::new()));
}
run_command( run_command(
cwd, cwd,
"dpkg-source", "dpkg-source",
@@ -652,9 +658,7 @@ pub fn run_source_build(
if let Some(keyid) = signing_key.filter(|_| do_sign) { if let Some(keyid) = signing_key.filter(|_| do_sign) {
crate::utils::gpg::validate_key_id(&keyid)?; crate::utils::gpg::validate_key_id(&keyid)?;
if let Some(u) = &ui { view.phase("Signing artifacts", Box::new(GenericClassifier::new()));
u.phase_custom("Signing artifacts", Box::new(GenericClassifier::new()));
}
log::info!("Signing {}", dsc_name); log::info!("Signing {}", dsc_name);
crate::utils::gpg::clearsign_file(&dsc_path, &keyid)?; crate::utils::gpg::clearsign_file(&dsc_path, &keyid)?;
@@ -1039,7 +1043,7 @@ mod tests {
.expect("write control"); .expect("write control");
std::fs::write(tree.join("debian/rules"), "#!/usr/bin/make -f\n").expect("write rules"); std::fs::write(tree.join("debian/rules"), "#!/usr/bin/make -f\n").expect("write rules");
let err = run_source_build(&tree, &SourceBuildOptions::default(), None) let err = run_source_build(&tree, &SourceBuildOptions::default(), &crate::report::Quiet)
.expect_err("binary-only entries must not build a source package"); .expect_err("binary-only entries must not build a source package");
let err = err.to_string(); let err = err.to_string();
assert!(err.contains("binary-only"), "{err}"); assert!(err.contains("binary-only"), "{err}");
@@ -1575,7 +1579,8 @@ mod differential_tests {
OrigSourceMode::Never => &["-sd"], OrigSourceMode::Never => &["-sd"],
}; };
run_dpkg(&golden_tree, source_style); run_dpkg(&golden_tree, source_style);
run_source_build(&ours_tree, opts, None).expect("native source pipeline failed"); run_source_build(&ours_tree, opts, &crate::report::Quiet)
.expect("native source pipeline failed");
let entry = let entry =
crate::debian::parse_changelog_entry(&ours_tree.join("debian/changelog")).unwrap(); crate::debian::parse_changelog_entry(&ours_tree.join("debian/changelog")).unwrap();
@@ -2160,7 +2165,7 @@ Provides: virtual-thing (= 2.0), plain-virtual
); );
// Ours: the native pipeline refuses likewise. // Ours: the native pipeline refuses likewise.
let err = run_source_build(&tree, &SourceBuildOptions::default(), None) let err = run_source_build(&tree, &SourceBuildOptions::default(), &crate::report::Quiet)
.expect_err("native source build must refuse a binary-only entry"); .expect_err("native source build must refuse a binary-only entry");
assert!(err.to_string().contains("binary-only"), "{err}"); assert!(err.to_string().contains("binary-only"), "{err}");
} }
+1050 -87
View File
File diff suppressed because it is too large Load Diff
+129 -1
View File
@@ -70,6 +70,12 @@ pub trait ContextDriver {
fn read_file(&self, path: &Path) -> io::Result<String>; fn read_file(&self, path: &Path) -> io::Result<String>;
fn write_file(&self, path: &Path, content: &str) -> io::Result<()>; fn write_file(&self, path: &Path, content: &str) -> io::Result<()>;
fn exists(&self, path: &Path) -> io::Result<bool>; fn exists(&self, path: &Path) -> io::Result<bool>;
/// Check if a path is a directory inside the context
///
/// Distinct from [`ContextDriver::exists`] because paths returned by
/// [`ContextDriver::list_files`] are context-relative and can only be
/// classified through the context, never with a host-side stat.
fn is_dir(&self, path: &Path) -> io::Result<bool>;
/// Clean up any resources held by the driver (e.g. unmount overlay filesystems). /// Clean up any resources held by the driver (e.g. unmount overlay filesystems).
/// Called before the chroot directory is removed. /// Called before the chroot directory is removed.
@@ -117,6 +123,28 @@ pub enum ContextConfig {
}, },
} }
impl ContextConfig {
/// Build an SSH context configuration from an endpoint of the form
/// `[ssh://][user@]host[:port]`.
pub fn from_endpoint(endpoint: &str) -> Result<Self, String> {
let re = regex::Regex::new(
r"^(?:ssh://)?(?:(?P<user>[^@]+)@)?(?P<host>[^:/]+)(?::(?P<port>\d+))?$",
)
.expect("valid endpoint regex");
let cap = re.captures(endpoint).ok_or_else(|| {
format!("Invalid endpoint format: '{endpoint}'. Expected [ssh://][user@]host[:port]")
})?;
let host = cap.name("host").unwrap().as_str().to_string();
let user = cap.name("user").map(|m| m.as_str().to_string());
let port = cap
.name("port")
.map(|m| m.as_str().parse::<u16>())
.transpose()
.map_err(|_| "Invalid port number".to_string())?;
Ok(ContextConfig::Ssh { host, user, port })
}
}
/// A context, allowing to run commands, read and write files, etc /// A context, allowing to run commands, read and write files, etc
pub struct Context { pub struct Context {
/// Configuration for the context /// Configuration for the context
@@ -221,12 +249,22 @@ impl Context {
} }
/// Make a command inside context /// Make a command inside context
///
/// Build tooling must not inherit the session's locale: dpkg-family
/// tools and perl-based packaging scripts change their output (and
/// dpkg-buildpackage treats some of it as data) with the environment,
/// and a translated or mixed locale leaks host state into builds. The
/// C locale is the default; a caller can still override it by setting
/// LANG/LC_ALL through [`ContextCommand::envs`] afterwards.
pub fn command<S: AsRef<OsStr>>(&self, program: S) -> ContextCommand<'_> { pub fn command<S: AsRef<OsStr>>(&self, program: S) -> ContextCommand<'_> {
ContextCommand { ContextCommand {
context: self, context: self,
program: program.as_ref().to_string_lossy().to_string(), program: program.as_ref().to_string_lossy().to_string(),
args: Vec::new(), args: Vec::new(),
env: Vec::new(), env: vec![
("LANG".to_string(), "C".to_string()),
("LC_ALL".to_string(), "C".to_string()),
],
cwd: None, cwd: None,
sink: None, sink: None,
} }
@@ -281,6 +319,15 @@ impl Context {
self.driver().as_ref().unwrap().exists(path) self.driver().as_ref().unwrap().exists(path)
} }
/// Check if a path is a directory inside context
///
/// Paths returned by [`Context::list_files`] are context-relative
/// (e.g. rooted inside the chroot for an unshare context): whether they
/// are directories can only be decided through the context.
pub fn is_dir(&self, path: &Path) -> io::Result<bool> {
self.driver().as_ref().unwrap().is_dir(path)
}
/// Clean up any resources held by the driver (e.g. unmount overlay filesystems). /// Clean up any resources held by the driver (e.g. unmount overlay filesystems).
/// Called before the chroot directory is removed. /// Called before the chroot directory is removed.
pub fn cleanup(&self) -> io::Result<()> { pub fn cleanup(&self) -> io::Result<()> {
@@ -455,3 +502,84 @@ fn contextualize_spawn_error(program: &str, e: io::Error) -> io::Error {
io::Error::new(e.kind(), format!("Could not run '{program}': {e}")) io::Error::new(e.kind(), format!("Could not run '{program}': {e}"))
} }
} }
#[cfg(test)]
mod endpoint_tests {
use super::*;
/// Every accepted endpoint spelling maps to the expected config.
#[test]
fn from_endpoint_parses_all_spellings() {
assert_eq!(
ContextConfig::from_endpoint("myhost"),
Ok(ContextConfig::Ssh {
host: "myhost".into(),
user: None,
port: None,
})
);
assert_eq!(
ContextConfig::from_endpoint("admin@myhost"),
Ok(ContextConfig::Ssh {
host: "myhost".into(),
user: Some("admin".into()),
port: None,
})
);
assert_eq!(
ContextConfig::from_endpoint("myhost:2222"),
Ok(ContextConfig::Ssh {
host: "myhost".into(),
user: None,
port: Some(2222),
})
);
assert_eq!(
ContextConfig::from_endpoint("ssh://admin@myhost:22"),
Ok(ContextConfig::Ssh {
host: "myhost".into(),
user: Some("admin".into()),
port: Some(22),
})
);
}
/// Non-numeric ports and extra segments are format errors; a
/// non-u16 numeric port is a port error.
#[test]
fn from_endpoint_rejects_malformed_endpoints() {
for bad in ["", "a/b/c", "host:notaport"] {
let err = ContextConfig::from_endpoint(bad).unwrap_err();
assert!(err.contains("Invalid endpoint format"), "{err}");
}
let err = ContextConfig::from_endpoint("host:99999").unwrap_err();
assert_eq!(err, "Invalid port number");
}
/// Commands run in the C locale whatever the session environment
/// carries: host locale variables must not leak into builds. An
/// explicit caller override still wins.
#[test]
fn commands_default_to_the_c_locale() {
let ctx = Context::new(ContextConfig::Local).unwrap();
let locale = ctx
.command("sh")
.arg("-c")
.arg("printf '%s' \"${LC_ALL:-unset}:${LANG:-unset}\"")
.output()
.unwrap()
.stdout;
assert_eq!(String::from_utf8_lossy(&locale), "C:C");
let locale = ctx
.command("sh")
.arg("-c")
.arg("printf '%s' \"$LC_ALL\"")
.env("LC_ALL", "C.UTF-8")
.output()
.unwrap()
.stdout;
assert_eq!(String::from_utf8_lossy(&locale), "C.UTF-8");
}
}
+52 -14
View File
@@ -24,33 +24,31 @@ impl ContextDriver for LocalDriver {
} }
fn create_temp_dir(&self) -> io::Result<String> { fn create_temp_dir(&self) -> io::Result<String> {
// Generate a unique temporary directory name with random string // Sub-second precision and an atomic create: two concurrent
// contexts racing on the same name must never share a directory,
// so the loser of a create falls through to the next attempt
// instead of probing for existence first (a probe-then-create
// window loses exactly when two callers arrive together).
let base_timestamp = SystemTime::now() let base_timestamp = SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH) .duration_since(SystemTime::UNIX_EPOCH)
.unwrap() .unwrap()
.as_secs(); .as_millis();
let mut attempt = 0; let mut attempt = 0;
loop { loop {
let work_dir_name = if attempt == 0 { let work_dir_name = if attempt == 0 {
format!("pkh-{}", base_timestamp) format!("pkh-{base_timestamp}")
} else { } else {
format!("pkh-{}-{}", base_timestamp, attempt) format!("pkh-{base_timestamp}-{attempt}")
}; };
let temp_dir_path = std::env::temp_dir().join(&work_dir_name); let temp_dir_path = std::env::temp_dir().join(&work_dir_name);
// Check if directory already exists match std::fs::create_dir(&temp_dir_path) {
if temp_dir_path.exists() { Ok(()) => return Ok(temp_dir_path.to_string_lossy().to_string()),
attempt += 1; Err(e) if e.kind() == io::ErrorKind::AlreadyExists => attempt += 1,
continue; Err(e) => return Err(e),
} }
// Create the directory
std::fs::create_dir_all(&temp_dir_path)?;
// Return the path as a string
return Ok(temp_dir_path.to_string_lossy().to_string());
} }
} }
@@ -157,6 +155,10 @@ impl ContextDriver for LocalDriver {
fn exists(&self, path: &Path) -> io::Result<bool> { fn exists(&self, path: &Path) -> io::Result<bool> {
Ok(path.exists()) Ok(path.exists())
} }
fn is_dir(&self, path: &Path) -> io::Result<bool> {
Ok(path.is_dir())
}
} }
fn copy_dir_recursive(src: &Path, dest: &Path) -> io::Result<()> { fn copy_dir_recursive(src: &Path, dest: &Path) -> io::Result<()> {
@@ -190,3 +192,39 @@ fn copy_dir_recursive(src: &Path, dest: &Path) -> io::Result<()> {
} }
Ok(()) Ok(())
} }
#[cfg(test)]
mod tests {
use super::*;
/// Concurrent callers must never share a temporary directory: the
/// create is atomic, so a lost race falls through to the next name
/// instead of both callers probing the same free name and unpacking
/// into the same directory.
#[test]
fn create_temp_dir_is_unique_under_concurrency() {
const CALLERS: usize = 8;
let (tx, rx) = std::sync::mpsc::channel();
let handles: Vec<_> = (0..CALLERS)
.map(|_| {
let tx = tx.clone();
std::thread::spawn(move || {
let dir = LocalDriver.create_temp_dir().unwrap();
tx.send(dir).unwrap();
})
})
.collect();
for handle in handles {
handle.join().unwrap();
}
drop(tx);
let mut names: Vec<String> = rx.iter().collect();
names.sort();
let unique: std::collections::BTreeSet<&String> = names.iter().collect();
assert_eq!(names.len(), unique.len(), "duplicate temp dirs: {names:?}");
for name in &unique {
std::fs::remove_dir(name).unwrap();
}
}
}
+34
View File
@@ -385,6 +385,40 @@ mod tests {
assert!(!dest.join("src/.svn").exists()); assert!(!dest.join("src/.svn").exists());
} }
/// The unshare driver maps context-relative paths onto the chroot root
/// on the host: `is_dir` must answer through that mapping (a host-side
/// stat of the unmapped path sees nothing), which is what lets the deb
/// package-directory search classify staged entries.
#[test]
fn test_unshare_is_dir_maps_through_the_chroot_root() {
let chroot = tempfile::tempdir().unwrap();
fs::create_dir_all(chroot.path().join("tmp/work/tree/debian")).unwrap();
fs::write(chroot.path().join("tmp/work/orig.tar.xz"), "tar").unwrap();
let base = Context::new(ContextConfig::Local).unwrap();
let ctx = Context::with_parent(
ContextConfig::Unshare {
path: chroot.path().to_string_lossy().to_string(),
parent: None,
},
Arc::new(base),
);
assert!(ctx.is_dir(std::path::Path::new("/tmp/work/tree")).unwrap());
assert!(
ctx.exists(std::path::Path::new("/tmp/work/tree/debian"))
.unwrap()
);
assert!(
!ctx.is_dir(std::path::Path::new("/tmp/work/orig.tar.xz"))
.unwrap()
);
assert!(
!ctx.exists(std::path::Path::new("/tmp/work/missing"))
.unwrap()
);
}
/// The overlay-mount path exposes the tree verbatim, so pruning happens /// The overlay-mount path exposes the tree verbatim, so pruning happens
/// after the fact: nested VCS metadata must be removed recursively. /// after the fact: nested VCS metadata must be removed recursively.
#[test] #[test]
+10
View File
@@ -296,6 +296,16 @@ impl ContextDriver for SchrootDriver {
)?; )?;
Ok(status.success()) Ok(status.success())
} }
fn is_dir(&self, path: &Path) -> io::Result<bool> {
let status = self.run(
"test",
&["-d".to_string(), path.to_string_lossy().to_string()],
&[],
None,
)?;
Ok(status.success())
}
} }
#[cfg(test)] #[cfg(test)]
+8
View File
@@ -306,6 +306,14 @@ impl ContextDriver for SshDriver {
Err(_) => Ok(false), Err(_) => Ok(false),
} }
} }
fn is_dir(&self, path: &Path) -> io::Result<bool> {
let sess = connect_ssh(&self.host, self.user.as_deref(), self.port)?;
let sftp = sess.sftp().map_err(io::Error::other)?;
// Same error tolerance as `exists`: an unreachable path is not a
// directory, and the caller decides what absence means.
Ok(sftp.stat(path).map(|stat| stat.is_dir()).unwrap_or(false))
}
} }
impl SshDriver { impl SshDriver {
+26 -17
View File
@@ -296,37 +296,41 @@ impl ContextDriver for UnshareDriver {
fn create_temp_dir(&self) -> io::Result<String> { fn create_temp_dir(&self) -> io::Result<String> {
// Create a temporary directory inside the chroot with unique naming // Create a temporary directory inside the chroot with unique naming
// Sub-second precision and an atomic create, like the local
// driver: concurrent callers racing on the same name must not
// share a directory, so an existing target falls through to the
// next attempt instead of a probe-then-create window.
let base_timestamp = std::time::SystemTime::now() let base_timestamp = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH) .duration_since(std::time::UNIX_EPOCH)
.unwrap() .unwrap()
.as_secs(); .as_millis();
let mut attempt = 0; let mut attempt = 0;
loop { loop {
let work_dir_name = if attempt == 0 { let work_dir_name = if attempt == 0 {
format!("pkh-build-{}", base_timestamp) format!("pkh-build-{base_timestamp}")
} else { } else {
format!("pkh-build-{}-{}", base_timestamp, attempt) format!("pkh-build-{base_timestamp}-{attempt}")
}; };
let work_dir_inside_chroot = format!("/tmp/{}", work_dir_name); let work_dir_inside_chroot = format!("/tmp/{work_dir_name}");
let host_path = Path::new(&self.path).join("tmp").join(&work_dir_name); let host_path = Path::new(&self.path).join("tmp").join(&work_dir_name);
// Check if directory already exists match std::fs::create_dir(&host_path) {
if host_path.exists() { Ok(()) => {
attempt += 1; debug!(
continue; "Created work directory: {} (host: {})",
work_dir_inside_chroot,
host_path.display()
);
}
Err(e) if e.kind() == io::ErrorKind::AlreadyExists => {
attempt += 1;
continue;
}
Err(e) => return Err(e),
} }
// Create the directory on the host filesystem
std::fs::create_dir_all(&host_path)?;
debug!(
"Created work directory: {} (host: {})",
work_dir_inside_chroot,
host_path.display()
);
// Return the path as it appears inside the chroot // Return the path as it appears inside the chroot
return Ok(work_dir_inside_chroot); return Ok(work_dir_inside_chroot);
} }
@@ -352,6 +356,11 @@ impl ContextDriver for UnshareDriver {
let host_path = Path::new(&self.path).join(path.to_string_lossy().trim_start_matches('/')); let host_path = Path::new(&self.path).join(path.to_string_lossy().trim_start_matches('/'));
self.parent().exists(&host_path) self.parent().exists(&host_path)
} }
fn is_dir(&self, path: &Path) -> io::Result<bool> {
let host_path = Path::new(&self.path).join(path.to_string_lossy().trim_start_matches('/'));
Ok(host_path.is_dir())
}
} }
impl UnshareDriver { impl UnshareDriver {
+35
View File
@@ -62,6 +62,20 @@ pub fn setup_environment(
.map_err(|e| format!("Invalid UTF-8 in dpkg-architecture output: {e}"))?; .map_err(|e| format!("Invalid UTF-8 in dpkg-architecture output: {e}"))?;
parse_dpkg_architecture_output(&dpkg_architecture, env); parse_dpkg_architecture_output(&dpkg_architecture, env);
// In-tree tools locate their libraries with the *host* pkg-config during
// cross builds (the kernel's tools/build feature checks derive their
// cflags/ldflags from `pkg-config --cflags/--libs`), whose search path
// only covers the build architecture's pkgconfig dirs. Point it at the
// target's so `libtraceevent` & co resolve to target-arch libraries:
// linux-riscv cross builds die in rtla's Makefile.config otherwise, even
// with the target -dev packages installed.
if let Some(multiarch) = env.get("DEB_HOST_MULTIARCH").cloned() {
env.insert(
"PKG_CONFIG_LIBDIR".to_string(),
format!("/usr/lib/{multiarch}/pkgconfig:/usr/share/pkgconfig"),
);
}
env.insert("DEB_BUILD_PROFILES".to_string(), "cross".to_string()); env.insert("DEB_BUILD_PROFILES".to_string(), "cross".to_string());
Ok(()) Ok(())
@@ -289,4 +303,25 @@ mod tests {
); );
assert!(cross_suites("noble", None, "not-a-distro").is_err()); assert!(cross_suites("noble", None, "not-a-distro").is_err());
} }
/// setup_environment exports the target multiarch pkg-config libdir:
/// tools' feature checks run the *host* pkg-config, which must find the
/// target's .pc files (rtla hard-errors on libtraceevent otherwise,
/// failing linux-riscv cross builds despite the target -dev packages
/// being installed).
#[test]
fn test_setup_environment_exports_cross_pkg_config_libdir() {
let mut env = HashMap::new();
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap());
setup_environment(&mut env, "riscv64", ctx).unwrap();
assert_eq!(
env.get("PKG_CONFIG_LIBDIR").map(String::as_str),
Some("/usr/lib/riscv64-linux-gnu/pkgconfig:/usr/share/pkgconfig")
);
assert_eq!(
env.get("DEB_BUILD_PROFILES").map(String::as_str),
Some("cross")
);
}
} }
+97 -297
View File
@@ -1,138 +1,24 @@
use crate::context::{self, Context, ContextConfig}; use crate::context::{self, Context, ContextConfig};
use crate::ui::deb::{DebUi, Phase}; use crate::deb::{Phase, enter_phase};
use crate::interrupt::CleanupHookGuard;
use crate::report::BuildView;
use directories::ProjectDirs; use directories::ProjectDirs;
use std::any::Any;
use std::error::Error; use std::error::Error;
use std::fs; use std::fs;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::process::Command; use std::process::Command;
use std::sync::Arc; use std::sync::Arc;
use std::sync::Mutex;
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::Duration;
use tar::Archive; use tar::Archive;
use xz2::read::XzDecoder; use xz2::read::XzDecoder;
// ---------------------------------------------------------------------------
// Process-global cleanup hooks
//
// On Ctrl-C, the SIGINT handler in `ui::deb` restores the terminal and then
// `libc::_exit(130)`s, skipping all destructors — including
// [`EphemeralContextGuard::drop`] — which leaks the freshly bootstrapped
// chroot under /tmp together with its bind-mounted /proc and any overlayfs
// mounts. To make interrupt-time cleanup possible anyway, resources register
// a self-contained cleanup hook here; the SIGINT handler drains and runs the
// registry right before exiting.
// ---------------------------------------------------------------------------
/// A boxed, send-safe cleanup hook body
type CleanupFn = Box<dyn Fn() + Send>;
/// A pending cleanup hook together with its registry id
struct CleanupHook {
id: u64,
f: CleanupFn,
}
/// Registry of cleanup hooks waiting to run at interrupt time
static CLEANUP_HOOKS: Mutex<Vec<CleanupHook>> = Mutex::new(Vec::new());
/// Source of the registry ids used to deregister a specific hook
static NEXT_CLEANUP_HOOK_ID: AtomicU64 = AtomicU64::new(1);
/// Register a hook to be run by [`run_cleanup_hooks`] (i.e. when the process
/// is interrupted), returning a guard whose drop deregisters the hook again
fn register_cleanup_hook(f: CleanupFn) -> CleanupHookGuard {
let id = NEXT_CLEANUP_HOOK_ID.fetch_add(1, Ordering::Relaxed);
CLEANUP_HOOKS.lock().unwrap().push(CleanupHook { id, f });
CleanupHookGuard(id)
}
/// RAII handle to a registered cleanup hook: dropping it (or an explicit
/// [`CleanupHookGuard::deregister`]) removes the hook from the registry so
/// the interrupt path can no longer run it
struct CleanupHookGuard(u64);
impl CleanupHookGuard {
/// Registry id of the hook (used to filter the registry in tests)
#[cfg(test)]
fn id(&self) -> u64 {
self.0
}
/// Remove the hook from the registry; returns whether it was still pending
fn deregister(&mut self) -> bool {
deregister_cleanup_hook(self.0)
}
}
impl Drop for CleanupHookGuard {
fn drop(&mut self) {
deregister_cleanup_hook(self.0);
}
}
/// Remove a hook from the registry; returns whether it was still pending
fn deregister_cleanup_hook(id: u64) -> bool {
let mut hooks = CLEANUP_HOOKS.lock().unwrap();
let len_before = hooks.len();
hooks.retain(|hook| hook.id != id);
hooks.len() != len_before
}
/// Drain and run every registered cleanup hook exactly once
///
/// Called from the SIGINT handler right before the process exits. Draining
/// uses `try_lock` with a bounded retry instead of a blocking lock: if the
/// signal interrupted the main thread while it held [`CLEANUP_HOOKS`] (inside
/// register/deregister), blocking on the same non-recursive mutex from the
/// handler would deadlock the process. Timing out therefore skips cleanup
/// (leaking, as before this registry existed) rather than hanging.
pub(crate) fn run_cleanup_hooks() {
run_drained_hooks(drain_cleanup_hooks());
}
/// Take every pending hook out of the registry, waiting at most ~1s for the
/// registry lock (see [`run_cleanup_hooks`] for why this must not block forever)
fn drain_cleanup_hooks() -> Vec<CleanupHook> {
const RETRIES: usize = 200;
const RETRY_DELAY: Duration = Duration::from_millis(5);
for _ in 0..RETRIES {
if let Ok(mut hooks) = CLEANUP_HOOKS.try_lock() {
return std::mem::take(&mut *hooks);
}
std::thread::sleep(RETRY_DELAY);
}
log::error!("Timed out waiting for the cleanup hook registry; skipping interrupt cleanup");
Vec::new()
}
/// Run drained hooks one by one, isolating panics so that one failing hook
/// cannot skip the remaining ones
fn run_drained_hooks(hooks: Vec<CleanupHook>) {
for CleanupHook { id, f } in hooks {
// Hooks are arbitrary user code; assert unwind safety so they can be
// run inside a catching context
if let Err(panic) = std::panic::catch_unwind(std::panic::AssertUnwindSafe(f)) {
log::error!("Cleanup hook {id} panicked: {}", panic_message(&panic));
}
}
}
/// Best-effort message extraction from a panic payload
fn panic_message(panic: &(dyn Any + Send)) -> String {
if let Some(s) = panic.downcast_ref::<&str>() {
(*s).to_string()
} else if let Some(s) = panic.downcast_ref::<String>() {
s.clone()
} else {
"non-string panic payload".to_string()
}
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Interrupt-time chroot cleanup // Interrupt-time chroot cleanup
//
// On Ctrl-C, the watchdog in `crate::interrupt` runs the hook registered in
// [`EphemeralContextGuard::new_with_context`] right before exiting — the
// interrupt sequence skips all destructors, which would otherwise leak the
// freshly bootstrapped chroot under /tmp together with its bind-mounted
// /proc and any overlayfs mounts.
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
/// Interrupt-time cleanup of an ephemeral chroot: unmount every host-side /// Interrupt-time cleanup of an ephemeral chroot: unmount every host-side
@@ -142,9 +28,9 @@ fn panic_message(panic: &(dyn Any + Send)) -> String {
/// Unlike [`EphemeralContextGuard::drop`], this deliberately does NOT go /// Unlike [`EphemeralContextGuard::drop`], this deliberately does NOT go
/// through the context manager, the ephemeral context's driver (whose /// through the context manager, the ephemeral context's driver (whose
/// `cleanup()` unmounts the tracked overlays) or the base context's command /// `cleanup()` unmounts the tracked overlays) or the base context's command
/// builder: the signal may arrive while the interrupted thread holds any of /// builder: interrupt-time hooks must be self-contained, and those
/// those mutexes, and re-locking them from the signal handler would deadlock. /// machineries may be mid-mutation on the interrupted thread. Instead it
/// Instead it only reads /proc/mounts and spawns umount/rm directly. /// only reads /proc/mounts and spawns umount/rm directly.
/// ///
/// It also differs from `drop` in that it removes the chroot regardless of /// It also differs from `drop` in that it removes the chroot regardless of
/// the build result: the build was aborted, and leaving a still-mounted /// the build result: the build was aborted, and leaving a still-mounted
@@ -174,31 +60,48 @@ fn sigint_cleanup_chroot(chroot_path: &Path) {
} }
} }
// Remove the chroot tree itself (tolerates a missing directory) // Remove the chroot tree itself (tolerates a missing directory). A
let status = privileged_command("rm", is_root) // child the Ctrl+C interrupted may still be finishing its writeout —
.arg("-rf") // dpkg defers SIGINT until it reaches a safe state — so retry while rm
.arg(chroot_path) // reports the tree non-empty instead of leaving it half-removed.
.status(); const RETRIES: usize = 10;
match status { const RETRY_DELAY: std::time::Duration = std::time::Duration::from_millis(300);
Ok(status) if status.success() => { let mut last = None;
for attempt in 0..=RETRIES {
if attempt > 0 {
std::thread::sleep(RETRY_DELAY);
}
last = Some(
privileged_command("rm", is_root)
.arg("-rf")
.arg(chroot_path)
.status(),
);
if matches!(&last, Some(Ok(status)) if status.success()) {
break;
}
}
match last {
Some(Ok(status)) if status.success() => {
log::debug!( log::debug!(
"Removed chroot {} during interrupt cleanup", "Removed chroot {} during interrupt cleanup",
chroot_path.display() chroot_path.display()
); );
} }
Ok(status) => { Some(Ok(status)) => {
log::error!( log::error!(
"Failed to remove chroot {} during interrupt cleanup \ "Failed to remove chroot {} during interrupt cleanup \
(rm exited with {status}); run `pkh prune`", (rm exited with {status}); run `pkh prune`",
chroot_path.display() chroot_path.display()
); );
} }
Err(e) => { Some(Err(e)) => {
log::error!( log::error!(
"Failed to run rm for chroot {} during interrupt cleanup: {e}; run `pkh prune`", "Failed to run rm for chroot {} during interrupt cleanup: {e}; run `pkh prune`",
chroot_path.display() chroot_path.display()
); );
} }
None => unreachable!("at least one rm attempt ran"),
} }
} }
@@ -312,7 +215,7 @@ impl EphemeralContextGuard {
series: &str, series: &str,
arch: Option<&str>, arch: Option<&str>,
base_ctx: Arc<Context>, base_ctx: Arc<Context>,
ui: Option<Arc<DebUi>>, view: &dyn BuildView,
) -> Result<Self, Box<dyn Error>> { ) -> Result<Self, Box<dyn Error>> {
// Save the globally-installed context so Drop can restore exactly // Save the globally-installed context so Drop can restore exactly
// this handle: concurrent builds install their own ephemeral // this handle: concurrent builds install their own ephemeral
@@ -333,15 +236,14 @@ impl EphemeralContextGuard {
// Register the interrupt-time cleanup hook before any heavy work: if // Register the interrupt-time cleanup hook before any heavy work: if
// the user hits Ctrl-C during bootstrap or the build itself, the // the user hits Ctrl-C during bootstrap or the build itself, the
// SIGINT handler unmounts and removes the chroot through this hook // interrupt watchdog unmounts and removes the chroot through this
// (see `sigint_cleanup_chroot`). This only works for a local base // hook (see `sigint_cleanup_chroot`). This only works for a local
// context: the hook must be self-contained (stored path + direct // base context: the hook must be self-contained (stored path +
// umount/rm subprocesses) and cannot go through `base_ctx`, whose // direct umount/rm subprocesses) and cannot go through `base_ctx`.
// driver mutex may be held by the interrupted thread. For remote or // For remote or nested bases the chroot lives elsewhere, and
// nested bases the chroot lives elsewhere, and leftovers stay // leftovers stay handled by `pkh prune` as before.
// handled by `pkh prune` as before.
let cleanup_hook = if matches!(base_ctx.config, ContextConfig::Local) { let cleanup_hook = if matches!(base_ctx.config, ContextConfig::Local) {
Some(register_cleanup_hook(Box::new({ Some(crate::interrupt::register_cleanup_hook(Box::new({
let chroot_path = chroot_path.clone(); let chroot_path = chroot_path.clone();
move || sigint_cleanup_chroot(&chroot_path) move || sigint_cleanup_chroot(&chroot_path)
}))) })))
@@ -355,13 +257,21 @@ impl EphemeralContextGuard {
// Download and extract the chroot tarball // Download and extract the chroot tarball
if let Err(e) = if let Err(e) =
Self::download_and_extract_chroot(series, arch, &chroot_path, base_ctx.clone(), &ui) Self::download_and_extract_chroot(series, arch, &chroot_path, base_ctx.clone(), view)
.await .await
{ {
// The guard (and its Drop) never materializes on this path, so // On a Ctrl+C the interrupt watchdog owns the tree: keep the
// stop tracking the chroot for interrupt cleanup; as before, a // hook registered (forgetting the guard) so it removes the
// failed bootstrap leaves its partial directory in place. // partial directory, instead of the historical behavior of
drop(cleanup_hook); // leaving it in place. Without an interrupt this is a plain
// bootstrap failure and the partial directory stays, as before.
if crate::interrupt::interrupted()
&& let Some(hook) = cleanup_hook
{
std::mem::forget(hook);
} else {
drop(cleanup_hook);
}
return Err(e); return Err(e);
} }
@@ -408,7 +318,7 @@ impl EphemeralContextGuard {
arch: Option<&str>, arch: Option<&str>,
chroot_path: &PathBuf, chroot_path: &PathBuf,
ctx: Arc<context::Context>, ctx: Arc<context::Context>,
ui: &Option<Arc<DebUi>>, view: &dyn BuildView,
) -> Result<(), Box<dyn Error>> { ) -> Result<(), Box<dyn Error>> {
// Clone ctx for use in create_device_nodes after download_chroot_tarball consumes it // Clone ctx for use in create_device_nodes after download_chroot_tarball consumes it
let ctx_for_devices = ctx.clone(); let ctx_for_devices = ctx.clone();
@@ -436,6 +346,11 @@ impl EphemeralContextGuard {
let poll_interval = 5; // Check every 5 seconds let poll_interval = 5; // Check every 5 seconds
while ctx.exists(&lockfile_path)? { while ctx.exists(&lockfile_path)? {
// Stop waiting on a Ctrl+C: the interrupt watchdog removes the
// (yet empty) chroot and exits without waiting for the poll
if crate::interrupt::interrupted() {
return Err("Interrupted while waiting for the chroot tarball".into());
}
if wait_time >= timeout { if wait_time >= timeout {
log::warn!( log::warn!(
"Lockfile {} exists and has been present for more than {} seconds. \ "Lockfile {} exists and has been present for more than {} seconds. \
@@ -464,10 +379,8 @@ impl EphemeralContextGuard {
series, series,
arch arch
); );
if let Some(u) = ui { enter_phase(view, Phase::PreparingChroot);
u.phase(Phase::PreparingChroot); Self::download_chroot_tarball(series, arch, &tarball_path, ctx, view).await?;
}
Self::download_chroot_tarball(series, arch, &tarball_path, ctx, ui).await?;
} else { } else {
log::debug!( log::debug!(
"Using cached chroot tarball for {} (arch: {:?})", "Using cached chroot tarball for {} (arch: {:?})",
@@ -478,16 +391,12 @@ impl EphemeralContextGuard {
// Extract tarball to chroot directory // Extract tarball to chroot directory
log::debug!("Extracting chroot tarball to {}...", chroot_path.display()); log::debug!("Extracting chroot tarball to {}...", chroot_path.display());
if let Some(u) = ui { enter_phase(view, Phase::ExtractingChroot);
u.phase(Phase::ExtractingChroot); Self::extract_tarball(&tarball_path, chroot_path, view)?;
}
Self::extract_tarball(&tarball_path, chroot_path, ui.as_deref())?;
// Create device nodes in the chroot // Create device nodes in the chroot
log::debug!("Creating device nodes in chroot..."); log::debug!("Creating device nodes in chroot...");
if let Some(u) = ui { enter_phase(view, Phase::FinalizingChroot);
u.phase(Phase::FinalizingChroot);
}
Self::create_device_nodes(chroot_path, ctx_for_devices.clone())?; Self::create_device_nodes(chroot_path, ctx_for_devices.clone())?;
// Bind mount /proc from host into chroot (before entering unshare namespace) // Bind mount /proc from host into chroot (before entering unshare namespace)
@@ -503,7 +412,7 @@ impl EphemeralContextGuard {
arch: Option<&str>, arch: Option<&str>,
tarball_path: &Path, tarball_path: &Path,
ctx: Arc<context::Context>, ctx: Arc<context::Context>,
ui: &Option<Arc<DebUi>>, view: &dyn BuildView,
) -> Result<(), Box<dyn Error>> { ) -> Result<(), Box<dyn Error>> {
// Create a lock file to make sure that noone tries to use the file while it's not fully downloaded // Create a lock file to make sure that noone tries to use the file while it's not fully downloaded
let lockfile_path = tarball_path.with_extension("lock"); let lockfile_path = tarball_path.with_extension("lock");
@@ -537,8 +446,8 @@ impl EphemeralContextGuard {
cmd.arg(series) cmd.arg(series)
.arg(tarball_path.to_string_lossy().to_string()); .arg(tarball_path.to_string_lossy().to_string());
if let Some(u) = ui { if let Some(s) = view.sink() {
cmd.capture(u.sink()); cmd.capture(s);
} }
let status = cmd.status()?; let status = cmd.status()?;
@@ -575,7 +484,7 @@ impl EphemeralContextGuard {
fn extract_tarball( fn extract_tarball(
tarball_path: &PathBuf, tarball_path: &PathBuf,
chroot_path: &PathBuf, chroot_path: &PathBuf,
ui: Option<&DebUi>, view: &dyn BuildView,
) -> Result<(), Box<dyn Error>> { ) -> Result<(), Box<dyn Error>> {
// Create the chroot directory // Create the chroot directory
fs::create_dir_all(chroot_path)?; fs::create_dir_all(chroot_path)?;
@@ -590,18 +499,19 @@ impl EphemeralContextGuard {
// too expensive for multi-hundred-MB chroot tarballs) // too expensive for multi-hundred-MB chroot tarballs)
let mut count = 0usize; let mut count = 0usize;
for entry in archive.entries()? { for entry in archive.entries()? {
// Bail on a Ctrl+C before the interrupt watchdog's rm -rf races
// this loop writing entries into the tree being removed
if crate::interrupt::interrupted() {
return Err("Interrupted while extracting the chroot".into());
}
let mut entry = entry?; let mut entry = entry?;
entry.unpack_in(chroot_path)?; entry.unpack_in(chroot_path)?;
count += 1; count += 1;
if count.is_multiple_of(100) if count.is_multiple_of(100) {
&& let Some(u) = ui view.message(&format!("Extracting chroot… ({count} files)"));
{
u.progress_message(&format!("Extracting chroot… ({count} files)"));
} }
} }
if let Some(u) = ui { view.message(&format!("Extracting chroot… ({count} files)"));
u.progress_message(&format!("Extracting chroot… ({count} files)"));
}
Ok(()) Ok(())
} }
@@ -711,6 +621,20 @@ impl EphemeralContextGuard {
impl Drop for EphemeralContextGuard { impl Drop for EphemeralContextGuard {
fn drop(&mut self) { fn drop(&mut self) {
// On Ctrl+C the interrupt watchdog owns the chroot teardown through
// the registered hook: duplicating it here would race the hook's
// umount/rm (mounts vanish under each other). Dropping this guard
// would normally deregister the hook, so while the watchdog runs it
// must be leaked instead to keep it registered (if it was already
// drained, forgetting is a harmless no-op).
if crate::interrupt::interrupted() {
context::manager().set_current_ephemeral(self.previous_context.clone());
if let Some(hook) = self.cleanup_hook.take() {
std::mem::forget(hook);
}
return;
}
// Deregister the interrupt-time cleanup hook first: the normal // Deregister the interrupt-time cleanup hook first: the normal
// cleanup below takes care of the chroot, so the hook must not fire // cleanup below takes care of the chroot, so the hook must not fire
// afterwards. (If a SIGINT arrived mid-drop and the hook is already // afterwards. (If a SIGINT arrived mid-drop and the hook is already
@@ -808,132 +732,8 @@ impl Drop for EphemeralContextGuard {
} }
#[cfg(test)] #[cfg(test)]
mod cleanup_registry_tests { mod chroot_cleanup_tests {
use super::*; use super::*;
use std::sync::atomic::AtomicUsize;
/// Serializes these tests: they drain the process-global registry, and
/// unrelated tests (e.g. live end-to-end builds) may hold registrations
/// concurrently that must be neither run nor lost. Poison-proof: a test
/// failing while holding the lock must not cascade into the others.
static TEST_LOCK: Mutex<()> = Mutex::new(());
fn test_lock() -> std::sync::MutexGuard<'static, ()> {
TEST_LOCK
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
}
/// Drain the registry and take out only the hooks with the given ids,
/// putting everything else back so unrelated registrations (e.g. hooks of
/// live end-to-end builds running concurrently) stay pending
fn take_hooks(ids: &[u64]) -> Vec<CleanupHook> {
let drained = drain_cleanup_hooks();
let mut mine = Vec::new();
let mut others = Vec::new();
for hook in drained {
if ids.contains(&hook.id) {
mine.push(hook);
} else {
others.push(hook);
}
}
CLEANUP_HOOKS.lock().unwrap().extend(others);
mine
}
/// Register a hook that counts its invocations
fn counting_hook() -> (CleanupHookGuard, Arc<AtomicUsize>) {
let counter = Arc::new(AtomicUsize::new(0));
let seen = counter.clone();
let guard = register_cleanup_hook(Box::new(move || {
seen.fetch_add(1, Ordering::SeqCst);
}));
(guard, counter)
}
/// Hooks run in registration order, and draining means each hook runs
/// exactly once even across repeated cleanup passes.
#[test]
fn hooks_run_once_in_registration_order() {
let _serial = test_lock();
let log = Arc::new(Mutex::new(Vec::new()));
let mut guards = Vec::new();
let mut ids = Vec::new();
for name in ["hook-a", "hook-b", "hook-c"] {
let log = log.clone();
// The returned guard must stay alive: dropping it deregisters
let guard = register_cleanup_hook(Box::new(move || log.lock().unwrap().push(name)));
ids.push(guard.id());
guards.push(guard);
}
// Only our own hooks are extracted; they run in registration order
let mine = take_hooks(&ids);
assert_eq!(mine.len(), ids.len());
run_drained_hooks(mine);
assert_eq!(*log.lock().unwrap(), vec!["hook-a", "hook-b", "hook-c"]);
// Draining removed them: a second pass runs nothing again
assert!(take_hooks(&ids).is_empty());
assert_eq!(*log.lock().unwrap(), vec!["hook-a", "hook-b", "hook-c"]);
drop(guards);
}
/// A panicking hook is contained by the runner: it neither aborts the
/// process nor skips the hooks registered around it.
#[test]
fn panicking_hook_does_not_skip_the_others() {
let _serial = test_lock();
// The hook below panics on purpose: do not record it as a test
// failure in the end-of-run matrix
let _quiet = crate::test_support::suppress_failure_recording();
let (before, ran_before) = counting_hook();
let boom = register_cleanup_hook(Box::new(|| panic!("cleanup exploded")));
let (after, ran_after) = counting_hook();
let ids = [before.id(), boom.id(), after.id()];
run_drained_hooks(take_hooks(&ids));
assert_eq!(ran_before.load(Ordering::SeqCst), 1);
assert_eq!(ran_after.load(Ordering::SeqCst), 1);
}
/// Explicit deregistration removes the hook: it is no longer drained and
/// never runs; a second deregistration reports it as already gone.
#[test]
fn deregistered_hook_never_runs() {
let _serial = test_lock();
let (mut guard, ran) = counting_hook();
assert!(guard.deregister());
assert!(!guard.deregister());
assert!(take_hooks(&[guard.id()]).is_empty());
assert_eq!(ran.load(Ordering::SeqCst), 0);
}
/// Dropping the registration guard deregisters the hook implicitly.
#[test]
fn dropping_the_guard_deregisters_the_hook() {
let _serial = test_lock();
let id;
let ran;
{
let (guard, counter) = counting_hook();
id = guard.id();
ran = counter;
drop(guard);
}
assert!(take_hooks(&[id]).is_empty());
assert_eq!(ran.load(Ordering::SeqCst), 0);
}
/// /proc/mounts path fields use octal escapes for whitespace and /// /proc/mounts path fields use octal escapes for whitespace and
/// backslashes; anything else must be kept verbatim. /// backslashes; anything else must be kept verbatim.
+653 -145
View File
@@ -1,17 +1,19 @@
/// Local binary package building /// Local binary package building
/// Directly calling 'debian/rules' in current context /// Directly calling 'debian/rules' in current context
use crate::context::{Context, ContextCommand, LineSink}; use crate::context::{Context, ContextCommand, LineSink};
use crate::deb::find_dsc_file; use crate::deb::{Phase, enter_phase, find_dsc_file};
use crate::ui::deb::{DebUi, Phase}; use crate::logfmt::QuiltClassifier;
use crate::ui::logfmt::QuiltClassifier; use crate::report::BuildView;
use log::warn; use log::warn;
use std::collections::{BTreeMap, HashMap}; use std::collections::{BTreeMap, BTreeSet, HashMap};
use std::error::Error; use std::error::Error;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::Arc; use std::sync::Arc;
use crate::apt; use crate::apt;
use crate::deb::cross; use crate::deb::cross;
use crate::debian::control::ControlInfo;
use crate::debian::deps::{Deps, Facts, ParseOpts, PkgRelation};
/// Attach the capture sink to a command when the live UI is active /// Attach the capture sink to a command when the live UI is active
fn cap<'a>( fn cap<'a>(
@@ -32,14 +34,15 @@ pub async fn build(
series: &str, series: &str,
pocket: Option<&str>, pocket: Option<&str>,
build_root: &str, build_root: &str,
package_dir: &Path,
cross: bool, cross: bool,
ppa: Option<&[&str]>, ppa: &[String],
inject_packages: Option<&[&str]>, inject_packages: &[String],
ctx: Arc<Context>, ctx: Arc<Context>,
ui: Option<Arc<DebUi>>, view: &dyn BuildView,
jobs: Option<usize>, jobs: Option<usize>,
) -> Result<Vec<PathBuf>, Box<dyn Error>> { ) -> Result<Vec<PathBuf>, Box<dyn Error>> {
let sink: Option<Arc<dyn LineSink>> = ui.as_ref().map(|u| u.sink()); let sink: Option<Arc<dyn LineSink>> = view.sink();
// Environment // Environment
let mut env = HashMap::<String, String>::new(); let mut env = HashMap::<String, String>::new();
@@ -83,55 +86,46 @@ pub async fn build(
let mut added_ppas: Vec<(&str, &str)> = Vec::new(); let mut added_ppas: Vec<(&str, &str)> = Vec::new();
// Add PPA repositories if specified // Add PPA repositories if specified
if let Some(ppas) = ppa { for ppa_str in ppa {
for ppa_str in ppas { let (ppa_user, ppa_name) = crate::package_info::split_ppa(ppa_str)?;
// PPA format: user/ppa_name let base_url = crate::package_info::ppa_to_base_url(ppa_user, ppa_name);
let parts: Vec<&str> = ppa_str.split('/').collect();
if parts.len() == 2 {
let base_url = crate::package_info::ppa_to_base_url(parts[0], parts[1]);
// Add new PPA source if not found // Add new PPA source if not found
if !sources.iter().any(|s| s.uri.contains(&base_url)) { if !sources.iter().any(|s| s.uri.contains(&base_url)) {
// Get host and target architectures // Get host and target architectures
let host_arch = crate::get_current_arch(); let host_arch = crate::get_current_arch();
let target_arch = arch; let target_arch = arch;
// Create architectures list with both host and target if different // Create architectures list with both host and target if different
let mut architectures = vec![host_arch.clone()]; let mut architectures = vec![host_arch.clone()];
if host_arch != *target_arch { if host_arch != *target_arch {
architectures.push(target_arch.to_string()); architectures.push(target_arch.to_string());
}
// Create suite list with all Ubuntu series
let suites = vec![series.to_string()];
let new_source = crate::apt::sources::SourceEntry {
enabled: true,
kind: crate::apt::sources::SourceKind::Deb,
components: vec!["main".to_string()],
architectures: architectures.clone(),
signed_by: None,
trusted: None,
suite: suites,
uri: base_url,
// No origin: saved to the pkh-owned added-sources file
origin: None,
};
sources.push(new_source);
modified = true;
added_ppas.push((parts[0], parts[1]));
log::info!(
"Added PPA: {} for series {} with architectures {:?}",
ppa_str,
series,
architectures
);
}
} else {
return Err(
format!("Invalid PPA format: '{}'. Expected: user/ppa_name", ppa_str).into(),
);
} }
// Create suite list with all Ubuntu series
let suites = vec![series.to_string()];
let new_source = crate::apt::sources::SourceEntry {
enabled: true,
kind: crate::apt::sources::SourceKind::Deb,
components: vec!["main".to_string()],
architectures: architectures.clone(),
signed_by: None,
trusted: None,
suite: suites,
uri: base_url,
// No origin: saved to the pkh-owned added-sources file
origin: None,
};
sources.push(new_source);
modified = true;
added_ppas.push((ppa_user, ppa_name));
log::info!(
"Added PPA: {} for series {} with architectures {:?}",
ppa_str,
series,
architectures
);
} }
} }
@@ -193,9 +187,7 @@ pub async fn build(
// Update package lists // Update package lists
log::debug!("Updating package lists for local build..."); log::debug!("Updating package lists for local build...");
if let Some(u) = &ui { enter_phase(view, Phase::UpdatingPackageLists);
u.phase(Phase::UpdatingPackageLists);
}
let status = cap( let status = cap(
ctx.command("apt-get").envs(env.clone()).arg("update"), ctx.command("apt-get").envs(env.clone()).arg("update"),
&sink, &sink,
@@ -234,17 +226,14 @@ pub async fn build(
cmd.arg(format!("libc6:{arch}")); cmd.arg(format!("libc6:{arch}"));
cmd.arg(format!("libc6-dev:{arch}")); cmd.arg(format!("libc6-dev:{arch}"));
} }
if let Some(u) = &ui { enter_phase(view, Phase::InstallingEssentials);
u.phase(Phase::InstallingEssentials);
}
let status = cap(&mut cmd, &sink).status()?; let status = cap(&mut cmd, &sink).status()?;
if !status.success() { if !status.success() {
return Err("Could not install essential packages for the build".into()); return Err("Could not install essential packages for the build".into());
} }
// Find the actual package directory // The package directory was resolved by the caller (the staged copy of
let package_dir = // the tree the user pointed at, or the name-pattern search fallback)
crate::deb::find_package_directory(Path::new(build_root), package, version, &ctx)?;
let package_dir_str = package_dir let package_dir_str = package_dir
.to_str() .to_str()
.ok_or("Invalid package directory path")?; .ok_or("Invalid package directory path")?;
@@ -261,78 +250,34 @@ pub async fn build(
} }
// Apply quilt patches if the package provides a patch series // Apply quilt patches if the package provides a patch series
apply_quilt_patches(package_dir_str, &env, ctx.clone(), &ui, &sink)?; apply_quilt_patches(package_dir_str, &env, ctx.clone(), view, &sink)?;
// Install injected packages if specified // Install injected packages if specified
if let Some(packages) = inject_packages { if !inject_packages.is_empty() {
install_injected_packages(packages, &env, ctx.clone(), &ui, &sink)?; install_injected_packages(inject_packages, &env, ctx.clone(), view, &sink)?;
} }
// Install arch-specific build dependencies // Resolve and install the Build-* dependencies with dpkg's cross
log::debug!("Installing arch-specific build dependencies..."); // semantics; this replaces the historical `apt-get build-dep` passes,
if let Some(u) = &ui { // whose `--host-architecture` resolution cannot express the
u.phase(Phase::InstallingBuildDeps); // Multi-Arch-aware variant choice dpkg's checker requires.
} install_build_dependencies(
let mut cmd = ctx.command("apt-get"); package,
cmd.current_dir(package_dir_str) version,
.envs(env.clone()) arch,
.arg("-y") series,
.arg("build-dep"); package_dir_str,
if cross { build_root,
cmd.arg(format!("--host-architecture={arch}")); cross,
} &env,
cmd.arg("--arch-only"); ctx.clone(),
let status = cap(&mut cmd, &sink).arg("./").status()?; view,
&sink,
// If build-dep fails, we try to explain the failure using dose-debcheck )?;
if !status.success() {
if let Some(u) = &ui {
u.suspend();
}
dose3_explain_dependencies(package, version, arch, build_root, cross, ctx.clone())?;
return Err("Could not install build-dependencies for the build".into());
}
// Install arch-independant build dependencies, only if the source declares
// any: without --arch-only this pass resolves the whole Build-Depends field
// too, which is redundant after the first pass and breaks cross builds.
let has_indep_deps = match ctx.read_file(&package_dir.join("debian/control")) {
Ok(control) => control
.lines()
.any(|l| l.to_ascii_lowercase().starts_with("build-depends-indep:")),
Err(e) => {
log::debug!("cannot read debian/control for indep build-deps: {}", e);
true
}
};
if has_indep_deps {
log::debug!("Installing arch-independant build dependencies...");
let mut cmd = ctx.command("apt-get");
cmd.current_dir(package_dir_str)
.envs(env.clone())
.arg("-y")
.arg("build-dep");
if cross {
cmd.arg(format!("--host-architecture={arch}"));
}
cmd.arg("./");
let status = cap(&mut cmd, &sink).status()?;
// If build-dep fails, we try to explain the failure using dose-debcheck
if !status.success() {
if let Some(u) = &ui {
u.suspend();
}
dose3_explain_dependencies(package, version, arch, build_root, cross, ctx.clone())?;
return Err("Could not install build-dependencies for the build".into());
}
}
// Run the build step // Run the build step
log::debug!("Building (debian/rules build) package..."); log::debug!("Building (debian/rules build) package...");
if let Some(u) = &ui { enter_phase(view, Phase::Building);
u.phase(Phase::Building);
}
let status = cap( let status = cap(
ctx.command("debian/rules") ctx.command("debian/rules")
.current_dir(package_dir_str) .current_dir(package_dir_str)
@@ -346,9 +291,7 @@ pub async fn build(
} }
// Run the 'binary' step to produce deb // Run the 'binary' step to produce deb
if let Some(u) = &ui { enter_phase(view, Phase::ProducingBinaries);
u.phase(Phase::ProducingBinaries);
}
let status = cap( let status = cap(
ctx.command("fakeroot") ctx.command("fakeroot")
.current_dir(package_dir_str) .current_dir(package_dir_str)
@@ -394,6 +337,416 @@ pub async fn build(
Ok(artifacts) Ok(artifacts)
} }
/// One candidate binary package for a dependency name, as reported by
/// `apt-cache show`.
#[derive(Debug, Clone)]
struct Candidate {
/// Debian architecture of the candidate (`amd64`, `arm64`, `all`, ...).
arch: String,
/// Multi-Arch attribute; an absent field means `no`.
multiarch: String,
}
/// Pure core of the install strategy: the apt install specification that
/// satisfies `rel` among the `candidates` available for its package name.
///
/// The mapping reproduces the dpkg cross semantics that
/// [`crate::debian::deps`] evaluates against (`Dpkg::Deps::KnownFacts
/// _find_package`), preferring the runnable build-architecture variant
/// whenever dpkg's rules accept several:
///
/// - unqualified: a `Multi-Arch: foreign` build-architecture candidate
/// (tools: the variant that runs on the build machine), then the
/// host-architecture candidate (the variant dpkg checks unqualified
/// dependencies against), then an `Architecture: all` candidate. A
/// foreign-arch candidate of a non-foreign package satisfies nothing in
/// cross mode — installing it would diverge from the checker;
/// - `:native`: the build-architecture candidate (`Architecture: all`
/// accepted); a `Multi-Arch: foreign` candidate aborts the lookup,
/// like dpkg's checker;
/// - `:any`: a `Multi-Arch: allowed` candidate, host architecture first;
/// - explicit `:arch`: exactly that candidate;
/// - a name with no candidate at all is passed through unqualified, so
/// apt resolves virtual packages (`debhelper-compat`, dbus session
/// alternatives, ...) to a satisfying provider.
///
/// `None` when no candidate can satisfy the relation per dpkg semantics:
/// the caller falls through to the next alternative of the clause.
fn install_spec_for(
rel: &PkgRelation,
candidates: &[Candidate],
build_arch: &str,
host_arch: &str,
) -> Option<String> {
let qualified = |arch: &str| format!("{}:{}", rel.package, arch);
match rel.arch_qualifier.as_deref() {
Some("native") => {
if candidates
.iter()
.any(|c| c.arch == build_arch && c.multiarch == "foreign")
{
return None;
}
candidates
.iter()
.find(|c| c.arch == build_arch)
.map(|_| qualified(build_arch))
.or_else(|| {
candidates
.iter()
.find(|c| c.arch == "all")
.map(|_| rel.package.clone())
})
}
Some("any") => candidates
.iter()
.filter(|c| c.multiarch == "allowed")
.find(|c| c.arch == host_arch)
.or_else(|| candidates.iter().find(|c| c.multiarch == "allowed"))
.map(|c| qualified(&c.arch)),
Some(qual) => candidates
.iter()
.find(|c| c.arch == qual)
.map(|_| qualified(qual)),
None => {
if candidates.is_empty() {
return Some(rel.package.clone());
}
candidates
.iter()
.find(|c| c.arch == build_arch && c.multiarch == "foreign")
.map(|_| qualified(build_arch))
.or_else(|| {
candidates
.iter()
.find(|c| c.arch == host_arch)
.map(|_| qualified(host_arch))
})
.or_else(|| {
candidates
.iter()
.find(|c| c.arch == "all")
.map(|_| rel.package.clone())
})
}
}
}
/// Pure core of the resolver loop: the next batch of install
/// specifications, one alternative per still-unsatisfied clause,
/// advancing each clause's cursor past the alternatives that no candidate
/// can satisfy per dpkg semantics.
fn next_install_pass(
clauses: &[Vec<PkgRelation>],
unsatisfied: &[usize],
cursors: &mut [usize],
candidates: &BTreeMap<String, Vec<Candidate>>,
build_arch: &str,
host_arch: &str,
) -> Vec<String> {
let mut specs = Vec::new();
for &i in unsatisfied {
let empty = Vec::new();
while cursors[i] < clauses[i].len() {
let rel = &clauses[i][cursors[i]];
let cands = candidates.get(&rel.package).unwrap_or(&empty);
let spec = install_spec_for(rel, cands, build_arch, host_arch);
cursors[i] += 1;
if let Some(spec) = spec {
specs.push(spec);
break;
}
}
}
specs
}
/// Query the candidates available in the build context for `names`: one
/// entry per (package, architecture). The plain name surfaces the
/// build-architecture and `all` candidates; the host-qualified name is
/// what surfaces foreign-arch candidates. Names missing from the indexes
/// and virtual names simply produce no stanza.
fn query_candidates(
ctx: &Arc<Context>,
names: &[String],
build_arch: &str,
host_arch: &str,
) -> BTreeMap<String, Vec<Candidate>> {
let mut query: Vec<String> = Vec::new();
for name in names {
query.push(name.clone());
if host_arch != build_arch {
query.push(format!("{name}:{host_arch}"));
}
}
let Ok(output) = ctx.command("apt-cache").arg("show").args(&query).output() else {
return BTreeMap::new();
};
let mut map: BTreeMap<String, Vec<Candidate>> = BTreeMap::new();
for para in crate::debian::control::parse_paragraphs(&String::from_utf8_lossy(&output.stdout)) {
let Some(name) = para.get("Package") else {
continue;
};
let Some(arch) = para.get("Architecture") else {
continue;
};
let entry = map.entry(name.to_string()).or_default();
if entry.iter().any(|c| c.arch == arch) {
continue;
}
entry.push(Candidate {
arch: arch.to_string(),
multiarch: para.get("Multi-Arch").unwrap_or("no").to_string(),
});
}
map
}
/// Build architecture inside the build context.
fn context_build_arch(ctx: &Arc<Context>) -> String {
ctx.command("dpkg")
.arg("--print-architecture")
.output()
.ok()
.filter(|o| o.status.success())
.map(|o| String::from_utf8_lossy(&o.stdout).trim().to_string())
.filter(|s| !s.is_empty())
.unwrap_or_else(crate::get_current_arch)
}
/// Active build profiles for the dependency evaluation: what the build
/// steps run with (`DEB_BUILD_PROFILES`, set to `cross` for cross
/// builds), else the vendor defaults.
fn build_profiles_for(env: &HashMap<String, String>, ctx: &Arc<Context>) -> Vec<String> {
match env.get("DEB_BUILD_PROFILES") {
Some(value) => value
.split(',')
.map(|p| p.trim().to_string())
.filter(|p| !p.is_empty())
.collect(),
None => {
let vendor = ctx
.read_file(Path::new("/etc/dpkg/origins/default"))
.ok()
.and_then(|content| crate::build::env::vendor_from_origins_content(&content))
.unwrap_or_else(crate::build::env::current_vendor);
crate::build::env::resolve_build_profiles(&[], &vendor)
}
}
}
/// Installed-package facts inside the build context. An unreadable status
/// database (fresh chroot) yields empty facts: everything declared then
/// installs.
fn load_context_facts(ctx: &Arc<Context>, build_arch: &str, host_arch: &str) -> Facts {
match ctx.read_file(Path::new("/var/lib/dpkg/status")) {
Ok(content) => Facts::from_status(&content, host_arch, build_arch),
Err(e) => {
log::debug!("cannot read the context dpkg status: {e}");
Facts::new(host_arch, build_arch)
}
}
}
/// Resolve and install the Build-* dependencies with dpkg's cross
/// semantics: the clauses of `debian/control` are evaluated natively (the
/// `crate::debian::deps` `dpkg-checkbuilddeps` equivalent) against the
/// context's installed-package state, and the unsatisfied ones install
/// through explicitly architecture-qualified names.
///
/// Build conflicts abort before anything installs. The loop installs one
/// alternative per still-unsatisfied clause per apt transaction and
/// re-evaluates after each; a clause whose alternatives no candidate can
/// satisfy per dpkg semantics aborts the build. On failure the dose3
/// diagnosis runs, like the historical `build-dep` passes.
#[allow(clippy::too_many_arguments)]
fn install_build_dependencies(
package: &str,
version: &str,
arch: &str,
series: &str,
package_dir: &str,
build_root: &str,
cross: bool,
env: &HashMap<String, String>,
ctx: Arc<Context>,
view: &dyn BuildView,
sink: &Option<Arc<dyn LineSink>>,
) -> Result<(), Box<dyn Error>> {
enter_phase(view, Phase::InstallingBuildDeps);
let host_arch = arch.to_string();
let build_arch = context_build_arch(&ctx);
let profiles = build_profiles_for(env, &ctx);
let control_content = ctx
.read_file(&Path::new(package_dir).join("debian/control"))
.map_err(|e| format!("cannot read debian/control: {e}"))?;
let control = ControlInfo::parse_content(&control_content)
.map_err(|e| format!("invalid debian/control in {package_dir}: {e}"))?;
let source = &control.source;
let deps_value = ["Build-Depends", "Build-Depends-Arch", "Build-Depends-Indep"]
.iter()
.filter_map(|f| source.get(f))
.collect::<Vec<_>>()
.join(", ");
let conflicts_value = [
"Build-Conflicts",
"Build-Conflicts-Arch",
"Build-Conflicts-Indep",
]
.iter()
.filter_map(|f| source.get(f))
.collect::<Vec<_>>()
.join(", ");
if deps_value.trim().is_empty() && conflicts_value.trim().is_empty() {
return Ok(());
}
let parse_opts = ParseOpts {
host_arch: host_arch.clone(),
build_arch: build_arch.clone(),
build_profiles: profiles,
reduce_restrictions: true,
union: false,
build_dep: true,
};
let mut clauses: Vec<Vec<PkgRelation>> = Deps::parse(&deps_value, &parse_opts)?
.clauses()
.map(<[PkgRelation]>::to_vec)
.collect();
// Package-specific workarounds (see data/quirks.yml), before anything
// derives candidate queries or install specs from the clauses.
crate::quirks::apply_dependency_quirks(package, series, &mut clauses, &parse_opts)?;
let conflict_clauses: Vec<Vec<PkgRelation>> = if conflicts_value.trim().is_empty() {
Vec::new()
} else {
let union_opts = ParseOpts {
union: true,
..parse_opts.clone()
};
Deps::parse(&conflicts_value, &union_opts)?
.clauses()
.map(<[PkgRelation]>::to_vec)
.collect()
};
let names: Vec<String> = clauses
.iter()
.chain(conflict_clauses.iter())
.flatten()
.map(|rel| rel.package.clone())
.collect::<BTreeSet<_>>()
.into_iter()
.collect();
let candidates = query_candidates(&ctx, &names, &build_arch, &host_arch);
let mut cursors = vec![0usize; clauses.len()];
loop {
let facts = load_context_facts(&ctx, &build_arch, &host_arch);
// Build conflicts use the same lookup as dependencies in dpkg's
// checker: a satisfied conflict clause aborts before anything
// installs.
let mut violated = Vec::new();
for alternatives in &conflict_clauses {
for rel in alternatives {
if facts.evaluate_relation(rel) == Some(true) {
violated.push(rel.output());
break;
}
}
}
if !violated.is_empty() {
view.suspend();
if let Err(e) =
dose3_explain_dependencies(package, version, arch, build_root, cross, ctx.clone())
{
log::debug!("dose-builddebcheck diagnosis failed: {e}");
}
return Err(format!(
"build dependencies conflict with installed packages: {}",
violated.join(", ")
)
.into());
}
let unsatisfied: Vec<usize> = (0..clauses.len())
.filter(|&i| {
!clauses[i]
.iter()
.any(|rel| facts.evaluate_relation(rel) == Some(true))
})
.collect();
if unsatisfied.is_empty() {
return Ok(());
}
let specs = next_install_pass(
&clauses,
&unsatisfied,
&mut cursors,
&candidates,
&build_arch,
&host_arch,
);
if specs.is_empty() {
view.suspend();
if let Err(e) =
dose3_explain_dependencies(package, version, arch, build_root, cross, ctx.clone())
{
log::debug!("dose-builddebcheck diagnosis failed: {e}");
}
let remaining: Vec<String> = unsatisfied
.iter()
.map(|&i| {
clauses[i]
.iter()
.map(PkgRelation::output)
.collect::<Vec<_>>()
.join(" | ")
})
.collect();
return Err(format!(
"Could not satisfy build dependencies per dpkg cross semantics: {}",
remaining.join(", ")
)
.into());
}
log::debug!("Installing build dependencies: {:?}", specs);
let mut cmd = ctx.command("apt-get");
cmd.envs(env.clone()).arg("-y").arg("install");
for spec in &specs {
cmd.arg(spec);
}
let status = cap(&mut cmd, sink).status()?;
if !status.success() {
view.suspend();
// Diagnosing a dependency failure the user interrupted themselves
// is wasted work
if !crate::interrupt::interrupted()
&& let Err(e) = dose3_explain_dependencies(
package,
version,
arch,
build_root,
cross,
ctx.clone(),
)
{
log::debug!("dose-builddebcheck diagnosis failed: {e}");
}
return Err(format!(
"Could not install build-dependencies for the build: {}",
specs.join(" ")
)
.into());
}
}
}
/// Collect the binary artifacts (.deb/.udeb) registered by the build in /// Collect the binary artifacts (.deb/.udeb) registered by the build in
/// `debian/files`, returning their paths inside the build context /// `debian/files`, returning their paths inside the build context
/// (`<build_root>/<filename>`). `debian/files` is the canonical record of /// (`<build_root>/<filename>`). `debian/files` is the canonical record of
@@ -495,7 +848,7 @@ fn apply_quilt_patches(
package_dir: &str, package_dir: &str,
env: &HashMap<String, String>, env: &HashMap<String, String>,
ctx: Arc<Context>, ctx: Arc<Context>,
ui: &Option<Arc<DebUi>>, view: &dyn BuildView,
sink: &Option<Arc<dyn LineSink>>, sink: &Option<Arc<dyn LineSink>>,
) -> Result<(), Box<dyn Error>> { ) -> Result<(), Box<dyn Error>> {
let series_path = Path::new(package_dir).join("debian/patches/series"); let series_path = Path::new(package_dir).join("debian/patches/series");
@@ -555,12 +908,10 @@ fn apply_quilt_patches(
} }
// Apply all patches listed in the series // Apply all patches listed in the series
if let Some(u) = ui { view.phase(
u.phase_with( Phase::ApplyingPatches.label(),
Phase::ApplyingPatches, Box::new(QuiltClassifier::new(total_patches)),
Box::new(QuiltClassifier::new(total_patches)), );
);
}
let mut patch_env = env.clone(); let mut patch_env = env.clone();
patch_env.insert("QUILT_PATCHES".to_string(), "debian/patches".to_string()); patch_env.insert("QUILT_PATCHES".to_string(), "debian/patches".to_string());
let status = cap( let status = cap(
@@ -631,17 +982,15 @@ fn pin_pocket(pocket_suite: &str, ctx: &Arc<Context>) -> Result<(), Box<dyn Erro
} }
fn install_injected_packages( fn install_injected_packages(
packages: &[&str], packages: &[String],
env: &HashMap<String, String>, env: &HashMap<String, String>,
ctx: Arc<Context>, ctx: Arc<Context>,
ui: &Option<Arc<DebUi>>, view: &dyn BuildView,
sink: &Option<Arc<dyn LineSink>>, sink: &Option<Arc<dyn LineSink>>,
) -> Result<(), Box<dyn Error>> { ) -> Result<(), Box<dyn Error>> {
log::info!("Installing injected packages: {:?}", packages); log::info!("Installing injected packages: {:?}", packages);
if let Some(u) = ui { enter_phase(view, Phase::InjectingPackages);
u.phase(Phase::InjectingPackages);
}
// Separate .deb files from package names // Separate .deb files from package names
let mut deb_files: Vec<String> = Vec::new(); let mut deb_files: Vec<String> = Vec::new();
@@ -661,7 +1010,7 @@ fn install_injected_packages(
); );
deb_files.push(chroot_path.to_string_lossy().to_string()); deb_files.push(chroot_path.to_string_lossy().to_string());
} else { } else {
package_names.push(pkg); package_names.push(pkg.as_str());
} }
} }
@@ -751,6 +1100,158 @@ mod tests {
use super::*; use super::*;
use crate::context::ContextConfig; use crate::context::ContextConfig;
fn cand(arch: &str, ma: &str) -> Candidate {
Candidate {
arch: arch.to_string(),
multiarch: ma.to_string(),
}
}
fn rel(s: &str) -> PkgRelation {
crate::debian::deps::parse_simple(s, true).unwrap()
}
/// Unqualified dependencies install the dpkg-satisfying variant,
/// preferring the runnable build-architecture one: Multi-Arch:
/// foreign candidates from the build architecture, then the
/// host-architecture candidate dpkg checks them against, then
/// `Architecture: all`. A foreign-arch candidate of a non-foreign
/// package satisfies nothing in cross mode.
#[test]
fn install_spec_unqualified_prefers_runnable_then_host() {
let host = "arm64";
let build = "amd64";
let spec = install_spec_for(
&rel("t"),
&[cand("amd64", "foreign"), cand("arm64", "same")],
build,
host,
)
.unwrap();
assert_eq!(spec, "t:amd64");
let spec = install_spec_for(
&rel("t"),
&[cand("arm64", "same"), cand("amd64", "no")],
build,
host,
)
.unwrap();
assert_eq!(spec, "t:arm64");
assert_eq!(
install_spec_for(&rel("t"), &[cand("amd64", "no")], build, host),
None
);
let spec = install_spec_for(&rel("t"), &[cand("all", "foreign")], build, host).unwrap();
assert_eq!(spec, "t");
// Native builds (host == build): a plain Multi-Arch: no candidate
// is the host candidate.
let spec = install_spec_for(&rel("t"), &[cand("amd64", "no")], "amd64", "amd64").unwrap();
assert_eq!(spec, "t:amd64");
}
/// Qualified dependencies install exactly the variant their qualifier
/// selects per dpkg semantics; virtual names (no candidates) pass
/// through unqualified for apt to resolve a provider.
#[test]
fn install_spec_qualifiers() {
let host = "arm64";
let build = "amd64";
// :native: the build-architecture candidate; an Architecture: all
// candidate passes under the plain name; a Multi-Arch: foreign
// candidate aborts the lookup, like dpkg's checker.
assert_eq!(
install_spec_for(&rel("t:native"), &[cand("amd64", "no")], build, host).unwrap(),
"t:amd64"
);
assert_eq!(
install_spec_for(&rel("t:native"), &[cand("all", "no")], build, host).unwrap(),
"t"
);
assert_eq!(
install_spec_for(&rel("t:native"), &[cand("amd64", "foreign")], build, host),
None
);
// :any: a Multi-Arch: allowed candidate, host first.
assert_eq!(
install_spec_for(
&rel("t:any"),
&[cand("amd64", "allowed"), cand("arm64", "allowed")],
build,
host
)
.unwrap(),
"t:arm64"
);
assert_eq!(
install_spec_for(&rel("t:any"), &[cand("riscv64", "allowed")], build, host).unwrap(),
"t:riscv64"
);
assert_eq!(
install_spec_for(&rel("t:any"), &[cand("amd64", "same")], build, host),
None
);
// Explicit qualifier: exactly that candidate.
assert_eq!(
install_spec_for(&rel("t:arm64"), &[cand("arm64", "same")], build, host).unwrap(),
"t:arm64"
);
assert_eq!(
install_spec_for(&rel("t:arm64"), &[cand("amd64", "same")], build, host),
None
);
// Virtual package: debhelper-compat and friends.
assert_eq!(
install_spec_for(&rel("t (= 13)"), &[], build, host).unwrap(),
"t"
);
}
/// The resolver loop tries the alternatives of a clause in order,
/// falling past the ones no candidate can satisfy, and stops
/// contributing a clause once its alternatives are exhausted.
#[test]
fn next_install_pass_falls_through_alternatives() {
// `a:arm64` has only an amd64 candidate: no candidate satisfies
// the explicit qualifier, so the clause falls through to `t`.
let clauses = vec![
vec![rel("a:arm64"), rel("t")],
vec![rel("u")],
vec![rel("v")],
];
let mut candidates = BTreeMap::new();
candidates.insert("a".to_string(), vec![cand("amd64", "same")]);
candidates.insert("t".to_string(), vec![cand("arm64", "same")]);
candidates.insert("v".to_string(), vec![cand("amd64", "foreign")]);
let mut cursors = vec![0; clauses.len()];
let host = "arm64";
let build = "amd64";
let specs = next_install_pass(&clauses, &[0, 1, 2], &mut cursors, &candidates, build, host);
assert_eq!(
specs,
vec![
"t:arm64".to_string(),
"u".to_string(),
"v:amd64".to_string()
]
);
assert_eq!(cursors, vec![2, 1, 1]);
// An exhausted clause contributes nothing; the loop aborts with
// the remaining clauses when nothing installs any more.
let specs = next_install_pass(&clauses, &[1], &mut cursors, &candidates, build, host);
assert!(specs.is_empty());
}
#[test] #[test]
fn detector_matches_local_options_options_and_dashed_spellings() { fn detector_matches_local_options_options_and_dashed_spellings() {
// The pkh scaffold spelling: bare token in local-options. // The pkh scaffold spelling: bare token in local-options.
@@ -855,6 +1356,13 @@ mod tests {
) )
.unwrap(); .unwrap();
apply_quilt_patches(tree.to_str().unwrap(), &HashMap::new(), ctx, &None, &None).unwrap(); apply_quilt_patches(
tree.to_str().unwrap(),
&HashMap::new(),
ctx,
&crate::report::Quiet,
&None,
)
.unwrap();
} }
} }
+512 -112
View File
@@ -5,7 +5,11 @@ pub(crate) mod ephemeral;
mod local; mod local;
use crate::context::{self, Context}; use crate::context::{self, Context};
use crate::ui::deb::{DebUi, Phase}; use crate::logfmt::{
AptInstallClassifier, AptUpdateClassifier, Classifier, GenericClassifier, MakeClassifier,
MmdebstrapClassifier, QuiltClassifier,
};
use crate::report::{BuildTarget, BuildView};
use std::error::Error; use std::error::Error;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::Arc; use std::sync::Arc;
@@ -17,67 +21,163 @@ pub enum BuildMode {
Local, Local,
} }
/// Phases of a binary build, announced to the [`BuildView`]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Phase {
/// Downloading the chroot tarball (mmdebstrap)
PreparingChroot,
/// Extracting the chroot tarball
ExtractingChroot,
/// Device nodes, /proc bind mount, etc.
FinalizingChroot,
/// apt-get update
UpdatingPackageLists,
/// Installing build-essential & co
InstallingEssentials,
/// quilt push -a
ApplyingPatches,
/// --inject packages
InjectingPackages,
/// apt-get build-dep
InstallingBuildDeps,
/// debian/rules build
Building,
/// fakeroot debian/rules binary
ProducingBinaries,
/// Retrieving produced .deb files
RetrievingArtifacts,
}
impl Phase {
/// Human-readable label displayed in the status bar
pub fn label(&self) -> &'static str {
match self {
Phase::PreparingChroot => "Preparing chroot",
Phase::ExtractingChroot => "Extracting chroot",
Phase::FinalizingChroot => "Finalizing chroot",
Phase::UpdatingPackageLists => "Updating package lists",
Phase::InstallingEssentials => "Installing essential packages",
Phase::ApplyingPatches => "Applying patches",
Phase::InjectingPackages => "Injecting packages",
Phase::InstallingBuildDeps => "Installing build dependencies",
Phase::Building => "Building package",
Phase::ProducingBinaries => "Producing binary packages",
Phase::RetrievingArtifacts => "Retrieving artifacts",
}
}
}
/// Default line classifier rewriting a phase's subprocess output
fn default_classifier(phase: Phase) -> Box<dyn Classifier> {
match phase {
Phase::PreparingChroot => Box::new(MmdebstrapClassifier::new()),
Phase::ExtractingChroot | Phase::FinalizingChroot => Box::new(GenericClassifier::new()),
Phase::UpdatingPackageLists => Box::new(AptUpdateClassifier::new()),
Phase::InstallingEssentials => Box::new(AptInstallClassifier::new("Installing essentials")),
Phase::ApplyingPatches => Box::new(QuiltClassifier::new(0)),
Phase::InjectingPackages => Box::new(AptInstallClassifier::new("Injecting packages")),
Phase::InstallingBuildDeps => {
Box::new(AptInstallClassifier::new("Installing build dependencies"))
}
Phase::Building | Phase::ProducingBinaries => Box::new(MakeClassifier::new()),
Phase::RetrievingArtifacts => Box::new(GenericClassifier::new()),
}
}
/// Enter `phase` on the view with its default line classifier
pub(crate) fn enter_phase(view: &dyn BuildView, phase: Phase) {
view.phase(phase.label(), default_classifier(phase));
}
/// Parameters of one [`build_binary_package`] call.
pub struct DebBuildOptions<'a> {
/// Target architecture; defaults to the host architecture.
pub arch: Option<String>,
/// Target distribution series; defaults to the changelog series
/// (UNRELEASED resolves to the vendor's development series).
pub series: Option<String>,
/// Distribution pocket to resolve build-dependencies from.
pub pocket: Option<String>,
/// Source tree to build; defaults to the process working directory.
pub cwd: Option<PathBuf>,
/// Cross-compile for the target architecture instead of using
/// qemu-binfmt.
pub cross: bool,
/// Build mode; defaults to [`BuildMode::Local`].
pub mode: Option<BuildMode>,
/// PPAs to add for build-dependencies (`user/ppa_name`).
pub ppa: Vec<String>,
/// Packages to inject into the build environment before build-dep
/// (.deb paths, archive names or PPA packages).
pub inject: Vec<String>,
/// Parallel build jobs; defaults to the core count available in the
/// build context.
pub jobs: Option<usize>,
/// Explicit build context; defaults to the current context.
pub ctx: Option<Arc<Context>>,
/// Where build events (phases, progress, outcome) are reported.
pub view: &'a dyn BuildView,
}
impl Default for DebBuildOptions<'_> {
fn default() -> Self {
static QUIET: crate::report::Quiet = crate::report::Quiet;
DebBuildOptions {
arch: None,
series: None,
pocket: None,
cwd: None,
cross: false,
mode: None,
ppa: Vec::new(),
inject: Vec::new(),
jobs: None,
ctx: None,
view: &QUIET,
}
}
}
/// Build package in 'cwd' to a .deb /// Build package in 'cwd' to a .deb
/// ///
/// Returns the list of produced artifacts (.deb files plus the upload /// Returns the list of produced artifacts (.deb files plus the upload
/// metadata `.buildinfo`/`.changes`) retrieved locally, identified from /// metadata `.buildinfo`/`.changes`) retrieved locally, identified from
/// `debian/files` and the native metadata generation rather than by /// `debian/files` and the native metadata generation rather than by
/// globbing the build root (which would surface stale files). When `ui` is /// globbing the build root (which would surface stale files). Subprocess
/// set, a live view (status bar + rolling log pane) is displayed and all /// output is captured through the view's sink (live view + tee log for the
/// subprocess output is captured through it; on failure the widget is /// terminal adapter); on failure the view is cleared and prints a summary
/// cleared and a summary of captured errors is printed. /// of captured errors.
#[allow(clippy::too_many_arguments)]
pub async fn build_binary_package( pub async fn build_binary_package(
arch: Option<&str>, opts: DebBuildOptions<'_>,
series: Option<&str>,
pocket: Option<&str>,
cwd: Option<&Path>,
cross: bool,
mode: Option<BuildMode>,
ppa: Option<&[&str]>,
inject_packages: Option<&[&str]>,
ctx: Option<Arc<Context>>,
ui: Option<Arc<DebUi>>,
jobs: Option<usize>,
) -> Result<Vec<PathBuf>, Box<dyn Error>> { ) -> Result<Vec<PathBuf>, Box<dyn Error>> {
let result = build_binary_package_impl( let view = opts.view;
arch, let result = build_binary_package_impl(opts).await;
series,
pocket,
cwd,
cross,
mode,
ppa,
inject_packages,
ctx,
&ui,
jobs,
)
.await;
if let (Some(u), Err(_)) = (&ui, &result) { if result.is_err() {
u.finish_failure(); view.finish_failure();
} }
result result
} }
/// Implementation of [`build_binary_package`], without failure handling /// Implementation of [`build_binary_package`], without failure handling
#[allow(clippy::too_many_arguments)]
async fn build_binary_package_impl( async fn build_binary_package_impl(
arch: Option<&str>, opts: DebBuildOptions<'_>,
series: Option<&str>,
pocket: Option<&str>,
cwd: Option<&Path>,
cross: bool,
mode: Option<BuildMode>,
ppa: Option<&[&str]>,
inject_packages: Option<&[&str]>,
ctx: Option<Arc<Context>>,
ui: &Option<Arc<DebUi>>,
jobs: Option<usize>,
) -> Result<Vec<PathBuf>, Box<dyn Error>> { ) -> Result<Vec<PathBuf>, Box<dyn Error>> {
let cwd = cwd.unwrap_or_else(|| Path::new(".")); let DebBuildOptions {
ref arch,
ref series,
ref pocket,
ref cwd,
cross,
ref mode,
ref ppa,
ref inject,
ref jobs,
ref ctx,
view,
} = opts;
let cwd = cwd.as_deref().unwrap_or_else(|| Path::new("."));
// Parse changelog to get package name, version and series // Parse changelog to get package name, version and series
let changelog_path = cwd.join("debian/changelog"); let changelog_path = cwd.join("debian/changelog");
@@ -101,44 +201,45 @@ async fn build_binary_package_impl(
&package_series &package_series
}; };
let current_arch = crate::get_current_arch(); let current_arch = crate::get_current_arch();
let arch = arch.unwrap_or(&current_arch); let arch = arch.as_deref().unwrap_or(&current_arch);
// Make sure we select a specific mode, either using user-requested // Make sure we select a specific mode, either using user-requested
// or by using default for user-supplied parameters // or by using default for user-supplied parameters
let mode = if let Some(m) = mode { let default_mode = BuildMode::Local;
m let mode = mode.as_ref().unwrap_or(&default_mode);
} else {
// By default, we use local build
BuildMode::Local
};
// Create an ephemeral unshare context for all Local builds // Create an ephemeral unshare context for all Local builds
// Use qemu_binfmt when target architecture differs from host and cross is not requested // Use qemu_binfmt when target architecture differs from host and cross is not requested
let chroot_arch = if mode == BuildMode::Local && arch != current_arch && !cross { let chroot_arch = if mode == &BuildMode::Local && arch != current_arch && !cross {
Some(arch) Some(arch)
} else { } else {
None None
}; };
// Use provided context or get current // Use provided context or get current
let base_ctx = ctx.unwrap_or_else(context::current); let base_ctx = ctx.clone().unwrap_or_else(context::current);
// Identify the target in the live UI once the changelog is parsed, so // Identify the target in the live view once the changelog is parsed, so
// even the chroot download output is attributed and tee'd // even the chroot download output is attributed and tee'd
if let Some(u) = ui { view.target(BuildTarget {
u.set_target(&package, &version, series, arch); package: &package,
} version: &version,
target: &format!("{series}/{arch}"),
display: format!("Building {package} ({version}) for {series}/{arch}"),
source_only: false,
tee_log: true,
});
// Create an ephemeral unshare context for all Local builds. It is kept in // Create an ephemeral unshare context for all Local builds. It is kept in
// this scope so it outlives the guarded section below and is only dropped // this scope so it outlives the guarded section below and is only dropped
// once the live view has been cleared. // once the live view has been cleared.
let mut guard = if mode == BuildMode::Local { let mut guard = if *mode == BuildMode::Local {
Some( Some(
ephemeral::EphemeralContextGuard::new_with_context( ephemeral::EphemeralContextGuard::new_with_context(
series, series,
chroot_arch, chroot_arch,
base_ctx.clone(), base_ctx.clone(),
ui.clone(), view,
) )
.await?, .await?,
) )
@@ -168,6 +269,19 @@ async fn build_binary_package_impl(
.ok_or("Cannot find parent directory name")?; .ok_or("Cannot find parent directory name")?;
let build_root = format!("{}/{}", build_root, parent_dir_name.to_str().unwrap()); let build_root = format!("{}/{}", build_root, parent_dir_name.to_str().unwrap());
// Resolve the package directory inside the staging area. The tree
// the caller pointed at is authoritative (its changelog defined the
// package/version/series above), so its staged copy wins; the
// name-pattern search only runs as a fallback.
let package_dir = resolve_package_directory(
Path::new(&build_root),
cwd,
&package,
&version,
series,
&build_ctx,
)?;
// Run the build using target build mode. It returns the exact set of // Run the build using target build mode. It returns the exact set of
// artifacts produced by this build (binary packages registered in // artifacts produced by this build (binary packages registered in
// debian/files plus the generated .buildinfo/.changes), as paths // debian/files plus the generated .buildinfo/.changes), as paths
@@ -179,14 +293,15 @@ async fn build_binary_package_impl(
&version, &version,
arch, arch,
series, series,
pocket, pocket.as_deref(),
&build_root, &build_root,
&package_dir,
cross, cross,
ppa, ppa,
inject_packages, inject,
build_ctx.clone(), build_ctx.clone(),
ui.clone(), view,
jobs, *jobs,
) )
.await? .await?
} }
@@ -194,9 +309,7 @@ async fn build_binary_package_impl(
// Retrieve the produced artifacts (binary packages plus the upload // Retrieve the produced artifacts (binary packages plus the upload
// metadata) to the parent directory. // metadata) to the parent directory.
if let Some(u) = ui { enter_phase(view, Phase::RetrievingArtifacts);
u.phase(Phase::RetrievingArtifacts);
}
let total_debs = remote_files.len(); let total_debs = remote_files.len();
let mut artifacts = Vec::with_capacity(total_debs); let mut artifacts = Vec::with_capacity(total_debs);
@@ -206,14 +319,10 @@ async fn build_binary_package_impl(
build_ctx.retrieve_path(remote_file, &local_dest)?; build_ctx.retrieve_path(remote_file, &local_dest)?;
artifacts.push(local_dest); artifacts.push(local_dest);
if let Some(u) = ui { view.progress("Retrieving artifacts", idx + 1, total_debs);
u.count_progress("Retrieving artifacts", idx + 1, total_debs);
}
} }
if let Some(u) = ui { view.finish_success(&artifacts);
u.finish_success(&artifacts, u.elapsed());
}
Ok(artifacts) Ok(artifacts)
} }
@@ -222,9 +331,7 @@ async fn build_binary_package_impl(
// Clear the live view before returning: the ephemeral guard is dropped at // Clear the live view before returning: the ephemeral guard is dropped at
// the end of this function and its cleanup commands (umount, rm -rf of // the end of this function and its cleanup commands (umount, rm -rf of
// the chroot) inherit the terminal, so they must not fight the widget. // the chroot) inherit the terminal, so they must not fight the widget.
if let Some(u) = ui { view.suspend();
u.suspend();
}
// Mark build as successful to trigger chroot cleanup // Mark build as successful to trigger chroot cleanup
if result.is_ok() if result.is_ok()
@@ -236,6 +343,38 @@ async fn build_binary_package_impl(
result result
} }
/// Resolve the package directory for a build inside the staged build root.
///
/// The tree the caller pointed at is authoritative: `cwd`'s changelog
/// already defined the package, version and series for this build, so its
/// staged copy is used outright when it carries a `debian/` tree. The
/// name-pattern search ([`find_package_directory`], including the quirks
/// overrides) only runs when that copy cannot be resolved — a default `.`
/// cwd has no basename, and the pointed-at tree may live outside the staged
/// parent. Embedded callers are the motivation: their working directory
/// names (`tree`, `checkout`, ...) match none of the search patterns.
pub(crate) fn resolve_package_directory(
build_root: &Path,
cwd: &Path,
package: &str,
version: &str,
series: &str,
ctx: &context::Context,
) -> Result<PathBuf, Box<dyn Error>> {
if let Some(tree_name) = cwd.file_name() {
let staged_tree = build_root.join(tree_name);
if ctx.is_dir(&staged_tree)? && ctx.exists(&staged_tree.join("debian"))? {
log::debug!(
"Using the staged copy of {} at {}",
cwd.display(),
staged_tree.display()
);
return Ok(staged_tree);
}
}
find_package_directory(build_root, package, version, series, ctx)
}
/// Find the current package directory by trying both patterns: /// Find the current package directory by trying both patterns:
/// - package/package /// - package/package
/// - package/package-origversion /// - package/package-origversion
@@ -244,10 +383,11 @@ pub(crate) fn find_package_directory(
parent_dir: &Path, parent_dir: &Path,
package: &str, package: &str,
version: &str, version: &str,
series: &str,
ctx: &context::Context, ctx: &context::Context,
) -> Result<PathBuf, Box<dyn Error>> { ) -> Result<PathBuf, Box<dyn Error>> {
// Check quirks first for custom package directories // Check quirks first for custom package directories
let custom_dirs = crate::quirks::get_package_directories(package); let custom_dirs = crate::quirks::get_package_directories(package, series);
for custom_dir in custom_dirs { for custom_dir in custom_dirs {
let package_dir = parent_dir.join(&custom_dir); let package_dir = parent_dir.join(&custom_dir);
if ctx.exists(&package_dir)? && ctx.exists(&package_dir.join("debian"))? { if ctx.exists(&package_dir)? && ctx.exists(&package_dir.join("debian"))? {
@@ -319,11 +459,13 @@ pub(crate) fn find_package_directory(
let entries = ctx.list_files(package_parent)?; let entries = ctx.list_files(package_parent)?;
let mut found_dirs = Vec::new(); let mut found_dirs = Vec::new();
for entry in entries { for entry in entries {
if entry.is_dir() { // list_files yields context-relative paths (e.g. rooted inside
if let Some(file_name) = entry.file_name() { // the chroot for an unshare context): classify through the
found_dirs.push(file_name.to_string_lossy().into_owned()); // context, a host-side stat would miss every entry.
} let is_dir = ctx.is_dir(&entry)?;
log::debug!(" - {}", entry.display()); log::debug!(" - {}", entry.display());
if is_dir && let Some(file_name) = entry.file_name() {
found_dirs.push(file_name.to_string_lossy().into_owned());
} }
} }
@@ -407,6 +549,101 @@ mod tests {
} }
} }
/// An unshare context mapped over `chroot_root`, parented on a local
/// context like the ephemeral build contexts are: exists/list_files/
/// is_dir answer through the path mapping, no namespace privileges
/// needed.
fn unshare_test_context(chroot_root: &Path) -> Context {
let base = Context::new(crate::context::ContextConfig::Local).unwrap();
Context::with_parent(
crate::context::ContextConfig::Unshare {
path: chroot_root.to_string_lossy().to_string(),
parent: None,
},
Arc::new(base),
)
}
/// The staging-area listing must classify entries through the context:
/// an unshare context returns build-root-relative paths that a host-side
/// stat never sees (they live under the chroot root on the host), which
/// used to silently empty the 'Found directories' list of the search
/// failure message — and with it every hint about the actual layout.
#[test]
fn find_package_directory_lists_staged_directories_through_the_context() {
let chroot = tempfile::tempdir().unwrap();
// Staged parent holding a single tree whose name matches none of
// the search patterns (the embedded-caller layout: <job>/tree)
let staged_parent = chroot.path().join("tmp/pkh-build-1/j-42");
std::fs::create_dir_all(staged_parent.join("tree/debian")).unwrap();
let ctx = unshare_test_context(chroot.path());
let err = find_package_directory(
Path::new("/tmp/pkh-build-1/j-42"),
"bc",
"1.07.1-1ubuntu1",
"questing",
&ctx,
)
.expect_err("no candidate matches a tree named 'tree'");
let message = err.to_string();
assert!(
message.contains("Found directories: tree"),
"error should list the staged directories through the context: {message}"
);
}
/// An explicit cwd must resolve to its staged copy even when its name
/// matches none of the search patterns: the pointed-at tree is what the
/// parsed changelog came from.
#[test]
fn resolve_package_directory_prefers_the_pointed_tree() {
let chroot = tempfile::tempdir().unwrap();
let staged_parent = chroot.path().join("tmp/pkh-build-1/j-42");
std::fs::create_dir_all(staged_parent.join("tree/debian/source")).unwrap();
let ctx = unshare_test_context(chroot.path());
let resolved = resolve_package_directory(
Path::new("/tmp/pkh-build-1/j-42"),
Path::new("/work/jobs/j-42/tree"),
"bc",
"1.07.1-1ubuntu1",
"questing",
&ctx,
)
.expect("the staged copy of the pointed-at tree must resolve");
assert_eq!(resolved, PathBuf::from("/tmp/pkh-build-1/j-42/tree"));
}
/// When the pointed-at tree is not in the staging area under its own
/// name, resolution falls back to the name-pattern search.
#[test]
fn resolve_package_directory_falls_back_to_the_name_search() {
let chroot = tempfile::tempdir().unwrap();
let staged_parent = chroot.path().join("tmp/pkh-build-1/j-42");
// Staged copy of a pulled tree: <pkg>/<pkg>-<origversion>
std::fs::create_dir_all(staged_parent.join("bc/bc-1.07.1/debian")).unwrap();
let ctx = unshare_test_context(chroot.path());
let resolved = resolve_package_directory(
Path::new("/tmp/pkh-build-1/j-42"),
// A tree never staged under that name
Path::new("/work/other/checkout"),
"bc",
"1.07.1-1ubuntu1",
"questing",
&ctx,
)
.expect("the pulled-tree layout must resolve via the name search");
assert_eq!(
resolved,
PathBuf::from("/tmp/pkh-build-1/j-42/bc/bc-1.07.1")
);
}
async fn test_build_end_to_end( async fn test_build_end_to_end(
package: &str, package: &str,
series: &str, series: &str,
@@ -440,25 +677,25 @@ mod tests {
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap()); let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap());
// Change directory to the package directory // Change directory to the package directory
let cwd = let cwd = crate::deb::find_package_directory(
crate::deb::find_package_directory(cwd, package, &package_info.stanza.version, &ctx) cwd,
.expect("Cannot find package directory"); package,
&package_info.stanza.version,
series,
&ctx,
)
.expect("Cannot find package directory");
log::debug!("Package directory: {}", cwd.display()); log::debug!("Package directory: {}", cwd.display());
log::info!("Starting binary package build..."); log::info!("Starting binary package build...");
crate::deb::build_binary_package( crate::deb::build_binary_package(DebBuildOptions {
arch, arch: arch.map(str::to_string),
Some(series), series: Some(series.to_string()),
None,
Some(&cwd),
cross, cross,
None, cwd: Some(cwd.to_path_buf()),
None, ctx: Some(ctx),
None, ..Default::default()
Some(ctx), })
None,
None,
)
.await .await
.expect("Cannot build binary package (deb)"); .expect("Cannot build binary package (deb)");
log::info!("Successfully built binary package"); log::info!("Successfully built binary package");
@@ -515,12 +752,34 @@ mod tests {
/// NOTE: Ideally, we want to run this in CI, but it takes more than 1h /// NOTE: Ideally, we want to run this in CI, but it takes more than 1h
/// to fully build the linux-riscv package on an amd64 builder, which is too /// to fully build the linux-riscv package on an amd64 builder, which is too
/// much time /// much time
/// The series is the current LTS (26.04) rather than an interim one:
/// interim series vanish from the mirrors a few months after their EOL
/// (questing is already unreachable), an LTS stays pullable for years.
#[ignore] #[ignore]
#[tokio::test] #[tokio::test]
#[test_log::test] #[test_log::test]
#[cfg(target_arch = "x86_64")] #[cfg(target_arch = "x86_64")]
async fn test_deb_linux_riscv_ubuntu_cross_end_to_end() { async fn test_deb_linux_riscv_ubuntu_cross_end_to_end() {
test_build_end_to_end("linux-riscv", "questing", None, Some("riscv64"), true).await; test_build_end_to_end("linux-riscv", "resolute", None, Some("riscv64"), true).await;
}
/// KNOWN-BROKEN cross build of the noble-era kernel, kept as an
/// ignored fixture to work from. Noble controls declare their build
/// tools unqualified (the `:native` idiom landed later), so exact dpkg
/// semantics demand host-architecture instances of them
/// (python3:riscv64, gcc-13:riscv64, clang-17:riscv64, ...) and the
/// resulting two-architecture install set is unsolvable: t64
/// libraries (libclang1-17t64) conflict with their own foreign-arch
/// variant, and the riscv64 toolchain instances drag depends chains
/// (gcc:riscv64) that do not resolve from the chroot sources. A real
/// run fails at the apt transaction with 'Unable to correct problems'.
/// Resolute-era controls declare :native properly; see the test above.
#[ignore]
#[tokio::test]
#[test_log::test]
#[cfg(target_arch = "x86_64")]
async fn test_deb_linux_riscv_noble_cross_end_to_end() {
test_build_end_to_end("linux-riscv", "noble", None, Some("riscv64"), true).await;
} }
/// This is a specific test case for the latest gcc package on Debian /// This is a specific test case for the latest gcc package on Debian
@@ -620,19 +879,14 @@ mod tests {
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap()); let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap());
crate::deb::build_binary_package( crate::deb::build_binary_package(DebBuildOptions {
Some("arm64"), arch: Some("arm64".to_string()),
Some("noble"), series: Some("noble".to_string()),
None, cwd: Some(pkg_dir),
Some(&pkg_dir), cross: true,
true, ctx: Some(ctx),
None, ..Default::default()
None, })
None,
Some(ctx),
None,
None,
)
.await .await
.expect("Cannot cross-build package declaring Build-Depends-Indep"); .expect("Cannot cross-build package declaring Build-Depends-Indep");
@@ -656,4 +910,150 @@ mod tests {
"arch-independant .deb not produced, got: {deb_files:?}" "arch-independant .deb not produced, got: {deb_files:?}"
); );
} }
/// A cross build whose build-dependencies cannot be satisfied under
/// dpkg's cross semantics must abort at the install step, naming the
/// unsatisfied dependency: the expected-failure counterpart of the
/// cross builds above.
#[tokio::test]
#[test_log::test]
#[cfg(target_arch = "x86_64")]
async fn test_deb_cross_unsatisfiable_build_dep_fails_end_to_end() {
let temp_dir = tempfile::tempdir().unwrap();
let pkg_dir = temp_dir.path().join("pkh-crosstest");
std::fs::create_dir_all(pkg_dir.join("debian/source")).unwrap();
std::fs::write(
pkg_dir.join("debian/changelog"),
"pkh-crosstest (1.0) noble; urgency=medium\n\n \
* Synthetic package declaring an unsatisfiable build-dependency.\n\n \
-- pkh tests <pkh@example.com> Tue, 15 Sep 2026 08:00:00 +0000\n",
)
.unwrap();
std::fs::write(
pkg_dir.join("debian/control"),
"Source: pkh-crosstest\n\
Section: devel\n\
Priority: optional\n\
Maintainer: pkh tests <pkh@example.com>\n\
Standards-Version: 4.7.4\n\
Build-Depends: pkh-no-such-package-xyz\n\
Architecture: any\n\
\n\
Package: pkh-crosstest\n\
Architecture: any\n\
Depends: ${misc:Depends}, ${shlibs:Depends}\n\
Description: Cross-build negative regression package\n \
Declares a build-dependency absent from the archive.\n",
)
.unwrap();
std::fs::write(
pkg_dir.join("debian/rules"),
"#!/usr/bin/make -f\n%:\n\tdh $@\n",
)
.unwrap();
use std::os::unix::fs::PermissionsExt;
let rules = pkg_dir.join("debian/rules");
let mut perms = std::fs::metadata(&rules).unwrap().permissions();
perms.set_mode(0o755);
std::fs::set_permissions(&rules, perms).unwrap();
std::fs::write(pkg_dir.join("debian/source/format"), "3.0 (native)\n").unwrap();
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap());
let err = crate::deb::build_binary_package(DebBuildOptions {
arch: Some("arm64".to_string()),
series: Some("noble".to_string()),
cwd: Some(pkg_dir),
cross: true,
ctx: Some(ctx),
..Default::default()
})
.await
.expect_err("an unsatisfiable build-dependency must fail the cross build");
assert!(
err.to_string().contains("pkh-no-such-package-xyz"),
"error should name the unsatisfied dependency: {err}"
);
}
/// An embedded-caller layout — the tree checked out at <job>/tree, a
/// name matching none of the search patterns — must build: the staged
/// copy of the tree the caller pointed at is resolved directly instead
/// of being re-derived from package/version names (which used to fail
/// with 'Could not find package directory').
#[tokio::test]
#[test_log::test]
async fn test_deb_builds_a_tree_named_directory_end_to_end() {
let temp_dir = tempfile::tempdir().unwrap();
let pkg_dir = temp_dir.path().join("j-42/tree");
std::fs::create_dir_all(pkg_dir.join("debian/source")).unwrap();
std::fs::write(
pkg_dir.join("debian/changelog"),
"pkh-treetest (1.0) noble; urgency=medium\n\n \
* Synthetic package built from a directory named 'tree'.\n\n \
-- pkh tests <pkh@example.com> Tue, 15 Sep 2026 08:00:00 +0000\n",
)
.unwrap();
std::fs::write(
pkg_dir.join("debian/control"),
"Source: pkh-treetest\n\
Section: devel\n\
Priority: optional\n\
Maintainer: pkh tests <pkh@example.com>\n\
Standards-Version: 4.7.4\n\
Build-Depends: debhelper-compat (= 13)\n\
Architecture: any\n\
\n\
Package: pkh-treetest\n\
Architecture: any\n\
Depends: ${misc:Depends}, ${shlibs:Depends}\n\
Description: Package-directory resolution regression package\n \
Its tree lives in a directory whose name matches none of the\n \
package-directory search patterns.\n",
)
.unwrap();
std::fs::write(
pkg_dir.join("debian/rules"),
"#!/usr/bin/make -f\n%:\n\tdh $@\n",
)
.unwrap();
use std::os::unix::fs::PermissionsExt;
let rules = pkg_dir.join("debian/rules");
let mut perms = std::fs::metadata(&rules).unwrap().permissions();
perms.set_mode(0o755);
std::fs::set_permissions(&rules, perms).unwrap();
std::fs::write(pkg_dir.join("debian/source/format"), "3.0 (native)\n").unwrap();
let ctx = Arc::new(Context::new(crate::context::ContextConfig::Local).unwrap());
crate::deb::build_binary_package(DebBuildOptions {
series: Some("noble".to_string()),
cwd: Some(pkg_dir),
ctx: Some(ctx),
..Default::default()
})
.await
.expect("a tree named 'tree' must build");
let deb_files: Vec<String> = std::fs::read_dir(temp_dir.path().join("j-42"))
.unwrap()
.filter_map(|e| e.ok())
.map(|e| e.file_name().to_string_lossy().to_string())
.collect();
assert!(
deb_files
.iter()
.any(|f| f.starts_with("pkh-treetest_1.0_") && f.ends_with(".deb")),
".deb not produced for the 'tree'-named directory, got: {deb_files:?}"
);
}
} }
+1 -4
View File
@@ -151,10 +151,7 @@ fn parse_one_entry(
// --- Body until trailer line ` -- Name <email> Date` // --- Body until trailer line ` -- Name <email> Date`
let mut body_lines: Vec<String> = Vec::new(); let mut body_lines: Vec<String> = Vec::new();
let mut trailer: Option<String> = None; let mut trailer: Option<String> = None;
loop { while let Some(line) = lines.peek().copied() {
let Some(line) = lines.peek().copied() else {
break;
};
let line = line.trim_end(); let line = line.trim_end();
if line.starts_with(" -- ") { if line.starts_with(" -- ") {
trailer = lines.next().map(|l| l.trim_end().to_string()); trailer = lines.next().map(|l| l.trim_end().to_string());
+227
View File
@@ -1349,6 +1349,233 @@ Provides: virt2 (>= 1.0), plain
assert_eq!(facts.evaluate_relation(&o("plain")), Some(false)); assert_eq!(facts.evaluate_relation(&o("plain")), Some(false));
} }
/// Cross-compilation semantics of the package lookup (the `Facts`
/// host/build split), mirroring `Dpkg::Deps::KnownFacts::_find_package`:
/// an unqualified dependency is satisfied by the HOST architecture
/// instance, by any instance of a `Multi-Arch: foreign` package, or by
/// an `Architecture: all` instance — never by a foreign-arch instance
/// of a `Multi-Arch: no`/`same` package. The build-architecture
/// instances only come into play through `:native`.
///
/// Verdicts marked «live» were probed against `dpkg-checkbuilddeps -a
/// <host>` on a real amd64 system carrying the build-arch instances.
#[test]
fn cross_lookup_matrix() {
const O: fn(&str) -> PkgRelation = |s| parse_simple(s, true).unwrap();
// host = arm64 (target), build = amd64 (machine): the instances
// below simulate what a cross-building amd64 machine has installed.
let facts = |ma_build: &str, ma_host: &str| {
let mut f = Facts::new("arm64", "amd64");
if !ma_build.is_empty() {
f.add_installed("t", "1.0", "amd64", ma_build);
}
if !ma_host.is_empty() {
f.add_installed("t", "1.0", "arm64", ma_host);
}
f
};
// Unqualified: only the host-arch instance satisfies...
assert_eq!(
facts("no", "").evaluate_relation(&O("t")),
Some(false),
"M-A:no build-arch instance must not satisfy an unqualified dep"
);
assert_eq!(
facts("same", "").evaluate_relation(&O("t")),
Some(false),
"M-A:same build-arch instance must not satisfy an unqualified dep"
);
assert_eq!(facts("", "no").evaluate_relation(&O("t")), Some(true));
assert_eq!(facts("", "same").evaluate_relation(&O("t")), Some(true));
// ...unless the package is Multi-Arch: foreign («live»: bison,
// flex: the natively-installed variant satisfies the cross check).
assert_eq!(facts("foreign", "").evaluate_relation(&O("t")), Some(true));
assert_eq!(facts("", "foreign").evaluate_relation(&O("t")), Some(true));
// `Architecture: all` instances satisfy unqualified dependencies
// whatever the Multi-Arch attribute.
let mut all = Facts::new("arm64", "amd64");
all.add_installed("t", "1.0", "all", "foreign");
assert_eq!(all.evaluate_relation(&O("t")), Some(true));
let mut all2 = Facts::new("arm64", "amd64");
all2.add_installed("t", "1.0", "all", "no");
assert_eq!(all2.evaluate_relation(&O("t")), Some(true));
// Versioned relations check the first matching instance only:
// insertion order decides which instance a dependency binds to,
// and an unsatisfying version does not fall through to later
// instances.
let mut mixed = Facts::new("arm64", "amd64");
mixed.add_installed("t", "0.5", "arm64", "no");
mixed.add_installed("t", "3.0", "amd64", "no");
assert_eq!(mixed.evaluate_relation(&O("t (>= 1)")), Some(false));
assert_eq!(mixed.evaluate_relation(&O("t (<< 1)")), Some(true));
// `:native`: the build-architecture instance satisfies («live»:
// gcc:native on an amd64 machine, whatever the target); an
// Architecture: all instance does too — but a Multi-Arch: foreign
// instance aborts the whole lookup, even on the build architecture
// («live»: flex:native with natively-installed M-A:foreign flex is
// unmet).
assert_eq!(
facts("no", "").evaluate_relation(&O("t:native")),
Some(true)
);
assert_eq!(
facts("same", "").evaluate_relation(&O("t:native")),
Some(true)
);
assert_eq!(
facts("foreign", "").evaluate_relation(&O("t:native")),
Some(false)
);
// An Architecture: all instance satisfies :native — unless it is
// Multi-Arch: foreign, which aborts the lookup like any foreign
// instance.
assert_eq!(all2.evaluate_relation(&O("t:native")), Some(true));
assert_eq!(all.evaluate_relation(&O("t:native")), Some(false));
assert_eq!(
facts("", "no").evaluate_relation(&O("t:native")),
Some(false)
);
// `:any`: only a Multi-Arch: allowed instance satisfies, on any
// architecture («live»: libssl-dev:any with M-A:same libssl-dev is
// unmet).
assert_eq!(
facts("allowed", "").evaluate_relation(&O("t:any")),
Some(true)
);
assert_eq!(
facts("", "allowed").evaluate_relation(&O("t:any")),
Some(true)
);
assert_eq!(
facts("same", "").evaluate_relation(&O("t:any")),
Some(false)
);
// Explicit architecture qualifier: only that exact instance.
assert_eq!(facts("no", "").evaluate_relation(&O("t:amd64")), Some(true));
assert_eq!(
facts("", "no").evaluate_relation(&O("t:amd64")),
Some(false)
);
}
/// The same cross matrix, validated against the real
/// `dpkg-checkbuilddeps`: for each fixture the exit status and the
/// reported unmet list must match, with host != build (the machine is
/// the native architecture; the host architecture is a foreign one).
#[test]
fn diff_checkbuilddeps_cross_matrix() {
let build_arch = arch::native().unwrap_or_else(|_| "amd64".into());
// Any foreign arch the dpkg tables know; the instances only exist
// in the synthetic status file.
let host_arch = if build_arch == "arm64" {
"riscv64".to_string()
} else {
"arm64".to_string()
};
let mk_status = |entries: &[(&str, &str)]| {
let mut s = String::new();
for (pkg_arch, ma) in entries {
let ma = if ma.is_empty() { "no" } else { ma };
s.push_str(&format!(
"Package: t\nStatus: install ok installed\nVersion: 1.0\nArchitecture: {pkg_arch}\nMulti-Arch: {ma}\n\n"
));
}
s
};
for (name, entries, dep) in [
// Unqualified: build-arch instances never satisfy, host-arch
// and Multi-Arch: foreign do.
("ma-no-build", &[("amd64", "no")] as &[(&str, &str)], "t"),
("ma-same-build", &[("amd64", "same")], "t"),
("ma-no-host", &[("arm64", "no")], "t"),
("ma-same-host", &[("arm64", "same")], "t"),
("ma-foreign-build", &[("amd64", "foreign")], "t"),
("all-build", &[("all", "foreign")], "t"),
// :native and :any qualifiers.
("native-build", &[("amd64", "no")], "t:native"),
("native-foreign-build", &[("amd64", "foreign")], "t:native"),
("any-allowed-build", &[("amd64", "allowed")], "t:any"),
("any-same-build", &[("amd64", "same")], "t:any"),
("explicit-build", &[("amd64", "no")], "t:amd64"),
("explicit-host", &[("arm64", "no")], "t:amd64"),
] {
// Substitute the foreign architecture for fixtures that name
// the host arch explicitly.
let dep = dep.replace("arm64", &host_arch);
let entries: Vec<(String, &str)> = entries
.iter()
.map(|(a, m)| (a.replace("amd64", &build_arch), *m))
.collect();
let entries: Vec<(&str, &str)> =
entries.iter().map(|(a, m)| (a.as_str(), *m)).collect();
let status = mk_status(&entries);
let verdict = |bd: &str, host: &str| {
diff_cross_case(bd, &status, &build_arch, host, |ours, real| {
assert_eq!(ours, real, "verdict mismatch for {name}")
})
};
verdict(&dep, &host_arch);
}
}
/// One differential cross case: run the real `dpkg-checkbuilddeps`
/// with `-a <host>` against a synthetic admindir, and the native
/// checker with the equivalent options on the same control, then hand
/// both verdicts to `compare`.
fn diff_cross_case(
bd: &str,
status: &str,
build_arch: &str,
host_arch: &str,
compare: impl Fn(bool, bool),
) {
let control_text = format!(
"Source: t\nMaintainer: a <a@b.c>\nBuild-Depends: {bd}\n\nPackage: t\nArchitecture: any\nDescription: x\n y\n"
);
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("control"), &control_text).unwrap();
let admindir = dir.path().join("admin");
std::fs::create_dir_all(&admindir).unwrap();
std::fs::write(admindir.join("status"), status).unwrap();
let output = std::process::Command::new("dpkg-checkbuilddeps")
.current_dir(dir.path())
.env("LC_ALL", "C")
.arg("--admindir")
.arg(&admindir)
.arg("-a")
.arg(host_arch)
.arg("-I")
.arg("control")
.output()
.expect("run dpkg-checkbuilddeps (is dpkg-dev installed?)");
let real_ok = output.status.success();
let opts = CheckOpts {
host_arch: host_arch.to_string(),
build_arch: build_arch.to_string(),
build_profiles: Vec::new(),
ignore_arch: false,
ignore_indep: false,
ignore_builtin: true,
admindir: admindir.to_path_buf(),
};
let control = ControlInfo::parse_content(&control_text).unwrap();
let ours_ok = check_build_depends(&control, &opts)
.expect("native check failure")
.is_ok();
compare(ours_ok, real_ok);
}
/// The same undecidable verdicts through the direct facts API: an /// The same undecidable verdicts through the direct facts API: an
/// unreadable provided version and an invalid (non-`=`) provide each /// unreadable provided version and an invalid (non-`=`) provide each
/// leave a versioned relation undecided, while a readable provider /// leave a versioned relation undecided, while a readable provider
+107
View File
@@ -87,6 +87,11 @@ struct DistData {
cross_pockets: Vec<String>, cross_pockets: Vec<String>,
#[serde(default)] #[serde(default)]
build_profiles: Vec<String>, build_profiles: Vec<String>,
/// Changelog suite names aliasing a distro-info series codename
/// ('unstable' for Debian's 'sid'): the two names identify the same
/// series ([`series_suite_alias`], [`resolve_suite_alias`])
#[serde(default)]
suite_aliases: HashMap<String, String>,
series: SeriesInfo, series: SeriesInfo,
} }
@@ -413,6 +418,30 @@ pub async fn get_dist_from_series(series: &str) -> Result<String, Box<dyn Error>
Err(format!("Unknown series: {}", series).into()) Err(format!("Unknown series: {}", series).into())
} }
/// The changelog suite name that aliases the series codename of `dist`
/// (Debian's 'unstable' for 'sid'): the two names identify the same
/// series. `None` when the series carries no suite alias.
pub fn series_suite_alias(dist: &str, series: &str) -> Option<String> {
dist_data(dist)
.ok()?
.suite_aliases
.iter()
.find(|(_suite, codename)| codename.as_str() == series)
.map(|(suite, _)| suite.clone())
}
/// Identify a changelog suite name with the distro-info series codename
/// it aliases (Debian's 'unstable' is 'sid'), and the dist that codename
/// belongs to. `None` when `suite` is not a known alias of any dist.
pub fn resolve_suite_alias(suite: &str) -> Option<(String, String)> {
for (dist, data) in DATA.dist.iter() {
if let Some(codename) = data.suite_aliases.get(suite) {
return Some((dist.clone(), codename.clone()));
}
}
None
}
/// Get the package pockets available for a given distribution, in search order /// Get the package pockets available for a given distribution, in search order
/// ///
/// The main archive ('') comes first so that a search without an explicit /// The main archive ('') comes first so that a search without an explicit
@@ -712,6 +741,30 @@ pub async fn get_debian_series_number(series: &str) -> Result<Option<String>, Bo
Ok(None) Ok(None)
} }
/// The release number of a distribution series, paired with the dist it
/// belongs to: the version column of the series data, stripped to its
/// leading token ("12" for Debian bookworm, "26.04" out of Ubuntu
/// resolute's "26.04 LTS"). `None` when the series carries no version at
/// all (Debian's rolling sid/experimental have an empty column;
/// pseudo-versions like "unstable" pass through, callers validate per
/// vendor). Errors when no known distribution carries the series.
pub async fn get_series_release_number(
series: &str,
) -> Result<Option<(String, String)>, Box<dyn Error>> {
let dist = get_dist_from_series(series).await?;
for info in get_ordered_series(&dist).await? {
if info.series == series {
let number = info
.version
.as_deref()
.and_then(|version| version.split_whitespace().next())
.map(str::to_string);
return Ok(number.map(|number| (dist, number)));
}
}
Ok(None)
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -1011,6 +1064,41 @@ mod tests {
assert!(series.contains(&"jammy".to_string())); assert!(series.contains(&"jammy".to_string()));
} }
/// Suite aliases identify a changelog suite name with the series
/// codename of the same series: Debian's 'unstable' is 'sid'
#[test]
fn test_suite_aliases() {
assert_eq!(
resolve_suite_alias("unstable"),
Some(("debian".to_string(), "sid".to_string()))
);
// A series codename or unknown suite is not an alias
assert_eq!(resolve_suite_alias("sid"), None);
assert_eq!(resolve_suite_alias("noble"), None);
assert_eq!(
series_suite_alias("debian", "sid"),
Some("unstable".to_string())
);
assert_eq!(series_suite_alias("debian", "trixie"), None);
assert_eq!(series_suite_alias("ubuntu", "noble"), None);
}
/// Every suite alias must map to a real series of its dist, or the
/// selector would offer a phantom entry
#[tokio::test]
async fn test_suite_aliases_target_real_series() {
for (dist, data) in DATA.dist.iter() {
for (suite, codename) in &data.suite_aliases {
let series = get_ordered_series_name(dist).await.unwrap_or_default();
assert!(
series.contains(codename),
"suite alias '{suite}' of {dist} maps to '{codename}', \
which is not a known series"
);
}
}
}
#[tokio::test] #[tokio::test]
async fn test_get_dist_from_series() { async fn test_get_dist_from_series() {
assert_eq!(get_dist_from_series("sid").await.unwrap(), "debian"); assert_eq!(get_dist_from_series("sid").await.unwrap(), "debian");
@@ -1033,6 +1121,25 @@ mod tests {
assert!(unknown_number.is_none()); assert!(unknown_number.is_none());
} }
#[tokio::test]
async fn test_get_series_release_number() {
let (dist, bookworm) = get_series_release_number("bookworm")
.await
.unwrap()
.unwrap();
assert_eq!(dist, "debian");
assert_eq!(bookworm, "12");
// Ubuntu LTS rows carry a " LTS" decoration: only the leading
// YY.MM token is the release number
let (dist, noble) = get_series_release_number("noble").await.unwrap().unwrap();
assert_eq!(dist, "ubuntu");
assert_eq!(noble, "24.04");
// No known dist carries the series
assert!(get_series_release_number("not-a-series").await.is_err());
}
#[tokio::test] #[tokio::test]
async fn test_get_keyring_urls_sid() { async fn test_get_keyring_urls_sid() {
// Test that 'sid' returns keyrings from the 3 latest released versions // Test that 'sid' returns keyrings from the 3 latest released versions
+310
View File
@@ -0,0 +1,310 @@
//! Passive interrupt state shared between the CLI and the library.
//!
//! Everything active about Ctrl+C lives in the CLI (`main.rs`): it installs
//! the SIGINT handler, wakes a watchdog thread, prints the interrupt notice
//! and exits with the conventional status 130. This module only holds the
//! state the library's own types need:
//!
//! - the interrupted flag ([`mark_interrupted`] / [`interrupted`]), read by
//! flows so they stand down while the watchdog tears everything down;
//! - the cleanup hook registry ([`register_cleanup_hook`]) for resources
//! that must not outlive the process (e.g. the ephemeral build chroot,
//! see [`crate::deb::ephemeral`]), drained and run by the CLI watchdog
//! right before exiting ([`run_cleanup_hooks`]);
//! - the reporter slot ([`set_reporter`]): the live build view registers
//! how to clear the terminal (and where the full log lives); the CLI
//! runs it as the first step of the shutdown.
//!
//! Nothing here installs signal handlers, prints or exits: a library
//! consumer embedding these types keeps its own signal disposition.
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::{Mutex, PoisonError};
/// How the live view reports an interrupt: it clears the terminal and
/// returns the log-file hint to print below the notice, if any
pub type Reporter = Box<dyn FnOnce() -> Option<String> + Send>;
/// A boxed, send-safe cleanup hook body
type CleanupFn = Box<dyn Fn() + Send>;
/// The reporter run before the cleanup hooks; taken out when it runs
static REPORTER: Mutex<Option<Reporter>> = Mutex::new(None);
/// Whether a Ctrl+C has been intercepted since the CLI installed the
/// handler
static INTERRUPTED: AtomicBool = AtomicBool::new(false);
/// Registry of cleanup hooks waiting to run at interrupt time
static CLEANUP_HOOKS: Mutex<Vec<CleanupHook>> = Mutex::new(Vec::new());
/// Source of the registry ids used to deregister a specific hook
static NEXT_CLEANUP_HOOK_ID: AtomicU64 = AtomicU64::new(1);
/// A pending cleanup hook together with its registry id
struct CleanupHook {
id: u64,
f: CleanupFn,
}
/// Record that a Ctrl+C has been intercepted; called by the CLI signal
/// handler
pub fn mark_interrupted() {
INTERRUPTED.store(true, Ordering::SeqCst);
}
/// Whether a Ctrl+C has been intercepted; flows use this to stay quiet and
/// to leave the cleanup to the CLI watchdog
pub fn interrupted() -> bool {
INTERRUPTED.load(Ordering::SeqCst)
}
/// Register how the live view reports an interrupt: the CLI watchdog runs
/// it as the first step of the shutdown, before the cleanup hooks. At most
/// one reporter runs per process: a later call replaces the one set before.
/// Without any reporter the watchdog only prints the plain notice.
pub fn set_reporter(report: Reporter) {
*REPORTER.lock().unwrap_or_else(PoisonError::into_inner) = Some(report);
}
/// Take the registered reporter out of the slot; `None` when no live view
/// registered one (`--verbose`, piped output)
pub fn take_reporter() -> Option<Reporter> {
REPORTER
.lock()
.unwrap_or_else(PoisonError::into_inner)
.take()
}
/// Register a hook to be run when the process is interrupted (after the
/// reporter), returning a guard whose drop deregisters the hook again.
///
/// Hooks must be self-contained — stored paths plus direct subprocesses —
/// and must never block indefinitely: they run in the watchdog while the
/// interrupted flow is still unwinding, and a second Ctrl+C during cleanup
/// is a no-op.
pub fn register_cleanup_hook(f: CleanupFn) -> CleanupHookGuard {
let id = NEXT_CLEANUP_HOOK_ID.fetch_add(1, Ordering::Relaxed);
CLEANUP_HOOKS
.lock()
.unwrap_or_else(PoisonError::into_inner)
.push(CleanupHook { id, f });
CleanupHookGuard(id)
}
/// RAII handle to a registered cleanup hook: dropping it (or an explicit
/// [`CleanupHookGuard::deregister`]) removes the hook from the registry so
/// the interrupt path can no longer run it
pub struct CleanupHookGuard(u64);
impl CleanupHookGuard {
/// Registry id of the hook (used to filter the registry in tests)
#[cfg(test)]
fn id(&self) -> u64 {
self.0
}
/// Remove the hook from the registry; returns whether it was still
/// pending
pub fn deregister(&mut self) -> bool {
deregister_cleanup_hook(self.0)
}
}
impl Drop for CleanupHookGuard {
fn drop(&mut self) {
deregister_cleanup_hook(self.0);
}
}
/// Remove a hook from the registry; returns whether it was still pending
fn deregister_cleanup_hook(id: u64) -> bool {
let mut hooks = CLEANUP_HOOKS.lock().unwrap_or_else(PoisonError::into_inner);
let len_before = hooks.len();
hooks.retain(|hook| hook.id != id);
hooks.len() != len_before
}
/// Drain and run every registered cleanup hook exactly once.
///
/// Called by the CLI watchdog right before the process exits. Draining uses
/// `try_lock` with a bounded retry instead of a blocking lock as a hard
/// upper bound on interrupt latency: the sequence must never hang waiting
/// for a lock, however unlikely a stalled holder is. Timing out therefore
/// skips cleanup (leaking) rather than hanging.
pub fn run_cleanup_hooks() {
run_drained_hooks(drain_cleanup_hooks());
}
/// Take every pending hook out of the registry, waiting at most ~1s for the
/// registry lock (see [`run_cleanup_hooks`] for why this must not block
/// forever)
fn drain_cleanup_hooks() -> Vec<CleanupHook> {
const RETRIES: usize = 200;
const RETRY_DELAY: std::time::Duration = std::time::Duration::from_millis(5);
for _ in 0..RETRIES {
if let Ok(mut hooks) = CLEANUP_HOOKS.try_lock() {
return std::mem::take(&mut *hooks);
}
std::thread::sleep(RETRY_DELAY);
}
log::error!("Timed out waiting for the cleanup hook registry; skipping interrupt cleanup");
Vec::new()
}
/// Run drained hooks one by one, isolating panics so that one failing hook
/// cannot skip the remaining ones
fn run_drained_hooks(hooks: Vec<CleanupHook>) {
for CleanupHook { id, f } in hooks {
// Hooks are arbitrary user code; assert unwind safety so they can be
// run inside a catching context
if let Err(panic) = std::panic::catch_unwind(std::panic::AssertUnwindSafe(f)) {
log::error!("Cleanup hook {id} panicked: {}", panic_message(&panic));
}
}
}
/// Best-effort message extraction from a panic payload
fn panic_message(panic: &(dyn std::any::Any + Send)) -> String {
if let Some(s) = panic.downcast_ref::<&str>() {
(*s).to_string()
} else if let Some(s) = panic.downcast_ref::<String>() {
s.clone()
} else {
"non-string panic payload".to_string()
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use std::sync::Mutex as StdMutex;
use std::sync::atomic::AtomicUsize;
/// Serializes these tests: they drain the process-global registry, and
/// unrelated tests may hold registrations concurrently that must be
/// neither run nor lost. Poison-proof: a test failing while holding the
/// lock must not cascade into the others.
static TEST_LOCK: StdMutex<()> = StdMutex::new(());
fn test_lock() -> std::sync::MutexGuard<'static, ()> {
TEST_LOCK.lock().unwrap_or_else(PoisonError::into_inner)
}
/// Drain the registry and take out only the hooks with the given ids,
/// putting everything else back so unrelated registrations stay pending
fn take_hooks(ids: &[u64]) -> Vec<CleanupHook> {
let drained = drain_cleanup_hooks();
let mut mine = Vec::new();
let mut others = Vec::new();
for hook in drained {
if ids.contains(&hook.id) {
mine.push(hook);
} else {
others.push(hook);
}
}
CLEANUP_HOOKS
.lock()
.unwrap_or_else(PoisonError::into_inner)
.extend(others);
mine
}
/// Register a hook that counts its invocations
fn counting_hook() -> (CleanupHookGuard, Arc<AtomicUsize>) {
let counter = Arc::new(AtomicUsize::new(0));
let seen = counter.clone();
let guard = register_cleanup_hook(Box::new(move || {
seen.fetch_add(1, Ordering::SeqCst);
}));
(guard, counter)
}
/// Hooks run in registration order, and draining means each hook runs
/// exactly once even across repeated cleanup passes.
#[test]
fn hooks_run_once_in_registration_order() {
let _serial = test_lock();
let log = Arc::new(StdMutex::new(Vec::new()));
let mut guards = Vec::new();
let mut ids = Vec::new();
for name in ["hook-a", "hook-b", "hook-c"] {
let log = log.clone();
// The returned guard must stay alive: dropping it deregisters
let guard = register_cleanup_hook(Box::new(move || {
log.lock().unwrap().push(name);
}));
ids.push(guard.id());
guards.push(guard);
}
// Only our own hooks are extracted; they run in registration order
let mine = take_hooks(&ids);
assert_eq!(mine.len(), ids.len());
run_drained_hooks(mine);
assert_eq!(*log.lock().unwrap(), vec!["hook-a", "hook-b", "hook-c"]);
// Draining removed them: a second pass runs nothing again
assert!(take_hooks(&ids).is_empty());
assert_eq!(*log.lock().unwrap(), vec!["hook-a", "hook-b", "hook-c"]);
drop(guards);
}
/// A panicking hook is contained by the runner: it neither aborts the
/// process nor skips the hooks registered around it.
#[test]
fn panicking_hook_does_not_skip_the_others() {
let _serial = test_lock();
// The hook below panics on purpose: do not record it as a test
// failure in the end-of-run matrix
let _quiet = crate::test_support::suppress_failure_recording();
let (before, ran_before) = counting_hook();
let boom = register_cleanup_hook(Box::new(|| panic!("cleanup exploded")));
let (after, ran_after) = counting_hook();
let ids = [before.id(), boom.id(), after.id()];
run_drained_hooks(take_hooks(&ids));
assert_eq!(ran_before.load(Ordering::SeqCst), 1);
assert_eq!(ran_after.load(Ordering::SeqCst), 1);
}
/// Explicit deregistration removes the hook: it is no longer drained and
/// never runs; a second deregistration reports it as already gone.
#[test]
fn deregistered_hook_never_runs() {
let _serial = test_lock();
let (mut guard, ran) = counting_hook();
assert!(guard.deregister());
assert!(!guard.deregister());
assert!(take_hooks(&[guard.id()]).is_empty());
assert_eq!(ran.load(Ordering::SeqCst), 0);
}
/// Dropping the registration guard deregisters the hook implicitly.
#[test]
fn dropping_the_guard_deregisters_the_hook() {
let _serial = test_lock();
let id;
let ran;
{
let (guard, counter) = counting_hook();
id = guard.id();
ran = counter;
drop(guard);
}
assert!(take_hooks(&[id]).is_empty());
assert_eq!(ran.load(Ordering::SeqCst), 0);
}
}
+19
View File
@@ -41,6 +41,11 @@ struct LaunchpadData {
ssh_host: String, ssh_host: String,
/// Port of the PPA SFTP upload server /// Port of the PPA SFTP upload server
ssh_port: u16, ssh_port: u16,
/// Host of the PPA upload queue over anonymous FTP (the transport
/// `pkh put` degrades to when the SSH connection never comes up)
ftp_host: String,
/// Port of the anonymous FTP upload queue
ftp_port: u16,
/// Upload queue incoming directory template (`{owner}`/`{ppa}`) /// Upload queue incoming directory template (`{owner}`/`{ppa}`)
incoming_template: String, incoming_template: String,
/// PPA package-content (apt repository) URL template /// PPA package-content (apt repository) URL template
@@ -53,6 +58,13 @@ embed_data! {
static ref LAUNCHPAD_DATA: LaunchpadData = "../data/launchpad.yml" static ref LAUNCHPAD_DATA: LaunchpadData = "../data/launchpad.yml"
} }
/// The PPA upload queue over anonymous FTP (host, port): the transport
/// dput-ng's plain `ppa:` profile pushes over, and the one `pkh put`
/// degrades to when the SSH connection itself never comes up.
pub(crate) fn ppa_ftp_queue() -> (String, u16) {
(LAUNCHPAD_DATA.ftp_host.clone(), LAUNCHPAD_DATA.ftp_port)
}
/// Base URL of the Launchpad REST API /// Base URL of the Launchpad REST API
fn api_base() -> &'static str { fn api_base() -> &'static str {
&LAUNCHPAD_DATA.api_base &LAUNCHPAD_DATA.api_base
@@ -422,6 +434,13 @@ mod tests {
assert_eq!(target.login, None); assert_eq!(target.login, None);
} }
/// The anonymous FTP fallback queue resolves from the same data the
/// dput-ng `ppa:` profile uses.
#[test]
fn ppa_ftp_queue_resolves() {
assert_eq!(ppa_ftp_queue(), ("ppa.launchpad.net".to_string(), 21));
}
#[test] #[test]
fn ppa_target_rejects_missing_separator() { fn ppa_target_rejects_missing_separator() {
assert!(ppa_target("just-a-name").is_err()); assert!(ppa_target("just-a-name").is_err());
+14
View File
@@ -19,8 +19,13 @@ pub mod deb;
pub mod debian; pub mod debian;
/// Obtain general information about distribution, series, etc /// Obtain general information about distribution, series, etc
pub mod distro_info; pub mod distro_info;
/// Passive interrupt state: the interrupted flag, the cleanup hook registry
/// and the live view's reporter slot (the CLI owns the signal handling)
pub mod interrupt;
/// Launchpad integration: PPA upload targets and account discovery /// Launchpad integration: PPA upload targets and account discovery
pub mod launchpad; pub mod launchpad;
/// Lint a source tree: lintian wrapper for full parity plus pkh-native checks (`pkh lint`)
pub mod lint;
/// Scaffold a new Debian source package (`pkh new`) /// Scaffold a new Debian source package (`pkh new`)
pub mod new; pub mod new;
/// Obtain information about one or multiple packages /// Obtain information about one or multiple packages
@@ -34,6 +39,15 @@ pub mod put;
/// Handle package-specific quirks and workarounds /// Handle package-specific quirks and workarounds
pub mod quirks; pub mod quirks;
/// Line classifiers rewriting raw subprocess output into display actions
/// and countable progress (pure logic, shared by build views)
pub mod logfmt;
/// Reporting ports: environment-agnostic build observation ([`BuildView`])
/// and question answering ([`Prompter`]), implemented by terminal views,
/// server bridges or the inert [`Quiet`]
pub mod report;
/// Terminal UI helpers (progress bars, live build views, prompts) /// Terminal UI helpers (progress bars, live build views, prompts)
pub mod ui; pub mod ui;
+106
View File
@@ -0,0 +1,106 @@
//! The check trait and the static check registry.
//!
//! Checks are zero-sized structs, one module per packaging area, registered
//! in one explicit array — reviewable, greppable, and free of proc macros.
//! A registry test fails the build on duplicate tag names, undocumented
//! tags or casing drift, so the catalog cannot rot silently.
use crate::lint::collect::LintData;
use crate::lint::emit::Emitter;
use crate::lint::tag::Tag;
/// A group of checks over one area of the packaging (control, changelog,
/// git workflow, ...). One check instance may emit any of the tags it
/// declares; the emitter resolves metadata and applies suppression.
pub trait Check: Sync {
/// Registry identifier, also the `--check` value (one word, e.g. `pkh-git`).
fn id(&self) -> &'static str;
/// Static metadata of every tag this check may emit.
fn tags(&self) -> &'static [Tag];
/// Run against the collected package information, reporting findings
/// through `emit`.
fn run(&self, data: &LintData, emit: &mut Emitter);
}
/// Every registered check, in catalog order.
pub static CHECKS: &[&dyn Check] = &[&super::checks::pkh::PkhGit as &dyn Check];
/// Look up a registered check by its id (`--check` value).
pub fn find_check(id: &str) -> Option<&'static dyn Check> {
CHECKS.iter().copied().find(|check| check.id() == id)
}
/// Look up tag metadata by tag name across the whole registry.
pub fn find_tag(name: &str) -> Option<&'static Tag> {
CHECKS
.iter()
.flat_map(|check| check.tags())
.find(|tag| tag.name == name)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn registry_tag_names_are_unique_and_well_formed() {
let mut seen = Vec::new();
for tag in CHECKS.iter().flat_map(|check| check.tags()) {
assert!(
!seen.contains(&tag.name),
"duplicate tag name: {}",
tag.name
);
seen.push(tag.name);
let valid = |c: char| c.is_ascii_lowercase() || c.is_ascii_digit() || "+.-".contains(c);
assert!(
tag.name.starts_with(valid)
&& tag.name.chars().all(valid)
&& !tag.name.ends_with('-'),
"tag name is not kebab-case: {}",
tag.name
);
assert!(
!tag.description.trim().is_empty(),
"tag without a description: {}",
tag.name
);
}
}
#[test]
fn check_ids_are_unique_and_resolvable() {
let mut seen = Vec::new();
for check in CHECKS {
assert!(
!seen.contains(&check.id()),
"duplicate check id: {}",
check.id()
);
seen.push(check.id());
assert!(
find_check(check.id()).is_some(),
"find_check cannot resolve its own registry: {}",
check.id()
);
}
}
#[test]
fn pkh_native_tags_are_namespaced() {
for check in CHECKS {
if check.id().starts_with("pkh-") {
for tag in check.tags() {
assert!(
tag.name.starts_with("pkh-"),
"pkh-native check '{}' emits non-namespaced tag '{}'",
check.id(),
tag.name
);
}
}
}
}
}
+4
View File
@@ -0,0 +1,4 @@
//! Check areas; one module per area, registered in [`crate::lint::check::CHECKS`].
/// Pkh-native workflow checks (git-centric trees, PPA uploads).
pub mod pkh;
+148
View File
@@ -0,0 +1,148 @@
//! Pkh-native checks: workflow knowledge lintian cannot have, because it
//! lives in pkh's flows (git-centric trees, PPA uploads, scaffolding).
use crate::lint::check::Check;
use crate::lint::collect::LintData;
use crate::lint::emit::Emitter;
use crate::lint::tag::{Certainty, Severity, Tag};
/// Metadata of every tag the pkh-native checks emit.
pub static TAGS: &[Tag] = &[Tag {
name: "pkh-debian-changes-not-committed",
severity: Severity::Warning,
certainty: Certainty::Certain,
experimental: false,
description: "The debian/ directory contains changes that are not committed to git. \
pkh builds and uploads the tree as-is (pkh deb, pkh put); committing first keeps \
the upload and the git history in sync.",
references: &[],
}];
/// Flags `debian/` content that exists in the tree but is not committed to
/// git. Skips trees outside any git repository: archive-pulled sources
/// legitimately have none.
pub struct PkhGit;
impl Check for PkhGit {
fn id(&self) -> &'static str {
"pkh-git"
}
fn tags(&self) -> &'static [Tag] {
TAGS
}
fn run(&self, data: &LintData, emit: &mut Emitter) {
let Some(git) = &data.git else {
return;
};
if git.dirty_debian.is_empty() {
return;
}
let examples: Vec<&str> = git
.dirty_debian
.iter()
.take(3)
.map(String::as_str)
.collect();
let more = if git.dirty_debian.len() > examples.len() {
", ..."
} else {
""
};
emit.tag(
"pkh-debian-changes-not-committed",
format!(
"{} uncommitted change(s) under debian/ (e.g. {}{more})",
git.dirty_debian.len(),
examples.join(", ")
),
);
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::Path;
fn write(path: &Path, content: &str) {
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(path, content).unwrap();
}
fn commit_all(repo: &git2::Repository) {
let signature = git2::Signature::now("pkh test", "test@example.com").unwrap();
let mut index = repo.index().unwrap();
index
.add_all(["*"], git2::IndexAddOption::DEFAULT, None)
.unwrap();
// write_tree alone does not persist the index; without this, a
// committed worktree still reads as index-deleted + untracked.
index.write().unwrap();
let tree_id = index.write_tree().unwrap();
let tree = repo.find_tree(tree_id).unwrap();
repo.commit(Some("HEAD"), &signature, &signature, "init", &tree, &[])
.unwrap();
}
#[test]
fn no_git_repository_is_no_finding() {
let dir = tempfile::tempdir().unwrap();
write(&dir.path().join("debian/control"), "Source: hello\n");
let data = LintData::collect(dir.path(), None, None);
assert!(data.git.is_none());
let mut findings = Vec::new();
let mut emitter = Emitter::new(data.source_name.clone(), &mut findings);
PkhGit.run(&data, &mut emitter);
assert!(findings.is_empty());
}
#[test]
fn clean_worktree_is_no_finding() {
let dir = tempfile::tempdir().unwrap();
write(&dir.path().join("debian/control"), "Source: hello\n");
let repo = git2::Repository::init(dir.path()).unwrap();
commit_all(&repo);
let data = LintData::collect(dir.path(), None, None);
let mut findings = Vec::new();
let mut emitter = Emitter::new(data.source_name.clone(), &mut findings);
PkhGit.run(&data, &mut emitter);
assert!(findings.is_empty());
}
#[test]
fn dirty_debian_tree_is_a_finding() {
let dir = tempfile::tempdir().unwrap();
write(&dir.path().join("debian/control"), "Source: hello\n");
write(&dir.path().join("hello.txt"), "upstream\n");
let repo = git2::Repository::init(dir.path()).unwrap();
commit_all(&repo);
// A modified tracked file and a fresh untracked patch: both count.
write(
&dir.path().join("debian/control"),
"Source: hello\nDepends: x\n",
);
write(&dir.path().join("debian/patches/new.patch"), "...\n");
write(&dir.path().join("hello.txt"), "changed upstream\n");
let data = LintData::collect(dir.path(), None, None);
let mut findings = Vec::new();
let mut emitter = Emitter::new(data.source_name.clone(), &mut findings);
PkhGit.run(&data, &mut emitter);
assert_eq!(findings.len(), 1);
let finding = &findings[0];
assert_eq!(finding.tag_name, "pkh-debian-changes-not-committed");
assert_eq!(finding.letter, 'W');
assert!(
finding
.message
.starts_with("2 uncommitted change(s) under debian/")
);
assert!(finding.message.contains("debian/control"));
assert!(finding.message.contains("debian/patches/new.patch"));
}
}
+98
View File
@@ -0,0 +1,98 @@
//! Collectors: package information gathered once per run and shared by every
//! check, mirroring lintian's collection phase without the on-disk lab.
//! Source trees are small, so everything is computed eagerly except git
//! status, which only exists when the tree is a git worktree.
use std::path::{Path, PathBuf};
use crate::debian::control::ControlInfo;
/// Git worktree state relevant to pkh's git-centric workflow checks.
pub struct GitStatus {
/// Repo-relative paths under `debian/` with uncommitted content
/// (modified, staged or untracked), sorted.
pub dirty_debian: Vec<String>,
}
/// Everything the checks and the report renderer know about the linted tree.
pub struct LintData {
/// Root of the source tree being linted.
pub root: PathBuf,
/// Source package name, resolved from `debian/control`, else from the
/// changelog's first line, else the directory name.
pub source_name: String,
/// Target distribution (`--dist`); unresolved when None.
pub dist: Option<String>,
/// Target series (`--series`); unresolved when None.
pub series: Option<String>,
/// Git worktree state; None when the tree is not inside a git repository.
pub git: Option<GitStatus>,
}
impl LintData {
/// Collect information about the source tree at `root`.
pub fn collect(root: &Path, dist: Option<&str>, series: Option<&str>) -> LintData {
LintData {
root: root.to_path_buf(),
source_name: resolve_source_name(root),
dist: dist.map(str::to_string),
series: series.map(str::to_string),
git: collect_git(root),
}
}
}
/// Source package name from `debian/control`, falling back to the changelog
/// header and then the directory name; the report needs a display name even
/// for broken trees.
fn resolve_source_name(root: &Path) -> String {
if let Ok(control) = ControlInfo::parse(&root.join("debian/control")) {
return control.source_name().to_string();
}
if let Ok(changelog) = std::fs::read_to_string(root.join("debian/changelog"))
&& let Some(first) = changelog.lines().next()
&& let Some(name) = first.split(" (").next()
&& !name.trim().is_empty()
{
return name.trim().to_string();
}
root.file_name()
.map(|name| name.to_string_lossy().to_string())
.unwrap_or_else(|| "package".to_string())
}
/// Dirty paths under `debian/` when the tree lives in a git worktree.
/// Untracked files count: a fresh patch nobody committed is exactly the
/// mistake the workflow checks exist to catch. None when there is no
/// repository (or git is somehow unusable here) — never a finding, since
/// archive-pulled trees legitimately have none.
fn collect_git(root: &Path) -> Option<GitStatus> {
let repo = git2::Repository::discover(root).ok()?;
let workdir = repo.workdir()?;
// Statuses are workdir-relative; trees nested inside a repository only
// care about their own slice of it.
let prefix = root.strip_prefix(workdir).unwrap_or(Path::new(""));
let debian_dir = prefix.join("debian");
let mut options = git2::StatusOptions::new();
options
.include_untracked(true)
.include_ignored(false)
.recurse_untracked_dirs(true);
let statuses = repo.statuses(Some(&mut options)).ok()?;
let mut dirty_debian = Vec::new();
for entry in statuses.iter() {
// CURRENT is the zero flag in libgit2, so any non-empty status is
// some kind of change (worktree or staged).
if entry.status().is_empty() {
continue;
}
let path = entry.path()?;
if Path::new(path).starts_with(&debian_dir) {
dirty_debian.push(path.to_string());
}
}
dirty_debian.sort();
Some(GitStatus { dirty_debian })
}
+118
View File
@@ -0,0 +1,118 @@
//! Findings, the run report, and the emitter checks report through.
//!
//! The emitter is deliberately thin: checks name a tag and give a message;
//! metadata, output letter and explanations come from the registry, so
//! findings from the native engine and findings parsed from the wrapped
//! lintian share one shape and one rendering path.
use crate::lint::check;
use crate::lint::tag::Tag;
/// Where a finding comes from.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Origin {
/// Emitted by pkh's native Rust checks.
Native,
/// Parsed from the wrapped lintian's output.
Lintian,
}
/// One lint finding, whatever produced it.
#[derive(Debug, Clone)]
pub struct Finding {
/// Output letter (`E`, `W`, `I`, `P`, `X`, `O`, `C`): severity for
/// native findings, verbatim from the output line in wrapper mode.
pub letter: char,
/// Stable tag name (`missing-debian-copyright-file`).
pub tag_name: String,
/// Free-form details after the tag name; empty when the tag stands alone.
pub message: String,
/// Package the finding belongs to (`hello`).
pub package: String,
/// Processable type lintian displays after the package name (`source`,
/// `changes`, ...); None for binary findings, which lintian prints
/// without a type.
pub processable_type: Option<String>,
/// Explanation lines shown by `--info`: the `N:` blocks lintian attaches
/// to the finding in wrapper mode, or the native tag's description and
/// references.
pub explanation: Vec<String>,
/// Native or parsed-from-lintian.
pub origin: Origin,
}
impl Finding {
/// Lowercase severity/classification name for this finding's letter, as
/// used in `--fail-on` values and JSON output.
pub fn severity_name(&self) -> &'static str {
match self.letter {
'E' => "error",
'W' => "warning",
'I' => "info",
'P' => "pedantic",
'X' => "experimental",
'O' => "overridden",
_ => "classification",
}
}
}
/// The full result of one lint run: every finding, from every source.
pub struct LintReport {
/// All findings in emission order: the wrapper's first, then the native
/// ones (deduplicated by tag name against the wrapper's).
pub findings: Vec<Finding>,
/// Display name of the linted source package.
pub source_name: String,
/// The wrapper was skipped because lintian is not installed; the
/// renderer prints a notice and the run is native-only.
pub wrapper_unavailable: bool,
/// Run-level notes (`N:` lines): how the linted artifact was obtained,
/// and similar context that is not a finding.
pub notes: Vec<String>,
}
/// Sink checks report findings through. Resolves tag metadata from the
/// registry so checks only ever name the tag they mean.
pub struct Emitter<'a> {
package: String,
findings: &'a mut Vec<Finding>,
}
impl<'a> Emitter<'a> {
/// Emitter for findings of `package` (e.g. the source package name),
/// appending into `findings`.
pub fn new(package: String, findings: &'a mut Vec<Finding>) -> Emitter<'a> {
Emitter { package, findings }
}
/// Emit `tag_name` with `message` as its detail line. Native findings
/// always report on the source package; an unknown tag name (a registry
/// bug) still produces a finding rather than panicking — the registry
/// test makes that path unreachable in practice.
pub fn tag(&mut self, tag_name: &str, message: impl Into<String>) {
let (letter, explanation) = match check::find_tag(tag_name) {
Some(tag) => (tag.letter(), explanation_of(tag)),
None => ('E', Vec::new()),
};
self.findings.push(Finding {
letter,
tag_name: tag_name.to_string(),
message: message.into(),
package: self.package.clone(),
processable_type: Some("source".to_string()),
explanation,
origin: Origin::Native,
});
}
}
/// The explanation `--info` shows for a native tag: its description plus any
/// references, prefixed like lintian's "Please refer to" pointers.
fn explanation_of(tag: &Tag) -> Vec<String> {
let mut lines = vec![tag.description.to_string()];
for reference in tag.references {
lines.push(format!("Please refer to {}", reference));
}
lines
}
+186
View File
@@ -0,0 +1,186 @@
//! `pkh lint`: lint a Debian source tree, with lintian feature parity.
//!
//! The strategy is *wrap first, port second* (see `plans/pkh-lint.md`): the
//! [wrapper](wrapper) runs the installed lintian over an ephemeral source
//! package for day-one parity with every lintian check, while the native
//! engine (check registry, collectors, emitter) hosts pkh-specific workflow
//! checks (`pkh-*` tags) and grows ported lintian checks incrementally.
//! Both findings merge into one report — deduplicated by tag name, which is
//! why native checks mirror lintian's tag names for equivalent checks —
//! rendered as lintian-shaped text or JSON, with lintian's exit-code
//! contract (0 clean, 1 findings at/above `--fail-on`, 2 runtime error).
pub mod check;
pub mod checks;
pub mod collect;
pub mod emit;
pub mod output;
pub mod tag;
pub mod wrapper;
use std::collections::HashSet;
use std::path::PathBuf;
use crate::lint::check::Check;
use crate::lint::collect::LintData;
use crate::lint::emit::{Finding, LintReport};
/// Knobs of one `pkh lint` run, built from the CLI in `main.rs`.
pub struct LintOptions {
/// Source tree to lint.
pub path: PathBuf,
/// Run the native engine only; never invoke the lintian wrapper.
pub native: bool,
/// Levels that make the exit code 1 (`--fail-on`, default: error).
pub fail_on: Vec<output::Level>,
/// Show tag explanations under each finding (`--info`, lintian's `-i`).
pub info: bool,
/// Also display info-level (`I:`) findings (lintian's `-I`).
pub display_info: bool,
/// Also display pedantic (`P:`) findings.
pub pedantic: bool,
/// Also display experimental (`X:`) findings.
pub experimental: bool,
/// Also display overridden (`O:`) findings.
pub show_overrides: bool,
/// Tag names to ignore for this run.
pub suppress_tags: Vec<String>,
/// Run only these native checks (`--check`, repeatable).
pub only_checks: Vec<String>,
/// Ignore any existing `pkh build` output and pack the tree fresh.
pub repack: bool,
/// Emit JSON instead of text.
pub json: bool,
/// Text colorization mode (`--color`, default auto: TTY without NO_COLOR).
pub color: output::ColorMode,
/// Target distribution (debian/ubuntu), when known.
pub dist: Option<String>,
/// Target series, when known.
pub series: Option<String>,
}
/// Run one lint: collect package information, run the native checks, wrap
/// lintian (unless `--native`), and merge everything into one report.
pub fn run(options: &LintOptions) -> Result<LintReport, String> {
let data = LintData::collect(
&options.path,
options.dist.as_deref(),
options.series.as_deref(),
);
let mut report = LintReport {
findings: Vec::new(),
source_name: data.source_name.clone(),
wrapper_unavailable: false,
notes: Vec::new(),
};
if !options.native {
match wrapper::run(&data.root, options)? {
Some(outcome) => {
report.findings.extend(outcome.findings);
report.notes = outcome.notes;
}
None => report.wrapper_unavailable = true,
}
}
let selected: Vec<&'static dyn Check> = if options.only_checks.is_empty() {
check::CHECKS.to_vec()
} else {
options
.only_checks
.iter()
.map(|id| check::find_check(id).ok_or_else(|| format!("Unknown --check '{id}'")))
.collect::<Result<_, _>>()?
};
let mut native = Vec::new();
{
let mut emitter = emit::Emitter::new(report.source_name.clone(), &mut native);
for check in selected {
check.run(&data, &mut emitter);
}
}
report.findings.extend(merge(&report.findings, native));
if !options.suppress_tags.is_empty() {
report
.findings
.retain(|finding| !options.suppress_tags.contains(&finding.tag_name));
}
Ok(report)
}
/// Append `native` findings to `lintian`'s, dropping native duplicates by
/// tag name (the wrapper's verdict wins for tags both engines produce).
fn merge(lintian: &[Finding], native: Vec<Finding>) -> Vec<Finding> {
let known: HashSet<&str> = lintian.iter().map(|f| f.tag_name.as_str()).collect();
native
.into_iter()
.filter(|finding| !known.contains(finding.tag_name.as_str()))
.collect()
}
/// The `--list-tags` catalog: one line per registered native tag.
pub fn list_tags() -> String {
let mut out = String::new();
for check in check::CHECKS {
for tag in check.tags() {
out.push_str(&format!(
"{} [{}] {}\n {}\n",
tag.letter(),
check.id(),
tag.name,
tag.description
));
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use crate::lint::emit::Origin;
fn finding(tag: &str, origin: Origin) -> Finding {
Finding {
letter: 'W',
tag_name: tag.to_string(),
message: String::new(),
package: "hello".to_string(),
processable_type: Some("source".to_string()),
explanation: Vec::new(),
origin,
}
}
#[test]
fn merge_drops_native_duplicates_by_tag_name() {
let lintian = vec![finding("no-debian-copyright-in-source", Origin::Lintian)];
let native = vec![
finding("no-debian-copyright-in-source", Origin::Native),
finding("pkh-debian-changes-not-committed", Origin::Native),
];
let merged = merge(&lintian, native);
assert_eq!(merged.len(), 1);
assert_eq!(merged[0].tag_name, "pkh-debian-changes-not-committed");
assert_eq!(merged[0].origin, Origin::Native);
}
#[test]
fn native_tags_resolve_from_the_registry() {
let tag = check::find_tag("pkh-debian-changes-not-committed")
.expect("pkh check tags must resolve");
assert_eq!(tag.letter(), 'W');
}
#[test]
fn unknown_native_tags_still_produce_findings() {
let mut sink = Vec::new();
let mut emitter = emit::Emitter::new("hello".to_string(), &mut sink);
emitter.tag("no-such-tag-anywhere", "boom");
assert_eq!(sink.len(), 1);
assert_eq!(sink[0].tag_name, "no-such-tag-anywhere");
}
}
+450
View File
@@ -0,0 +1,450 @@
//! Rendering of a lint report (text and JSON) and the exit-code decision.
//!
//! Text output keeps lintian's line shape (`<L>: <pkg> <type>: <tag>
//! <details>`) so findings read identically in both modes; JSON is the
//! schema-stable form for CI. Display filtering (what is shown) and the
//! `--fail-on` threshold (what makes the exit code 1) are independent, like
//! lintian's. Colorization happens here, at render time — findings are
//! captured plain (`--color never` is passed to lintian) and re-painted by
//! severity, so colors are a renderer concern that survives the switch from
//! wrapped lintian output to native checks.
use crossterm::style::Stylize;
use serde_json::json;
use crate::lint::LintOptions;
use crate::lint::emit::{Finding, LintReport, Origin};
/// When to colorize the text report.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum ColorMode {
/// Colorize when stdout is a terminal and `NO_COLOR` is unset.
#[default]
Auto,
/// Always colorize (piping, CI logs).
Always,
/// Never colorize.
Never,
}
impl ColorMode {
/// Parse one `--color` keyword.
pub fn parse(word: &str) -> Option<ColorMode> {
match word {
"auto" => Some(ColorMode::Auto),
"always" => Some(ColorMode::Always),
"never" => Some(ColorMode::Never),
_ => None,
}
}
/// Whether the text renderer should emit ANSI colors.
fn should_color(self) -> bool {
match self {
ColorMode::Always => true,
ColorMode::Never => false,
ColorMode::Auto => {
std::io::IsTerminal::is_terminal(&std::io::stdout())
&& std::env::var_os("NO_COLOR").is_none()
}
}
}
}
/// The output letter, painted with its severity color when `color` is set:
/// errors red (bold), warnings yellow, info cyan, pedantic/experimental
/// magenta, overridden green. Mirrors lintian's tty palette closely enough
/// for muscle memory.
fn paint_letter(letter: char, color: bool) -> String {
if !color {
return letter.to_string();
}
match letter {
'E' => "E".red().bold().to_string(),
'W' => "W".yellow().to_string(),
'I' => "I".cyan().to_string(),
'P' => "P".magenta().to_string(),
'X' => "X".magenta().to_string(),
'O' => "O".green().to_string(),
_ => letter.to_string(),
}
}
/// A named severity class, mirroring the values lintian's `--fail-on`
/// accepts. `Experimental` is the `X:` pseudo-level (the letter hides the
/// underlying severity) and `Overridden` lets gates count suppressed tags.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Level {
/// `E:` findings.
Error,
/// `W:` findings.
Warning,
/// `I:` findings.
Info,
/// `P:` findings.
Pedantic,
/// `X:` findings.
Experimental,
/// `O:` findings (only reported when overrides are shown).
Overridden,
}
impl Level {
/// The output letter the level stands for.
pub fn letter(self) -> char {
match self {
Level::Error => 'E',
Level::Warning => 'W',
Level::Info => 'I',
Level::Pedantic => 'P',
Level::Experimental => 'X',
Level::Overridden => 'O',
}
}
/// Parse one comma-separated `--fail-on` keyword.
pub fn parse(word: &str) -> Option<Level> {
match word.trim() {
"error" => Some(Level::Error),
"warning" => Some(Level::Warning),
"info" => Some(Level::Info),
"pedantic" => Some(Level::Pedantic),
"experimental" => Some(Level::Experimental),
"override" => Some(Level::Overridden),
_ => None,
}
}
}
/// Parse the whole `--fail-on` value (comma-separated level names).
pub fn parse_fail_on(spec: &str) -> Result<Vec<Level>, String> {
let levels: Vec<Level> = spec
.split(',')
.filter(|word| !word.trim().is_empty())
.map(Level::parse)
.collect::<Option<_>>()
.ok_or_else(|| {
format!(
"Invalid --fail-on value '{spec}': expected comma-separated \
error, warning, info, pedantic, experimental or override"
)
})?;
if levels.is_empty() {
return Err("Empty --fail-on value: expected at least one level".to_string());
}
Ok(levels)
}
/// The letter a finding displays as for display-level purposes: overridden
/// findings carry their `O:` letter, classification tags are JSON-only.
fn is_displayed(finding: &Finding, options: &LintOptions) -> bool {
match finding.letter {
'E' | 'W' => true,
'I' => options.display_info,
'P' => options.pedantic,
'X' => options.experimental,
'O' => options.show_overrides,
_ => false,
}
}
/// Exit code of the run: 1 when any finding (except classification tags) is
/// at a level listed in `--fail-on`, 0 otherwise. Runtime failures never
/// reach this function — the caller exits 2 directly.
pub fn exit_code(report: &LintReport, options: &LintOptions) -> i32 {
let failed = report
.findings
.iter()
.any(|finding| match level_of_letter(finding.letter) {
Some(level) => options.fail_on.contains(&level),
None => false,
});
if failed { 1 } else { 0 }
}
/// The level a finding's letter maps to for `--fail-on` purposes; None for
/// classification tags, which lintian also never fails on.
fn level_of_letter(letter: char) -> Option<Level> {
match letter {
'E' => Some(Level::Error),
'W' => Some(Level::Warning),
'I' => Some(Level::Info),
'P' => Some(Level::Pedantic),
'X' => Some(Level::Experimental),
'O' => Some(Level::Overridden),
_ => None,
}
}
/// Render the report as lintian-shaped text: one `<L>: <pkg> <type>: <tag>
/// <details>` line per displayed finding, explanations under `--info`, then
/// an `N:` summary line.
pub fn render_text(report: &LintReport, options: &LintOptions) -> String {
let color = options.color.should_color();
let mut out = String::new();
if report.wrapper_unavailable {
out.push_str(
"N: lintian is not installed; showing pkh-native checks only \
(install lintian for full check coverage)\n",
);
}
for note in &report.notes {
out.push_str(&format!("N: {note}\n"));
}
let mut shown_counts = [('E', 0), ('W', 0), ('I', 0), ('P', 0), ('X', 0), ('O', 0)];
let mut shown = 0;
for finding in &report.findings {
if !is_displayed(finding, options) {
continue;
}
shown += 1;
if let Some((_, count)) = shown_counts
.iter_mut()
.find(|(letter, _)| *letter == finding.letter)
{
*count += 1;
}
out.push_str(&format!(
"{}: {}\n",
paint_letter(finding.letter, color),
line_subject(finding)
));
if options.info {
for line in &finding.explanation {
if line.is_empty() {
out.push_str("N:\n");
} else {
out.push_str(&format!("N: {}\n", line.trim_end()));
}
}
}
}
let mut parts = Vec::new();
for (letter, count) in shown_counts {
if count > 0 {
parts.push(format!("{} {}", count, paint_letter(letter, color)));
}
}
let hidden = report.findings.len() - shown;
if parts.is_empty() {
out.push_str(&format!("N: no displayed tags; {hidden} hidden\n"));
} else {
out.push_str(&format!(
"N: {shown} tag(s) shown ({}); {hidden} hidden\n",
parts.join(", ")
));
}
out
}
/// `hello source: tag details` — the part of a lintian line after the
/// letter, reproduced identically for both origins.
fn line_subject(finding: &Finding) -> String {
let mut line = String::from(&finding.package);
if let Some(ptype) = &finding.processable_type {
line.push(' ');
line.push_str(ptype);
}
line.push_str(": ");
line.push_str(&finding.tag_name);
if !finding.message.is_empty() {
line.push(' ');
line.push_str(&finding.message);
}
line
}
/// Render the report as pretty-printed JSON: every finding (displayed or
/// not, flagged as such) plus a summary carrying the exit-code verdict.
pub fn render_json(report: &LintReport, options: &LintOptions) -> String {
let findings: Vec<serde_json::Value> = report
.findings
.iter()
.map(|finding| {
json!({
"letter": finding.letter.to_string(),
"tag": finding.tag_name,
"severity": finding.severity_name(),
"package": finding.package,
"processable_type": finding.processable_type,
"message": finding.message,
"origin": match finding.origin {
Origin::Native => "native",
Origin::Lintian => "lintian",
},
"overridden": finding.letter == 'O',
"displayed": is_displayed(finding, options),
"explanation": if finding.explanation.is_empty() {
json!(null)
} else {
json!(finding.explanation)
},
})
})
.collect();
let document = json!({
"source": report.source_name,
"wrapper_unavailable": report.wrapper_unavailable,
"notes": report.notes,
"findings": findings,
"summary": {
"failed": exit_code(report, options) == 1,
},
});
serde_json::to_string_pretty(&document).expect("lint report JSON is serializable")
}
#[cfg(test)]
mod tests {
use super::*;
fn options(fail_on: &[Level]) -> LintOptions {
LintOptions {
path: std::path::PathBuf::from("."),
native: false,
fail_on: fail_on.to_vec(),
info: false,
display_info: false,
pedantic: false,
experimental: false,
show_overrides: false,
suppress_tags: Vec::new(),
only_checks: Vec::new(),
repack: false,
json: false,
color: ColorMode::Never,
dist: None,
series: None,
}
}
fn report(findings: &[(&str, char)]) -> LintReport {
LintReport {
findings: findings
.iter()
.map(|(tag, letter)| Finding {
letter: *letter,
tag_name: (*tag).to_string(),
message: String::new(),
package: "hello".to_string(),
processable_type: Some("source".to_string()),
explanation: Vec::new(),
origin: Origin::Lintian,
})
.collect(),
source_name: "hello".to_string(),
wrapper_unavailable: false,
notes: Vec::new(),
}
}
#[test]
fn fail_on_parses_and_rejects_unknown_levels() {
assert_eq!(
parse_fail_on("error, warning").unwrap(),
vec![Level::Error, Level::Warning]
);
assert!(parse_fail_on("bogus").is_err());
assert!(parse_fail_on(" ").is_err());
}
#[test]
fn exit_code_only_counts_displayed_severity_letters() {
let default_opts = options(&[Level::Error]);
// Errors fail, warnings alone do not (the default threshold).
assert_eq!(exit_code(&report(&[("t", 'E')]), &default_opts), 1);
assert_eq!(
exit_code(&report(&[("t", 'W'), ("u", 'I')]), &default_opts),
0
);
// Overridden findings never fail unless explicitly requested.
assert_eq!(exit_code(&report(&[("t", 'O')]), &default_opts), 0);
assert_eq!(
exit_code(
&report(&[("t", 'O')]),
&options(&[Level::Error, Level::Overridden])
),
1
);
// Classification tags never fail.
assert_eq!(exit_code(&report(&[("t", 'C')]), &default_opts), 0);
}
#[test]
fn text_rendering_respects_display_levels() {
let base = report(&[("e", 'E'), ("i", 'I'), ("o", 'O'), ("x", 'X')]);
let default = render_text(&base, &options(&[Level::Error]));
assert!(default.contains("E: hello source: e"));
assert!(!default.contains("I: "));
assert!(!default.contains("O: "));
assert!(!default.contains("X: "));
assert!(default.ends_with("N: 1 tag(s) shown (1 E); 3 hidden\n"));
let everything = LintOptions {
display_info: true,
experimental: true,
show_overrides: true,
..options(&[Level::Error])
};
let full = render_text(&base, &everything);
for prefix in ["E:", "I:", "O:", "X:"] {
assert!(
full.contains(&format!("{prefix} hello source:")),
"{prefix}"
);
}
assert!(full.ends_with("N: 4 tag(s) shown (1 E, 1 I, 1 X, 1 O); 0 hidden\n"));
}
#[test]
fn colorization_is_render_only_and_mode_switchable() {
let run_report = report(&[("e", 'E'), ("w", 'W'), ("o", 'O')]);
let plain = options(&[Level::Error]);
let mut forced = options(&[Level::Error]);
forced.color = ColorMode::Always;
let plain = render_text(&run_report, &plain);
let colored = render_text(&run_report, &forced);
assert!(
!plain.contains('\u{1b}'),
"never/auto-on-pipe must stay plain"
);
assert!(colored.contains('\u{1b}'), "always must colorize");
// Colors wrap the letters only; the lintian line shape is intact.
assert!(colored.contains("hello source: e"));
// Mode parsing round-trips.
assert_eq!(ColorMode::parse("always"), Some(ColorMode::Always));
assert_eq!(ColorMode::parse("bogus"), None);
}
#[test]
fn json_round_trips_with_verdicts() {
let run_report = report(&[("e", 'E'), ("w", 'W')]);
let default_opts = options(&[Level::Error]);
let document: serde_json::Value =
serde_json::from_str(&render_json(&run_report, &default_opts)).unwrap();
assert_eq!(document["source"], "hello");
assert_eq!(document["findings"][0]["letter"], "E");
assert_eq!(document["findings"][0]["severity"], "error");
assert_eq!(document["findings"][0]["origin"], "lintian");
assert_eq!(document["summary"]["failed"], true);
// An error finding fails at the warning threshold too, but not at
// pedantic-only, which nothing in this report reaches.
let warn_only = options(&[Level::Warning]);
assert_eq!(exit_code(&run_report, &warn_only), 1);
let document: serde_json::Value =
serde_json::from_str(&render_json(&run_report, &warn_only)).unwrap();
assert_eq!(document["summary"]["failed"], true);
let pedantic_only = options(&[Level::Pedantic]);
assert_eq!(exit_code(&run_report, &pedantic_only), 0);
let document: serde_json::Value =
serde_json::from_str(&render_json(&run_report, &pedantic_only)).unwrap();
assert_eq!(document["summary"]["failed"], false);
}
}
+85
View File
@@ -0,0 +1,85 @@
//! Lint tag model: severities, certainties and static tag metadata.
//!
//! Tags are the atomic diagnostics of a lint run, named after lintian's
//! model: a stable machine-readable name (`missing-debian-copyright-file`),
//! a severity, a certainty and a description. Checks declare the tags they
//! may emit as static [`Tag`] values; the wrapper's parsed findings carry the
//! letter lintian printed instead.
/// Severity of a finding, matching lintian's severity ladder.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub enum Severity {
/// Policy violation or broken package data (`E:`).
Error,
/// Probable bug or policy deviation (`W:`).
Warning,
/// Informational note about packaging choices (`I:`).
Info,
/// Nitpick most packages may legitimately ignore (`P:`).
Pedantic,
}
impl Severity {
/// The output letter lintian displays this severity as (`E`, `W`, ...).
pub fn letter(self) -> char {
match self {
Severity::Error => 'E',
Severity::Warning => 'W',
Severity::Info => 'I',
Severity::Pedantic => 'P',
}
}
/// Lowercase name used in `--fail-on` values and JSON output.
pub fn name(self) -> &'static str {
match self {
Severity::Error => "error",
Severity::Warning => "warning",
Severity::Info => "info",
Severity::Pedantic => "pedantic",
}
}
}
/// How sure a check is that a finding is real. Not acted upon yet (a future
/// `--fail-on error,certain` would consume it), but captured from day one so
/// severity tuning is data-driven later.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Certainty {
/// The finding is a fact (e.g. a missing file).
Certain,
/// The finding is likely but has legitimate exceptions.
Possible,
/// The finding is a guess from weak signals.
WildGuess,
}
/// Static metadata of one tag: what checks declare and the renderer resolves.
#[derive(Debug)]
pub struct Tag {
/// Stable machine-readable name, lintian-compatible kebab-case
/// (`pkh-debian-changes-not-committed`).
pub name: &'static str,
/// Severity the tag reports at.
pub severity: Severity,
/// How sure checks are when emitting this tag.
pub certainty: Certainty,
/// Whether the tag is experimental (`X:` output, hidden by default).
pub experimental: bool,
/// One-paragraph explanation, shown by `--info` and `--list-tags`.
pub description: &'static str,
/// References (policy sections, URLs) shown by `--info`.
pub references: &'static [&'static str],
}
impl Tag {
/// The output letter for this tag: experimental tags render as `X:`
/// regardless of their severity, like lintian.
pub fn letter(&self) -> char {
if self.experimental {
'X'
} else {
self.severity.letter()
}
}
}
+531
View File
@@ -0,0 +1,531 @@
//! The lintian wrapper: pkh's day-one feature-parity layer.
//!
//! Lintian only accepts built package files, never source trees, so the
//! wrapper needs a source artifact. It lints the `pkh build` output next to
//! the tree when it matches the current changelog entry and nothing in the
//! tree is newer than it (the fast path: no packing at all); otherwise it
//! packs the tree fresh with `dpkg-source -b` inside a temporary directory
//! and lints the resulting `.dsc` (`--repack` forces that path). Nothing is
//! written back into the linted tree; the temp directory is removed on drop.
//!
//! Parsing is load-bearing here (findings are merged with the native ones,
//! rendered uniformly and exported as JSON), so the parser is pinned by
//! golden tests captured from real lintian output. Lintian's own exit code
//! is *not* authoritative: lintian uses 2 both for "fail-on met" and for
//! runtime errors, while pkh derives the verdict from the parsed findings
//! and reserves 2 for actual runtime failures.
use std::io::ErrorKind;
use std::path::{Path, PathBuf};
use std::process::Command;
use std::time::{SystemTime, UNIX_EPOCH};
use regex::Regex;
use crate::lint::LintOptions;
use crate::lint::emit::{Finding, Origin};
/// What one wrapper run produced: the parsed lintian findings plus `N:`
/// notes about how the artifact being linted was obtained.
pub struct WrapperOutcome {
/// Lintian's findings.
pub findings: Vec<Finding>,
/// Human-visible notes (artifact reuse, fresh packing) rendered as `N:`.
pub notes: Vec<String>,
}
/// Run the wrapper over the source tree at `root`:
/// - `Ok(Some(outcome))` — lintian ran; these are its findings and notes,
/// - `Ok(None)` — lintian is not installed; the caller falls back to
/// native-only with a notice,
/// - `Err(message)` — runtime failure (tree unpacked badly, lintian crashed
/// without reportable output); the caller exits 2.
///
/// The artifact linted is the `pkh build` output next to the tree when it
/// matches the current changelog entry and no tree content is newer;
/// otherwise the tree is packed fresh (with `--repack` forcing that path).
pub fn run(root: &Path, options: &LintOptions) -> Result<Option<WrapperOutcome>, String> {
let tmp = TempDir::new()?;
let root = root
.canonicalize()
.map_err(|e| format!("Cannot lint '{}': {e}", root.display()))?;
let (dsc, notes) = match usable_build_output(&root, options.repack) {
Some(dsc) => {
let note = format!(
"linting pkh build output {}",
crate::report::display_path(&dsc)
);
(dsc, vec![note])
}
None => {
let dsc = pack(&root, &tmp)?;
let reason = if options.repack {
"the tree was packed fresh with dpkg-source (--repack ignored \
the existing pkh build output)"
.to_string()
} else if expected_build_output(&root).is_some() {
"the tree changed since pkh build, so it was packed fresh \
with dpkg-source (rerun pkh build to lint the build output)"
.to_string()
} else {
"no pkh build output next to the tree, so it was packed fresh \
with dpkg-source (pkh build produces one)"
.to_string()
};
(dsc, vec![reason])
}
};
let mut lintian = Command::new("lintian");
lintian
.env("LC_ALL", "C")
.args(["--no-cfg", "--color", "never"])
.arg("--info")
.arg(&dsc);
// Per-distro scoping: lintian auto-detects the *host* vendor, but pkh
// knows the *target* distro (-d); make the two agree, which matters on
// cross-distro hosts (linting an Ubuntu package on Debian or back).
if let Some(dist) = &options.dist
&& matches!(dist.as_str(), "ubuntu" | "debian")
{
lintian.arg("--profile").arg(dist);
}
if options.display_info {
lintian.arg("--display-info");
}
if options.pedantic {
lintian.arg("--pedantic");
}
if options.experimental {
lintian.arg("--display-experimental");
}
if options.show_overrides {
lintian.arg("--show-overrides");
}
if !options.suppress_tags.is_empty() {
lintian
.arg("--suppress-tags")
.arg(options.suppress_tags.join(","));
}
let output = match lintian.output() {
Ok(output) => output,
Err(e) if e.kind() == ErrorKind::NotFound => return Ok(None),
Err(e) => return Err(format!("Could not run lintian: {e}")),
};
let stdout = String::from_utf8_lossy(&output.stdout);
let findings = parse(&stdout);
if !output.status.success() && findings.is_empty() {
return Err(format!(
"lintian exited with {} without reporting findings:\n{}",
output.status,
String::from_utf8_lossy(&output.stderr).trim()
));
}
Ok(Some(WrapperOutcome { findings, notes }))
}
/// Pack the tree into an ephemeral source package and return its `.dsc`.
///
/// Compression is deliberately weak (`-Zgzip -z1`): the artifact only ever
/// goes to lintian and is deleted with the temp directory, and xz on a
/// large tree dominates the whole run (measured: 9.8 s xz vs 2.7 s gzip on
/// a 111 MB tree). Locale-independent subprocess output: dpkg messages can
/// be localized, and the error sniffing relies on English wording.
fn pack(root: &Path, tmp: &TempDir) -> Result<PathBuf, String> {
// 3.0 (quilt) trees need the orig tarball(s) reachable from the working
// directory, and dpkg-source searches cwd — link them from the tree's
// parent, where pkh build / git ubuntu export-orig leave them.
link_orig_tarballs(root, tmp.path());
let output = Command::new("dpkg-source")
.env("LC_ALL", "C")
.args(["-b", "-Zgzip", "-z1"])
.arg(root)
.current_dir(tmp.path())
.output()
.map_err(|e| format!("Could not run dpkg-source: {e}"))?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
if stderr.to_lowercase().contains("uncommitted") {
return Err(
"The source tree has uncommitted changes that dpkg-source refuses \
to pack. Commit them first, or use --native to lint with pkh's \
native checks only."
.to_string(),
);
}
return Err(format!(
"dpkg-source -b failed, the tree may not be a valid source package:\n{}",
stderr.trim()
));
}
find_dsc(tmp.path())
}
/// The `pkh build` output matching the tree's current changelog entry, when
/// it exists and no tree content is newer than it: linting a stale artifact
/// would report the packaging of the past, so staleness forces a fresh pack.
fn usable_build_output(root: &Path, force_repack: bool) -> Option<PathBuf> {
if force_repack {
return None;
}
let dsc = expected_build_output(root)?;
let built = std::fs::metadata(&dsc).ok()?.modified().ok()?;
if tree_newer_than(root, built) {
return None;
}
Some(dsc)
}
/// The `pkh build` output path matching the tree's current changelog entry
/// (`../<source>_<version>.dsc`, pkh build's own naming), if it exists.
fn expected_build_output(root: &Path) -> Option<PathBuf> {
let entry =
crate::debian::changelog::parse_changelog_entry(&root.join("debian/changelog")).ok()?;
let dsc = root
.parent()?
.join(format!("{}_{}.dsc", entry.source, entry.version.no_epoch()));
std::fs::metadata(&dsc).ok()?;
Some(dsc)
}
/// Whether any tree content is newer than `built`. Skips `.git` and `.pc`:
/// commits and quilt bookkeeping churn their mtimes without touching what
/// the source package contains.
fn tree_newer_than(root: &Path, built: SystemTime) -> bool {
const SKIP: &[&str] = &[".git", ".pc"];
let mut stack = vec![root.to_path_buf()];
while let Some(dir) = stack.pop() {
let Ok(entries) = std::fs::read_dir(&dir) else {
continue;
};
for entry in entries.flatten() {
let name = entry.file_name();
if SKIP.iter().any(|skip| name.to_string_lossy() == *skip) {
continue;
}
match entry.file_type() {
Ok(ft) if ft.is_dir() => stack.push(entry.path()),
_ => {
let newer = entry
.metadata()
.ok()
.and_then(|m| m.modified().ok())
.is_some_and(|modified| modified > built);
if newer {
return true;
}
}
}
}
}
false
}
/// Symlink the orig (and orig component) tarballs for the tree's upstream
/// version from the tree's parent into `dest`, ignoring absence — native
/// trees have none, and missing tarballs surface as a dpkg-source error.
fn link_orig_tarballs(root: &Path, dest: &Path) {
let Some(parent) = root.parent() else {
return;
};
let Ok(entry) = crate::debian::changelog::parse_changelog_entry(&root.join("debian/changelog"))
else {
return;
};
let prefixes = [
format!("{}_{}.orig.tar.", entry.source, entry.version.upstream),
format!("{}_{}.orig-", entry.source, entry.version.upstream),
];
let Ok(entries) = std::fs::read_dir(parent) else {
return;
};
for candidate in entries.flatten() {
let name = candidate.file_name();
let name = name.to_string_lossy();
if !candidate.file_type().is_ok_and(|ft| ft.is_file())
|| !prefixes
.iter()
.any(|prefix| name.starts_with(prefix.as_str()))
{
continue;
}
let _ = std::os::unix::fs::symlink(parent.join(name.as_ref()), dest.join(name.as_ref()));
}
}
/// Parse lintian's output into findings. Tag lines carry the finding; `N:`
/// note lines following a tag line are its explanation (`--info` output) and
/// attach to it. Anything else is ignored.
fn parse(text: &str) -> Vec<Finding> {
let tag_line = Regex::new(r"^(?P<letter>[EWIPXOC]): (?P<rest>.*)$").expect("static regex");
let subject = Regex::new(
r"^(?P<pkg>\S+?)(?: (?P<ptype>source|binary|udeb|changes|buildinfo))?: (?P<tag>\S+)(?: (?P<details>.*))?$",
)
.expect("static regex");
let note_line = Regex::new(r"^N:(?: (?P<text>.*))?$").expect("static regex");
let mut findings: Vec<Finding> = Vec::new();
for line in text.lines() {
if let Some(note) = note_line.captures(line) {
// Attach to the finding above, like lintian lays out --info.
if let (Some(text), Some(last)) = (note.name("text"), findings.last_mut())
&& !text.as_str().trim().is_empty()
{
last.explanation.push(text.as_str().trim().to_string());
}
continue;
}
let Some(head) = tag_line.captures(line) else {
continue;
};
let Some(subject) = subject.captures(&head["rest"]) else {
continue;
};
findings.push(Finding {
letter: head["letter"].chars().next().unwrap_or('E'),
tag_name: subject["tag"].to_string(),
message: subject
.name("details")
.map_or(String::new(), |d| d.as_str().to_string()),
package: subject["pkg"].to_string(),
processable_type: subject.name("ptype").map(|p| p.as_str().to_string()),
explanation: Vec::new(),
origin: Origin::Lintian,
});
}
findings
}
/// The single `.dsc` the ephemeral source package produced.
fn find_dsc(dir: &Path) -> Result<PathBuf, String> {
let mut entries: Vec<PathBuf> = std::fs::read_dir(dir)
.map_err(|e| format!("Cannot read the temporary build directory: {e}"))?
.flatten()
.map(|entry| entry.path())
.filter(|path| path.extension().is_some_and(|ext| ext == "dsc"))
.collect();
match entries.pop() {
Some(dsc) => Ok(dsc),
None => Err(
"dpkg-source produced no .dsc; the tree may not be a valid source package".to_string(),
),
}
}
/// Temporary directory removed on drop; the hand-rolled stand-in for
/// `tempfile`, which is dev-only in this crate.
struct TempDir(PathBuf);
impl TempDir {
fn new() -> Result<TempDir, String> {
let unique = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0);
let path = std::env::temp_dir().join(format!("pkh-lint-{}-{unique}", std::process::id()));
std::fs::create_dir(&path)
.map_err(|e| format!("Could not create a temporary directory: {e}"))?;
Ok(TempDir(path))
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Captured from lintian 2.129.0ubuntu2.1 on a broken native source
/// package (default display level).
const SOURCE_OUTPUT: &str = "\
E: hello source: malformed-debian-changelog-version 0.1-1 (for native) [debian/changelog:1]
E: hello source: package-uses-debhelper-but-lacks-build-depends [debian/rules]
W: hello source: debhelper-but-no-misc-depends hello
W: hello source: debhelper-compat-file-is-missing
W: hello source: no-debian-copyright-in-source
";
/// Captured from lintian 2.129.0ubuntu2.1 on a binary package: binary
/// findings carry no processable type after the package name.
const BINARY_OUTPUT: &str = "\
E: badpkg: description-too-short test
E: badpkg: extended-description-is-empty
W: badpkg: empty-binary-package
W: badpkg: recommended-field badpkg_1.0-1_all.deb Priority
";
/// Captured with --show-overrides: overridden findings print as `O:`.
const OVERRIDDEN_OUTPUT: &str = "\
E: hello source: malformed-debian-changelog-version 0.1-1 (for native) [debian/changelog:1]
O: hello source: debhelper-compat-file-is-missing
";
/// Captured with --info: each tag line is followed by `N:` explanation
/// lines that belong to it.
const INFO_OUTPUT: &str = "\
N:
E: hello source: malformed-debian-changelog-version 0.1-1 (for native) [debian/changelog:1]
N:
N: The version string in the latest changelog entry was not parsed correctly.
N: Usually, that means it does not conform to policy.
N:
N:
E: hello source: package-uses-debhelper-but-lacks-build-depends [debian/rules]
N:
N: If a package uses debhelper, it must declare a Build-Depends on debhelper
N: or on the debhelper-compat virtual package. For example:
N:
";
#[test]
fn parses_source_output_with_type() {
let findings = parse(SOURCE_OUTPUT);
assert_eq!(findings.len(), 5);
let first = &findings[0];
assert_eq!(first.letter, 'E');
assert_eq!(first.package, "hello");
assert_eq!(first.processable_type.as_deref(), Some("source"));
assert_eq!(first.tag_name, "malformed-debian-changelog-version");
assert_eq!(first.message, "0.1-1 (for native) [debian/changelog:1]");
assert_eq!(findings[2].letter, 'W');
}
#[test]
fn parses_binary_output_without_type() {
let findings = parse(BINARY_OUTPUT);
assert_eq!(findings.len(), 4);
let first = &findings[0];
assert_eq!(first.package, "badpkg");
assert_eq!(first.processable_type, None);
assert_eq!(first.tag_name, "description-too-short");
// Details containing a file name with dots survive intact.
assert_eq!(findings[3].message, "badpkg_1.0-1_all.deb Priority");
}
#[test]
fn parses_overridden_lines() {
let findings = parse(OVERRIDDEN_OUTPUT);
assert_eq!(findings.len(), 2);
assert_eq!(findings[1].letter, 'O');
assert_eq!(findings[1].tag_name, "debhelper-compat-file-is-missing");
assert_eq!(findings[1].message, "");
}
#[test]
fn attaches_info_notes_to_the_preceding_finding() {
let findings = parse(INFO_OUTPUT);
assert_eq!(findings.len(), 2);
assert_eq!(
findings[0].explanation,
vec![
"The version string in the latest changelog entry was not parsed correctly.",
"Usually, that means it does not conform to policy.",
]
);
assert_eq!(findings[1].explanation.len(), 2);
assert!(findings[1].explanation[0].starts_with("If a package uses debhelper"));
}
#[test]
fn ignores_stray_lines() {
let findings =
parse("N: lintian ran\ngarbage line\nC: hello source: some-classification\n");
// The C: classification line parses (kept for JSON), garbage drops.
assert_eq!(findings.len(), 1);
assert_eq!(findings[0].letter, 'C');
}
/// A lintable tree at `<outer>/<name>` with a changelog entry
/// `pkg (1.0-1) resolute`; returns the outer dir (the tree's parent,
/// where pkh build would place artifacts).
fn tree() -> (tempfile::TempDir, std::path::PathBuf) {
let outer = tempfile::tempdir().unwrap();
let root = outer.path().join("pkg-1.0");
std::fs::create_dir_all(root.join("debian")).unwrap();
std::fs::write(
root.join("debian/changelog"),
"pkg (1.0-1) resolute; urgency=medium\n\n * x\n\n -- J <j@e.org> Sat, 19 Sep 2026 12:00:00 +0000\n",
)
.unwrap();
(outer, root)
}
fn dsc_of(outer: &tempfile::TempDir) -> std::path::PathBuf {
outer.path().join("pkg_1.0-1.dsc")
}
#[test]
fn current_build_output_is_reused() {
let (outer, root) = tree();
std::fs::write(dsc_of(&outer), "dummy dsc").unwrap();
// The dsc was written after every tree file: current.
assert_eq!(usable_build_output(&root, false), Some(dsc_of(&outer)));
// Forcing repack skips it.
assert_eq!(usable_build_output(&root, true), None);
}
#[test]
fn stale_or_mismatched_build_output_is_rejected() {
let (outer, root) = tree();
// A tree file written after the dsc makes the artifact stale. The
// sleep crosses the coarse clock tick mtimes are stamped with, so
// the control file is strictly newer than the dsc.
std::fs::write(dsc_of(&outer), "dummy dsc").unwrap();
std::thread::sleep(std::time::Duration::from_millis(10));
std::fs::write(root.join("debian/control"), "Source: pkg\n").unwrap();
assert_eq!(usable_build_output(&root, false), None);
// A dsc of a different version does not represent this tree.
let old_dsc = outer.path().join("pkg_0.9-1.dsc");
std::fs::write(&old_dsc, "dummy dsc").unwrap();
assert_eq!(usable_build_output(&root, false), None);
}
#[test]
fn missing_build_output_is_rejected() {
let (outer, root) = tree();
assert_eq!(usable_build_output(&root, false), None);
assert!(!dsc_of(&outer).exists());
}
#[test]
fn orig_tarballs_are_linked_for_packing() {
let (outer, root) = tree();
std::fs::write(outer.path().join("pkg_1.0.orig.tar.xz"), "orig").unwrap();
std::fs::write(outer.path().join("pkg_1.0.orig-data.tar.gz"), "comp").unwrap();
std::fs::write(outer.path().join("unrelated_1.0.orig.tar.xz"), "no").unwrap();
std::fs::write(outer.path().join("pkg_1.0-1.dsc"), "no").unwrap();
let dest = tempfile::tempdir().unwrap();
link_orig_tarballs(&root, dest.path());
assert!(
dest.path()
.join("pkg_1.0.orig.tar.xz")
.symlink_metadata()
.is_ok()
);
assert!(
dest.path()
.join("pkg_1.0.orig-data.tar.gz")
.symlink_metadata()
.is_ok()
);
assert!(!dest.path().join("unrelated_1.0.orig.tar.xz").exists());
// The dsc is not an orig tarball and must not be linked.
assert!(!dest.path().join("pkg_1.0-1.dsc").exists());
}
}
View File
+501 -231
View File
@@ -3,12 +3,9 @@ use std::io::Write;
extern crate clap; extern crate clap;
use clap::{Command, arg, command}; use clap::{Command, arg, command};
use pkh::context::ContextConfig;
extern crate flate2; extern crate flate2;
use pkh::changelog::generate_entry;
use indicatif_log_bridge::LogWrapper; use indicatif_log_bridge::LogWrapper;
use log::{error, info}; use log::{error, info};
@@ -23,6 +20,155 @@ fn current_dir_or_exit() -> std::path::PathBuf {
} }
} }
/// CLI-side Ctrl+C wiring. The passive state (interrupted flag, cleanup
/// hook registry, reporter slot) lives in `pkh::interrupt`; everything that
/// installs, prints or exits lives here: the SIGINT handler only wakes a
/// watchdog through a self-pipe (async-signal-safe), and the watchdog runs
/// the whole shutdown in thread context — the live view's reporter clears
/// the terminal, the notice is printed, further Ctrl+C is absorbed as a
/// no-op, the cleanup hooks release their resources, and the process exits
/// with the conventional status 130, skipping destructors.
mod interrupt {
use std::sync::atomic::{AtomicBool, AtomicI32, Ordering};
/// Whether the interrupt notice has been shown; the first caller prints
/// it, later ones stay silent
static NOTICE_SHOWN: AtomicBool = AtomicBool::new(false);
/// Whether handler, self-pipe and watchdog are in place
static INSTALLED: AtomicBool = AtomicBool::new(false);
/// Write end of the self-pipe the signal handler wakes the watchdog
/// through; `-1` until [`install`] set it up
static SELF_PIPE_WRITE: AtomicI32 = AtomicI32::new(-1);
/// Install the process-global Ctrl+C (SIGINT) handler; idempotent.
///
/// When the self-pipe or the watchdog cannot be set up, the default
/// SIGINT disposition is kept (the process dies immediately) rather
/// than installing a handler that could not run the shutdown.
pub fn install() {
if INSTALLED.swap(true, Ordering::SeqCst) {
return;
}
let mut fds = [0 as libc::c_int; 2];
// SAFETY: pipe(2) into a two-element array we own
if unsafe { libc::pipe(fds.as_mut_ptr()) } != 0 {
INSTALLED.store(false, Ordering::SeqCst);
return;
}
let (read_fd, write_fd) = (fds[0], fds[1]);
// The write end is used from the signal handler: non-blocking, so
// even a full pipe degrades to a dropped wake-up instead of
// blocking the handler.
// SAFETY: fcntl(2) on a pipe file descriptor we just created
unsafe {
let flags = libc::fcntl(write_fd, libc::F_GETFL);
libc::fcntl(write_fd, libc::F_SETFL, flags | libc::O_NONBLOCK);
}
SELF_PIPE_WRITE.store(write_fd, Ordering::SeqCst);
let spawned = std::thread::Builder::new()
.name("pkh-interrupt".to_string())
.spawn(move || watchdog(read_fd));
if spawned.is_err() {
// SAFETY: closing pipe file descriptors we just created
unsafe {
libc::close(read_fd);
libc::close(write_fd);
}
SELF_PIPE_WRITE.store(-1, Ordering::SeqCst);
INSTALLED.store(false, Ordering::SeqCst);
return;
}
// SAFETY: installing a signal handler whose body only records the
// interruption and writes to the self-pipe (async-signal-safe)
unsafe {
libc::signal(libc::SIGINT, on_sigint as *const () as usize);
}
}
/// Never returns: park the calling thread until the watchdog exits the
/// process
///
/// The watchdog owns the interrupt shutdown; a caller that would
/// otherwise reach its own `std::process::exit` and kill the process
/// mid-cleanup must park here instead.
pub fn wait_for_shutdown() -> ! {
loop {
std::thread::sleep(std::time::Duration::from_millis(100));
}
}
/// Signal handler body: record the interruption and wake the watchdog
/// through the self-pipe
extern "C" fn on_sigint(_sig: libc::c_int) {
pkh::interrupt::mark_interrupted();
let fd = SELF_PIPE_WRITE.load(Ordering::SeqCst);
if fd >= 0 {
// SAFETY: write(2) of one byte to the self-pipe is
// async-signal-safe; a failed write (e.g. EAGAIN) drops the
// wake-up instead of blocking the handler
unsafe {
libc::write(fd, b"x".as_ptr().cast(), 1);
}
}
}
/// Watchdog body: block until the signal handler's byte arrives, then
/// run the shutdown sequence
fn watchdog(read_fd: libc::c_int) {
let mut byte = [0u8; 1];
// SAFETY: read(2) into a local buffer of the announced length
let received = unsafe { libc::read(read_fd, byte.as_mut_ptr().cast(), 1) };
// The write end is never closed, so a short read cannot happen in
// practice; on error there is nothing to clean up either way.
if received > 0 {
run_interrupt_sequence();
}
}
/// Reporter, notice, cleanup hooks, exit 130: the whole shutdown, run
/// in the watchdog thread immediately on Ctrl+C — never in the signal
/// handler itself
///
/// Further Ctrl+C while this runs writes bytes nobody reads: absorbed
/// as a no-op (send SIGTERM/SIGKILL if a hook ever hangs).
fn run_interrupt_sequence() {
let hint =
pkh::interrupt::take_reporter().and_then(|report| {
match std::panic::catch_unwind(std::panic::AssertUnwindSafe(report)) {
Ok(hint) => hint,
Err(_) => {
log::error!("Interrupt reporter panicked");
None
}
}
});
show_notice();
if let Some(hint) = hint {
eprintln!("{hint}");
}
pkh::interrupt::run_cleanup_hooks();
// SAFETY: raw exit bypassing destructors, intended at interrupt time
unsafe {
libc::_exit(130);
}
}
/// Print the interrupt notice, once per process: the first caller
/// prints it, later ones stay silent
fn show_notice() {
if NOTICE_SHOWN.swap(true, Ordering::SeqCst) {
return;
}
eprintln!("CTRL+C: Build interrupted by user.");
}
}
fn main() { fn main() {
let rt = tokio::runtime::Runtime::new().unwrap(); let rt = tokio::runtime::Runtime::new().unwrap();
let logger = let logger =
@@ -34,7 +180,6 @@ fn main() {
LogWrapper::new(multi.clone(), logger).try_init().unwrap(); LogWrapper::new(multi.clone(), logger).try_init().unwrap();
let matches = command!() let matches = command!()
.subcommand_required(true) .subcommand_required(true)
.disable_version_flag(true)
.subcommand( .subcommand(
Command::new("new") Command::new("new")
.about("Scaffold a new Debian source package (buildable right away)") .about("Scaffold a new Debian source package (buildable right away)")
@@ -178,8 +323,12 @@ fn main() {
Command::new("chlog") Command::new("chlog")
.about("Auto-generate changelog entry, editing it, committing it afterwards") .about("Auto-generate changelog entry, editing it, committing it afterwards")
.arg(arg!(-s --series <series> "Target distribution series").required(false)) .arg(arg!(-s --series <series> "Target distribution series").required(false))
.arg(arg!(--backport "This changelog is for a backport entry").required(false)) .arg(arg!(--backport "Number the entry as a backport of the target series (Debian: 1.0-1 becomes 1.0-1~bpo12+1; Ubuntu: 3.1-1ubuntu2 becomes 3.1-1ubuntu2~24.04.1)").required(false)
.arg(arg!(-v --version <version> "Target version").required(false)), .conflicts_with_all(["nmu", "rebuild"]))
.arg(arg!(--nmu "Number the entry as a non-maintainer upload (1.0-1 becomes 1.0-1.1, native 1.0 becomes 1.0+nmu1)").required(false)
.conflicts_with("rebuild"))
.arg(arg!(--rebuild "Number the entry as a no-change rebuild (1.0-1 becomes 1.0-1build1)").required(false))
.arg(arg!(-v --version <version> "Target version (overrides the --backport/--nmu/--rebuild numbering)").required(false)),
) )
.subcommand( .subcommand(
Command::new("build") Command::new("build")
@@ -216,31 +365,61 @@ fn main() {
.long_help("Show raw tool output instead of the live build view.\nAlso implied by RUST_LOG=debug for pkh's own logs.")), .long_help("Show raw tool output instead of the live build view.\nAlso implied by RUST_LOG=debug for pkh's own logs.")),
) )
.subcommand( .subcommand(
Command::new("context") Command::new("lint")
.about("Manage contexts") .about("Lint the package (lintian wrapper + pkh-native checks)")
.subcommand_required(true) .arg(arg!([path] "Source tree to lint (default: the current directory)").required(false))
.subcommand( .arg(arg!(-d --dist <dist> "Target distribution (debian, ubuntu)").required(false))
Command::new("create") .arg(arg!(-s --series <series> "Target distribution series").required(false))
.about("Create a new context") .arg(arg!(--native "Run pkh-native checks only, without the lintian wrapper").required(false))
.arg(arg!(<name> "Context name")) .arg(arg!(--info "Show tag explanations under each finding").required(false))
.arg(arg!(--type <type> "Context type: ssh (only type supported for now)")) .arg(
.arg(arg!(--endpoint <endpoint> "Context endpoint (for example: ssh://user@host:port)")) clap::Arg::new("display_info")
.long("display-info")
.action(clap::ArgAction::SetTrue)
.help("Also display info-level tags (I:)"),
) )
.subcommand( .arg(arg!(--pedantic "Also display pedantic tags (P:)").required(false))
Command::new("rm") .arg(arg!(--experimental "Also display experimental tags (X:)").required(false))
.about("Remove a context") .arg(
.arg(arg!(<name> "Context name")) clap::Arg::new("show_overrides")
.long("show-overrides")
.action(clap::ArgAction::SetTrue)
.help("Also display overridden tags (O:)"),
) )
.subcommand( .arg(
Command::new("ls") clap::Arg::new("fail_on")
.about("List contexts") .long("fail-on")
.value_name("LEVELS")
.help("Comma-separated severities failing the run: error, warning, info, pedantic, experimental, override (default: error)"),
) )
.subcommand(Command::new("show").about("Show current context")) .arg(
.subcommand( clap::Arg::new("suppress_tags")
Command::new("use") .long("suppress-tags")
.about("Set current context") .value_name("LIST")
.arg(arg!(<name> "Context name")) .help("Comma-separated tag names to ignore for this run"),
) )
.arg(
clap::Arg::new("check")
.long("check")
.value_name("NAME")
.action(clap::ArgAction::Append)
.help("Run only this pkh-native check (can be specified multiple times)"),
)
.arg(arg!(--repack "Ignore existing pkh build output and pack the tree fresh for linting").required(false))
.arg(arg!(--json "Emit the report as JSON").required(false))
.arg(
clap::Arg::new("color")
.long("color")
.value_name("WHEN")
.value_parser(["auto", "always", "never"])
.help("Colorize the report: auto, always or never (default: auto)"),
)
.arg(
clap::Arg::new("list_tags")
.long("list-tags")
.action(clap::ArgAction::SetTrue)
.help("Print the pkh-native tag catalog and exit"),
),
) )
.subcommand( .subcommand(
Command::new("prune") Command::new("prune")
@@ -321,10 +500,14 @@ fn main() {
// the structural self-checks inside `scaffold` always run), with // the structural self-checks inside `scaffold` always run), with
// the scaffold outcome (e.g. a failed vendoring) shaping the // the scaffold outcome (e.g. a failed vendoring) shaping the
// offer. // offer.
let prompter = pkh::ui::prompt::TerminalPrompter;
if let Err(e) = rt.block_on(async { if let Err(e) = rt.block_on(async {
let opts = pkh::new::questions::run(cli).await?; let opts = pkh::new::questions::run(cli, &prompter).await?;
let outcome = pkh::new::scaffold(opts.clone(), &multi)?; let outcome = pkh::new::scaffold(opts.clone(), &multi)?;
pkh::new::questions::offer_verification(&opts, &outcome, &multi, no_verify).await; pkh::new::questions::offer_verification(
&opts, &outcome, &multi, no_verify, &prompter,
)
.await;
Ok::<(), Box<dyn std::error::Error>>(()) Ok::<(), Box<dyn std::error::Error>>(())
}) { }) {
error!("{}", e); error!("{}", e);
@@ -349,15 +532,14 @@ fn main() {
let (pb, progress_callback) = pkh::ui::create_progress_bar(&multi); let (pb, progress_callback) = pkh::ui::create_progress_bar(&multi);
// Convert PPA to base URL if provided // Convert PPA to base URL if provided
let base_url = ppa.map(|ppa_str| { let base_url = match ppa.map(pkh::package_info::split_ppa) {
// PPA format: user/ppa_name Some(Ok((user, name))) => Some(pkh::package_info::ppa_to_base_url(user, name)),
let parts: Vec<&str> = ppa_str.split('/').collect(); Some(Err(e)) => {
if parts.len() != 2 || parts[0].is_empty() || parts[1].is_empty() { error!("{e}");
error!("Invalid PPA format: '{}'. Expected: user/ppa_name", ppa_str);
std::process::exit(1); std::process::exit(1);
} }
pkh::package_info::ppa_to_base_url(parts[0], parts[1]) None => None,
}); };
// Since pull is async, we need to block on it // Since pull is async, we need to block on it
if let Err(e) = rt.block_on(async { if let Err(e) = rt.block_on(async {
@@ -386,79 +568,74 @@ fn main() {
let cwd = current_dir_or_exit(); let cwd = current_dir_or_exit();
let version = sub_matches.get_one::<String>("version").map(|s| s.as_str()); let version = sub_matches.get_one::<String>("version").map(|s| s.as_str());
let cli_series = sub_matches.get_one::<String>("series").map(|s| s.as_str()); let cli_series = sub_matches.get_one::<String>("series").map(|s| s.as_str());
let kind = if sub_matches
.get_one::<bool>("backport")
.copied()
.unwrap_or(false)
{
pkh::changelog::EntryKind::Backport
} else if sub_matches.get_one::<bool>("nmu").copied().unwrap_or(false) {
pkh::changelog::EntryKind::Nmu
} else if sub_matches
.get_one::<bool>("rebuild")
.copied()
.unwrap_or(false)
{
pkh::changelog::EntryKind::Rebuild
} else {
pkh::changelog::EntryKind::Normal
};
// Determine target series: CLI flag > interactive selector > current changelog series // Determine target series: CLI flag > interactive selector > current changelog series
let target_series = if let Some(s) = cli_series { let target_series = if let Some(s) = cli_series {
Some(s.to_string()) Some(s.to_string())
} else { } else {
// Parse current changelog to determine the default series
let changelog_path = cwd.join("debian/changelog"); let changelog_path = cwd.join("debian/changelog");
match pkh::changelog::parse_changelog_header(&changelog_path) { match rt.block_on(pkh::changelog::series_candidates(&changelog_path)) {
Ok((_pkg, _ver, current_series)) => { Some(pkh::changelog::SeriesCandidates::Choose {
// UNRELEASED is not a real series: offer it as a options,
// pinned first entry (selecting it keeps the changelog values,
// unreleased) on top of the current vendor's series default,
// list, defaulting to the development series. Any fallback,
// other series resolves through the series list of }) => match pkh::ui::select_series(&options, &default) {
// its own distribution. Ok(selected) => {
match rt.block_on(async { Some(pkh::changelog::selected_series(&options, &values, selected))
if pkh::distro_info::is_unreleased(&current_series) {
// Vendors keep original casing ("Ubuntu"),
// while the series data keys are lowercase
let dist = pkh::build::env::current_vendor().to_lowercase();
let mut series_list =
vec![pkh::distro_info::UNRELEASED.to_string()];
series_list.extend(
pkh::distro_info::get_ordered_series_name(&dist).await?,
);
Ok(series_list)
} else {
let dist =
pkh::distro_info::get_dist_from_series(&current_series).await?;
pkh::distro_info::get_ordered_series_name(&dist).await
}
}) {
Ok(series_list) => {
// Default to the development series (the
// first real entry) when the changelog is
// UNRELEASED, not to the pinned entry itself
let default = if pkh::distro_info::is_unreleased(&current_series)
&& series_list.len() > 1
{
series_list[1].clone()
} else {
current_series.clone()
};
match pkh::ui::select_series(&series_list, &default) {
Ok(selected) => Some(selected),
Err(e) => {
error!(
"Series selection failed: {}. Using current series '{}' instead.",
e, current_series
);
Some(current_series)
}
}
}
Err(_) => {
// Could not fetch series list, use current series as default
Some(current_series)
}
} }
} Err(e) => {
Err(_) => None, error!(
"Series selection failed: {}. Using current series '{}' instead.",
e, fallback
);
Some(fallback)
}
},
// Could not fetch the series list: use the current series
Some(pkh::changelog::SeriesCandidates::Keep(current)) => Some(current),
// No parsable changelog: leave the series decision to
// generate_entry
None => None,
} }
}; };
if let Err(e) = generate_entry( let entry = match rt.block_on(pkh::changelog::generate_entry(
"debian/changelog", "debian/changelog",
Some(&cwd), Some(&cwd),
version, version,
target_series.as_deref(), target_series.as_deref(),
) { kind,
error!("{}", e); )) {
std::process::exit(1); Ok(entry) => entry,
} Err(e) => {
error!("{}", e);
std::process::exit(1);
}
};
println!(
"Found package: {}, version: {}",
entry.package, entry.previous_version
);
println!("New version: {}", entry.new_version);
println!("Added new changelog entry to {}", entry.path.display());
let editor = match std::env::var("EDITOR") { let editor = match std::env::var("EDITOR") {
Ok(e) => e, Ok(e) => e,
@@ -485,18 +662,25 @@ fn main() {
} }
Some(("build", sub_matches)) => { Some(("build", sub_matches)) => {
let cwd = current_dir_or_exit(); let cwd = current_dir_or_exit();
interrupt::install();
let verbose = sub_matches let verbose = sub_matches
.get_one::<bool>("verbose") .get_one::<bool>("verbose")
.copied() .copied()
.unwrap_or(false); .unwrap_or(false);
// Live build view: disabled by --verbose or when stdout is not a // Live build view, unless --verbose (DebUi additionally disables
// terminal (DebUi handles the non-TTY case itself) // itself when stdout is not a terminal)
let ui = if verbose { let quiet = pkh::report::Quiet;
let live = if verbose {
None None
} else { } else {
Some(std::sync::Arc::new(pkh::ui::deb::DebUi::new(&multi))) Some(pkh::ui::deb::DebUi::new(&multi))
}; };
let view: &dyn pkh::report::BuildView = live
.as_ref()
.map(|v| v as &dyn pkh::report::BuildView)
.unwrap_or(&quiet);
let prompter = pkh::ui::prompt::TerminalPrompter;
let orig_source = match sub_matches.get_one::<String>("orig").map(String::as_str) { let orig_source = match sub_matches.get_one::<String>("orig").map(String::as_str) {
Some("always") => pkh::build::OrigSourceMode::Always, Some("always") => pkh::build::OrigSourceMode::Always,
@@ -504,27 +688,58 @@ fn main() {
_ => pkh::build::OrigSourceMode::Auto, _ => pkh::build::OrigSourceMode::Auto,
}; };
if let Err(e) = pkh::build::build_source_package( match pkh::build::build_source_package(pkh::build::BuildSourceOptions {
Some(&cwd), source: Some(cwd),
pkh::build::SourceBuildOptions { options: pkh::build::SourceBuildOptions {
orig_source, orig_source,
..Default::default() ..Default::default()
}, },
ui, view,
) { prompter: &prompter,
error!("{}", e); }) {
// Unmet build dependencies/conflicts exit with status 3, Ok(output) => {
// like dpkg-buildpackage does. // The live view lists the artifacts itself when it
if e.downcast_ref::<pkh::debian::deps::UnmetBuildDependencies>() // renders; otherwise (verbose mode or non-TTY stdout)
.is_some() // print them as plain lines.
{ if !view.is_enabled() {
std::process::exit(3); for artifact in output.artifacts() {
println!(" {}", pkh::report::display_path(&artifact));
}
}
if output.signed {
println!("Package built and signed successfully!");
} else {
println!("Package built successfully (unsigned).");
}
}
Err(e) => {
// On Ctrl+C the interrupt watchdog owns the shutdown
// (see `pkh deb`): park here instead of racing it
if pkh::interrupt::interrupted() {
interrupt::wait_for_shutdown();
}
// The unmet-dependency diagnostics first, then the
// summary: the exact rendering the flow used to do.
if let Some(unmet) =
e.downcast_ref::<pkh::debian::deps::UnmetBuildDependencies>()
{
eprintln!("{}", unmet.0.message());
}
error!("{}", e);
// Unmet build dependencies/conflicts exit with status 3,
// like dpkg-buildpackage does.
if e.downcast_ref::<pkh::debian::deps::UnmetBuildDependencies>()
.is_some()
{
std::process::exit(3);
}
std::process::exit(1);
} }
std::process::exit(1);
} }
} }
Some(("put", sub_matches)) => { Some(("put", sub_matches)) => {
let cwd = current_dir_or_exit(); let cwd = current_dir_or_exit();
interrupt::install();
let ppa = sub_matches.get_one::<String>("ppa").map(|s| s.as_str()); let ppa = sub_matches.get_one::<String>("ppa").map(|s| s.as_str());
let changes = sub_matches let changes = sub_matches
.get_one::<String>("changes") .get_one::<String>("changes")
@@ -543,41 +758,48 @@ fn main() {
std::process::exit(1); std::process::exit(1);
}; };
let view = pkh::ui::deb::DebUi::new(&multi);
let prompter = pkh::ui::prompt::TerminalPrompter;
let options = pkh::put::PutOptions { let options = pkh::put::PutOptions {
ppa: ppa.to_string(), ppa: ppa.to_string(),
changes, changes,
force, force,
cwd, cwd,
view: &view,
prompter: &prompter,
}; };
if let Err(e) = rt.block_on(async { pkh::put::put(&options, &multi).await }) { if let Err(e) = rt.block_on(async { pkh::put::put(&options).await }) {
// On Ctrl+C the interrupt watchdog owns the shutdown (see
// `pkh deb`): park here instead of racing it
if pkh::interrupt::interrupted() {
interrupt::wait_for_shutdown();
}
error!("{}", e); error!("{}", e);
std::process::exit(1); std::process::exit(1);
} }
} }
Some(("deb", sub_matches)) => { Some(("deb", sub_matches)) => {
let cwd = current_dir_or_exit(); let cwd = current_dir_or_exit();
let series = sub_matches.get_one::<String>("series").map(|s| s.as_str()); // Ctrl+C during the build must say what happened and release the
let pocket = sub_matches.get_one::<String>("pocket").map(|s| s.as_str()); // ephemeral chroot instead of dying on the default disposition.
let arch = sub_matches.get_one::<String>("arch").map(|s| s.as_str()); // The live view (when enabled) registers its own reporter on top
let cross = sub_matches.get_one::<bool>("cross").unwrap_or(&false); // of this to clear the widget first.
let ppa: Vec<&str> = sub_matches interrupt::install();
let series = sub_matches.get_one::<String>("series").cloned();
let pocket = sub_matches.get_one::<String>("pocket").cloned();
let arch = sub_matches.get_one::<String>("arch").cloned();
let cross = sub_matches
.get_one::<bool>("cross")
.copied()
.unwrap_or(false);
let ppa: Vec<String> = sub_matches
.get_many::<String>("ppa") .get_many::<String>("ppa")
.map(|v| v.map(|s| s.as_str()).collect()) .map(|v| v.cloned().collect())
.unwrap_or_default(); .unwrap_or_default();
let ppa = if ppa.is_empty() { let inject: Vec<String> = sub_matches
None
} else {
Some(ppa.as_slice())
};
let inject_packages: Vec<&str> = sub_matches
.get_many::<String>("inject") .get_many::<String>("inject")
.map(|v| v.map(|s| s.as_str()).collect()) .map(|v| v.cloned().collect())
.unwrap_or_default(); .unwrap_or_default();
let inject_packages = if inject_packages.is_empty() {
None
} else {
Some(inject_packages.as_slice())
};
let mode: Option<&str> = sub_matches.get_one::<String>("mode").map(|s| s.as_str()); let mode: Option<&str> = sub_matches.get_one::<String>("mode").map(|s| s.as_str());
let mode: Option<pkh::deb::BuildMode> = match mode { let mode: Option<pkh::deb::BuildMode> = match mode {
Some("local") => Some(pkh::deb::BuildMode::Local), Some("local") => Some(pkh::deb::BuildMode::Local),
@@ -596,127 +818,51 @@ fn main() {
}) })
}); });
// Live build view: disabled by --verbose or when stdout is not a // Live build view, unless --verbose (DebUi additionally disables
// terminal (DebUi handles the non-TTY case itself) // itself when stdout is not a terminal)
let ui = if verbose { let quiet = pkh::report::Quiet;
let live = if verbose {
None None
} else { } else {
Some(std::sync::Arc::new(pkh::ui::deb::DebUi::new(&multi))) Some(pkh::ui::deb::DebUi::new(&multi))
}; };
let view: &dyn pkh::report::BuildView = live
.as_ref()
.map(|v| v as &dyn pkh::report::BuildView)
.unwrap_or(&quiet);
let result = rt.block_on(async { let result = rt.block_on(async {
pkh::deb::build_binary_package( pkh::deb::build_binary_package(pkh::deb::DebBuildOptions {
arch, arch,
series, series,
pocket, pocket,
Some(cwd.as_path()), cwd: Some(cwd.clone()),
*cross, cross,
mode, mode,
ppa, ppa,
inject_packages, inject,
None,
ui.clone(),
jobs, jobs,
) view,
..Default::default()
})
.await .await
}); });
match result { match result {
Ok(artifacts) => { Ok(_) => info!("Done."),
let _ = artifacts;
info!("Done.");
}
Err(e) => { Err(e) => {
// On Ctrl+C the interrupt watchdog owns the shutdown: it
// has already shown the notice, is releasing the build
// resources, and will exit with 130 — park here instead
// of racing it with another exit.
if pkh::interrupt::interrupted() {
interrupt::wait_for_shutdown();
}
error!("{}", e); error!("{}", e);
std::process::exit(1); std::process::exit(1);
} }
} }
} }
Some(("context", sub_matches)) => {
let mgr = pkh::context::manager();
match sub_matches.subcommand() {
Some(("create", args)) => {
let name = args.get_one::<String>("name").unwrap();
let type_str = args
.get_one::<String>("type")
.map(|s| s.as_str())
.unwrap_or("local");
let context = match type_str {
"local" => ContextConfig::Local,
"ssh" => {
let endpoint =
args.get_one::<String>("endpoint").unwrap_or_else(|| {
error!(
"An --endpoint is required to create an ssh context. \
Expected format: [ssh://][user@]host[:port]"
);
std::process::exit(1);
});
// Parse host, user, port from endpoint
// Formats: [ssh://][user@]host[:port]
let endpoint_re = regex::Regex::new(r"^(?:ssh://)?(?:(?P<user>[^@]+)@)?(?P<host>[^:/]+)(?::(?P<port>\d+))?$").unwrap();
let endpoint_cap = endpoint_re.captures(endpoint).unwrap_or_else(|| {
error!("Invalid endpoint format: '{}'. Expected [ssh://][user@]host[:port]", endpoint);
std::process::exit(1);
});
let host = endpoint_cap.name("host").unwrap().as_str().to_string();
let user = endpoint_cap.name("user").map(|m| m.as_str().to_string());
let port = endpoint_cap.name("port").map(|m| {
m.as_str().parse::<u16>().unwrap_or_else(|_| {
error!("Invalid port number");
std::process::exit(1);
})
});
ContextConfig::Ssh { host, user, port }
}
_ => {
error!("Unknown context type: {}", type_str);
std::process::exit(1);
}
};
if let Err(e) = mgr.add_context(name, context) {
error!("Failed to create context: {}", e);
std::process::exit(1);
}
info!("Context '{}' created.", name);
}
Some(("rm", args)) => {
let name = args.get_one::<String>("name").unwrap();
if let Err(e) = mgr.remove_context(name) {
error!("Failed to remove context: {}", e);
std::process::exit(1);
}
info!("Context '{}' removed.", name);
}
Some(("ls", _)) => {
let contexts = mgr.list_contexts();
let current = mgr.current_name();
for ctx in contexts {
if ctx == current {
println!("* {}", ctx);
} else {
println!(" {}", ctx);
}
}
}
Some(("show", _)) => {}
Some(("use", args)) => {
let name = args.get_one::<String>("name").unwrap();
if let Err(e) = mgr.set_current(name) {
error!("Failed to set context: {}", e);
std::process::exit(1);
}
info!("Switched to context '{}'.", name);
}
_ => unreachable!(),
}
}
Some(("prune", sub_matches)) => { Some(("prune", sub_matches)) => {
let dry_run = sub_matches let dry_run = sub_matches
.get_one::<bool>("dry_run") .get_one::<bool>("dry_run")
@@ -769,6 +915,130 @@ fn main() {
} }
} }
} }
Some(("lint", sub_matches)) => {
if sub_matches.get_flag("list_tags") {
print!("{}", pkh::lint::list_tags());
std::process::exit(0);
}
let path = sub_matches
.get_one::<String>("path")
.map(std::path::PathBuf::from)
.unwrap_or_else(current_dir_or_exit);
let fail_on = match pkh::lint::output::parse_fail_on(
sub_matches
.get_one::<String>("fail_on")
.map(String::as_str)
.unwrap_or("error"),
) {
Ok(levels) => levels,
Err(e) => {
error!("{}", e);
std::process::exit(2);
}
};
let options = pkh::lint::LintOptions {
path,
native: sub_matches.get_flag("native"),
fail_on,
info: sub_matches.get_flag("info"),
display_info: sub_matches.get_flag("display_info"),
pedantic: sub_matches.get_flag("pedantic"),
experimental: sub_matches.get_flag("experimental"),
show_overrides: sub_matches.get_flag("show_overrides"),
suppress_tags: sub_matches
.get_one::<String>("suppress_tags")
.map(|list| {
list.split(',')
.map(str::trim)
.filter(|tag| !tag.is_empty())
.map(str::to_string)
.collect()
})
.unwrap_or_default(),
only_checks: sub_matches
.get_many::<String>("check")
.map(|values| values.cloned().collect())
.unwrap_or_default(),
repack: sub_matches.get_flag("repack"),
json: sub_matches.get_flag("json"),
color: match sub_matches.get_one::<String>("color").map(String::as_str) {
Some("always") => pkh::lint::output::ColorMode::Always,
Some("never") => pkh::lint::output::ColorMode::Never,
_ => pkh::lint::output::ColorMode::Auto,
},
dist: sub_matches.get_one::<String>("dist").cloned(),
series: sub_matches.get_one::<String>("series").cloned(),
};
match pkh::lint::run(&options) {
Ok(report) => {
if options.json {
println!("{}", pkh::lint::output::render_json(&report, &options));
} else {
print!("{}", pkh::lint::output::render_text(&report, &options));
}
std::process::exit(pkh::lint::output::exit_code(&report, &options));
}
Err(e) => {
error!("{}", e);
std::process::exit(2);
}
}
}
_ => unreachable!("Exhausted list of subcommands and subcommand_required prevents `None`"), _ => unreachable!("Exhausted list of subcommands and subcommand_required prevents `None`"),
} }
} }
#[cfg(test)]
mod tests {
use super::interrupt;
/// End-to-end check of the whole sequence: installed handler → self-pipe
/// → watchdog → notice + hooks → exit status 130. The sequence ends in
/// `libc::_exit`, so it cannot be exercised in-process: this test
/// re-spawns the test binary in child mode (env var), where the test
/// installs the handler and raises SIGINT at itself.
#[test]
fn sigint_sequence_prints_the_notice_and_exits_130() {
const CHILD_ENV: &str = "PKH_SIGINT_TEST_CHILD";
if std::env::var(CHILD_ENV).is_ok() {
// Child mode: install, register a pending hook, then interrupt
// ourselves. If the sequence never runs, the sleep below turns
// the failure into a wrong (zero) exit code instead of a hang.
interrupt::install();
// Alive until the watchdog drains it: a dropped guard would
// deregister the hook and the drain would run empty
let _hook = pkh::interrupt::register_cleanup_hook(Box::new(|| ()));
// SAFETY: kill(2) to our own process with SIGINT
unsafe {
libc::kill(libc::getpid(), libc::SIGINT);
}
std::thread::sleep(std::time::Duration::from_secs(30));
std::process::exit(0);
}
let exe = std::env::current_exe().expect("locate the test executable");
let output = std::process::Command::new(exe)
// --nocapture: libtest's capture buffer would otherwise swallow
// the watchdog's notice (threads spawned during a test inherit
// the capture), and the process exits before the harness prints
// anything it captured
.args([
"--exact",
"tests::sigint_sequence_prints_the_notice_and_exits_130",
"--test-threads=1",
"--nocapture",
])
.env(CHILD_ENV, "1")
.output()
.expect("re-spawn the test binary");
let stderr = String::from_utf8_lossy(&output.stderr);
assert_eq!(output.status.code(), Some(130), "child stderr:\n{stderr}");
assert!(
stderr.contains("CTRL+C: Build interrupted by user."),
"child stderr:\n{stderr}"
);
}
}
+1 -1
View File
@@ -436,7 +436,7 @@ pub fn create_orig_tarball_excluding(
log::info!( log::info!(
"Created orig tarball {}", "Created orig tarball {}",
crate::ui::display_path(&tarball_path) crate::report::display_path(&tarball_path)
); );
Ok(tarball_path) Ok(tarball_path)
} }
+4 -4
View File
@@ -251,7 +251,7 @@ fn print_success(opts: &NewOptions, outcome: &ScaffoldOutcome) {
// `display_path` yields an empty string when the target is the cwd // `display_path` yields an empty string when the target is the cwd
// itself (Here mode): `Created .` would be cryptic, so spell the // itself (Here mode): `Created .` would be cryptic, so spell the
// location out; the skeleton/path modes keep the `<dir>` display. // location out; the skeleton/path modes keep the `<dir>` display.
let display = crate::ui::display_path(&target); let display = crate::report::display_path(&target);
let location = if display.is_empty() { let location = if display.is_empty() {
"package in the current directory".to_string() "package in the current directory".to_string()
} else { } else {
@@ -819,7 +819,7 @@ mod tests {
let output = crate::build::run_source_build( let output = crate::build::run_source_build(
&source, &source,
&crate::build::SourceBuildOptions::default(), &crate::build::SourceBuildOptions::default(),
None, &crate::report::Quiet,
) )
.unwrap(); .unwrap();
let dsc = std::fs::read_to_string(&output.dsc).unwrap(); let dsc = std::fs::read_to_string(&output.dsc).unwrap();
@@ -885,7 +885,7 @@ mod tests {
let output = crate::build::run_source_build( let output = crate::build::run_source_build(
&dir.path().join("mytool"), &dir.path().join("mytool"),
&crate::build::SourceBuildOptions::default(), &crate::build::SourceBuildOptions::default(),
None, &crate::report::Quiet,
) )
.unwrap(); .unwrap();
@@ -916,7 +916,7 @@ mod tests {
let output = crate::build::run_source_build( let output = crate::build::run_source_build(
&dir.path().join("mytool"), &dir.path().join("mytool"),
&crate::build::SourceBuildOptions::default(), &crate::build::SourceBuildOptions::default(),
None, &crate::report::Quiet,
) )
.unwrap(); .unwrap();
assert!(output.dsc.exists(), "{:?} missing", output.dsc); assert!(output.dsc.exists(), "{:?} missing", output.dsc);
+4 -4
View File
@@ -183,7 +183,7 @@ pub fn create_vendor_component(
log::info!( log::info!(
"Created vendored-dependencies component {}", "Created vendored-dependencies component {}",
crate::ui::display_path(&component_path) crate::report::display_path(&component_path)
); );
Ok(component_path) Ok(component_path)
} }
@@ -283,7 +283,7 @@ fn git_archive_tarball(
log::info!( log::info!(
"Created orig tarball from git archive of {tag}: {}", "Created orig tarball from git archive of {tag}: {}",
crate::ui::display_path(&dest) crate::report::display_path(&dest)
); );
Ok(dest) Ok(dest)
} }
@@ -311,7 +311,7 @@ fn download_release(
Ok(path) => { Ok(path) => {
log::info!( log::info!(
"Created orig tarball from the release download of {tag}: {}", "Created orig tarball from the release download of {tag}: {}",
crate::ui::display_path(&path) crate::report::display_path(&path)
); );
Ok(path) Ok(path)
} }
@@ -352,7 +352,7 @@ fn fetch_and_repack(
log::info!( log::info!(
"Created orig tarball from {}: {}", "Created orig tarball from {}: {}",
source, source,
crate::ui::display_path(&dest) crate::report::display_path(&dest)
); );
Ok(dest) Ok(dest)
} }
+91 -83
View File
@@ -1,11 +1,11 @@
//! The `pkh new` interactive wizard. //! The `pkh new` interactive wizard.
//! //!
//! [`run`] is the single entry point: on an interactive terminal it asks the //! [`run`] is the single entry point: when the prompter can interact it asks
//! questions of the spec's "Proposed UX" transcript, fills a //! the questions of the spec's "Proposed UX" transcript, fills a
//! [`NewCli`] with the answers (explicit flags are never re-asked), and //! [`NewCli`] with the answers (explicit flags are never re-asked), and
//! reuses [`options::resolve`] as the single source of truth for defaults, //! reuses [`options::resolve`] as the single source of truth for defaults,
//! detection and validation — so the non-interactive and interactive paths //! detection and validation — so the non-interactive and interactive paths
//! cannot drift apart. Without a terminal (or with `--defaults`) it goes //! cannot drift apart. Headless (or with `--defaults`) it goes
//! straight through [`options::resolve`], whose error lists every missing //! straight through [`options::resolve`], whose error lists every missing
//! answer. //! answer.
//! //!
@@ -13,11 +13,10 @@
//! verification builds of the spec ([`offer_verification`]); a failed //! verification builds of the spec ([`offer_verification`]); a failed
//! verification never undoes the scaffold. //! verification never undoes the scaffold.
//! //!
//! The prompt calls live in `run_wizard` and `offer_verification` only; //! The prompter calls live in `run_wizard` and `offer_verification` only;
//! everything else in this module is pure and unit-tested. //! everything else in this module is pure and unit-tested.
use std::error::Error; use std::error::Error;
use std::io::IsTerminal;
use std::path::PathBuf; use std::path::PathBuf;
use indicatif::MultiProgress; use indicatif::MultiProgress;
@@ -28,7 +27,7 @@ use crate::new::licenses;
use crate::new::options::{self, NewCli, NewOptions, SourceDir, SourceFormat, TemplateId}; use crate::new::options::{self, NewCli, NewOptions, SourceDir, SourceFormat, TemplateId};
use crate::new::origin::GitOrigin; use crate::new::origin::GitOrigin;
use crate::new::templates::{self, ProbeResult, ScaffoldOutcome}; use crate::new::templates::{self, ProbeResult, ScaffoldOutcome};
use crate::ui::prompt; use crate::report::{Prompter, Validator};
/// Answer of the "where is the source code?" question: fresh skeleton. /// Answer of the "where is the source code?" question: fresh skeleton.
const SOURCE_SKELETON: &str = "Create a new project skeleton here"; const SOURCE_SKELETON: &str = "Create a new project skeleton here";
@@ -40,8 +39,8 @@ const SOURCE_PATH: &str = "Package the sources in another directory…";
/// The "everything else" entry of the license menu. /// The "everything else" entry of the license menu.
const LICENSE_OTHER: &str = "Other (enter a SPDX identifier)"; const LICENSE_OTHER: &str = "Other (enter a SPDX identifier)";
/// Labels of the interactive `select` questions. `prompt::select` renders /// Labels of the interactive `select` questions. The prompter renders
/// `> <label><answer>` verbatim — unlike [`prompt::text`], it appends no /// `> <label><answer>` verbatim — unlike [`Prompter::text`], it appends no
/// formatting of its own — so each label carries its own separator: /// formatting of its own — so each label carries its own separator:
/// field-style prompts end with `": "`, question-style ones with `"? "`. /// field-style prompts end with `": "`, question-style ones with `"? "`.
const LANGUAGE_LABEL: &str = "Which language/build system is your program using? "; const LANGUAGE_LABEL: &str = "Which language/build system is your program using? ";
@@ -62,20 +61,13 @@ const SELECT_LABELS: [&str; 6] = [
ORIG_LABEL, ORIG_LABEL,
]; ];
/// Run the `pkh new` flow: the wizard on an interactive terminal, plain /// Run the `pkh new` flow: the wizard when the prompter can interact,
/// [`options::resolve`] otherwise (and with `--defaults`). /// plain [`options::resolve`] otherwise (and with `--defaults`).
pub async fn run(cli: NewCli) -> Result<NewOptions, Box<dyn Error>> { pub async fn run(cli: NewCli, prompter: &dyn Prompter) -> Result<NewOptions, Box<dyn Error>> {
if cli.defaults || !is_interactive() { if cli.defaults || !prompter.interactive() {
return Ok(options::resolve(cli).await?); return Ok(options::resolve(cli).await?);
} }
run_wizard(cli).await run_wizard(cli, prompter).await
}
/// Whether both ends of the terminal are interactive; the wizard and the
/// verification offers only run when this holds (the prompts' non-TTY
/// fallbacks would otherwise silently take defaults).
fn is_interactive() -> bool {
std::io::stdin().is_terminal() && std::io::stdout().is_terminal()
} }
/// The wizard question flow (spec "Proposed UX"), in order: /// The wizard question flow (spec "Proposed UX"), in order:
@@ -87,7 +79,10 @@ fn is_interactive() -> bool {
/// (`empty` template only), git init — then the summary screen and the /// (`empty` template only), git init — then the summary screen and the
/// final `Generate?` confirmation. Every question with an explicit flag /// final `Generate?` confirmation. Every question with an explicit flag
/// answer is skipped (flag > detected/probe > default merge order). /// answer is skipped (flag > detected/probe > default merge order).
async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> { async fn run_wizard(
mut cli: NewCli,
prompter: &dyn Prompter,
) -> Result<NewOptions, Box<dyn Error>> {
let cwd = std::env::current_dir()?; let cwd = std::env::current_dir()?;
let mut detect_dir = cli.source.clone().unwrap_or_else(|| cwd.clone()); let mut detect_dir = cli.source.clone().unwrap_or_else(|| cwd.clone());
let (detection, mut probe) = detect_and_probe(&detect_dir); let (detection, mut probe) = detect_and_probe(&detect_dir);
@@ -101,7 +96,12 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
// basename of the current directory. // basename of the current directory.
if cli.name.is_none() { if cli.name.is_none() {
let default = default_package_name(&cwd, probe.as_ref()); let default = default_package_name(&cwd, probe.as_ref());
let answer = ask_text("Package name", &default, options::validate_source_name)?; let answer = ask_text(
prompter,
"Package name",
&default,
options::validate_source_name,
)?;
cli.name = Some(answer); cli.name = Some(answer);
} }
@@ -142,7 +142,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
.unwrap_or(TemplateId::EMPTY) .unwrap_or(TemplateId::EMPTY)
.display_name() .display_name()
.to_string(); .to_string();
let id = select_template(&menu, &default)?; let id = select_template(prompter, &menu, &default)?;
cli.lang = Some(id.as_str().to_string()); cli.lang = Some(id.as_str().to_string());
} }
LanguageChoice::Ambiguous(candidates) => { LanguageChoice::Ambiguous(candidates) => {
@@ -157,7 +157,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
.join(", ") .join(", ")
); );
let menu = language_menu(&candidates); let menu = language_menu(&candidates);
let id = select_template(&menu, &menu[0])?; let id = select_template(prompter, &menu, &menu[0])?;
cli.lang = Some(id.as_str().to_string()); cli.lang = Some(id.as_str().to_string());
} }
} }
@@ -193,14 +193,14 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
} else { } else {
SOURCE_HERE SOURCE_HERE
}; };
let answer = select_from(SOURCE_LABEL, &options, default, |answer| { let answer = select_from(prompter, SOURCE_LABEL, &options, default, |answer| {
options.contains(&answer.to_string()) options.contains(&answer.to_string())
})?; })?;
if answer == SOURCE_HERE { if answer == SOURCE_HERE {
cli.source = Some(cwd.clone()); cli.source = Some(cwd.clone());
} else if answer == SOURCE_PATH { } else if answer == SOURCE_PATH {
let validator = |path: &str| validate_directory_answer(path); let validator = |path: &str| validate_directory_answer(path);
let path = prompt::text("Source directory", "", Some(&validator))?; let path = prompter.text("Source directory", "", Some(&validator))?;
cli.source = Some(PathBuf::from(path)); cli.source = Some(PathBuf::from(path));
} }
// SOURCE_SKELETON: cli.source stays unset (the name decides). // SOURCE_SKELETON: cli.source stays unset (the name decides).
@@ -266,7 +266,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
.unwrap_or_else(|| "0.1.0".to_string()); .unwrap_or_else(|| "0.1.0".to_string());
let revision = cli.revision.unwrap_or(1); let revision = cli.revision.unwrap_or(1);
let validate = move |version: &str| options::validate_upstream_version(version, revision); let validate = move |version: &str| options::validate_upstream_version(version, revision);
let answer = ask_text("Upstream version", &default, validate)?; let answer = ask_text(prompter, "Upstream version", &default, validate)?;
cli.upstream_version = Some(answer.clone()); cli.upstream_version = Some(answer.clone());
// The typed version names an existing tag HEAD is not on: offer to // The typed version names an existing tag HEAD is not on: offer to
@@ -279,7 +279,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
"Version {answer} matches tag {tag}, but HEAD is not that \ "Version {answer} matches tag {tag}, but HEAD is not that \
tag. Check out {tag} now?" tag. Check out {tag} now?"
); );
if prompt::confirm(&question, false)? { if prompter.confirm(&question, false)? {
let dir = packaged_dir.as_deref().expect("origin implies a directory"); let dir = packaged_dir.as_deref().expect("origin implies a directory");
crate::new::origin::checkout_tag(dir, tag)?; crate::new::origin::checkout_tag(dir, tag)?;
log::info!( log::info!(
@@ -311,7 +311,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
} else { } else {
"Snapshot this working tree".to_string() "Snapshot this working tree".to_string()
}; };
let answer = select_from(ORIG_LABEL, &labels, &default, |answer| { let answer = select_from(prompter, ORIG_LABEL, &labels, &default, |answer| {
labels.contains(&answer.to_string()) labels.contains(&answer.to_string())
})?; })?;
let chosen = choices let chosen = choices
@@ -322,14 +322,14 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
cli.orig_from = Some(chosen.to_string()); cli.orig_from = Some(chosen.to_string());
if chosen == "path" { if chosen == "path" {
let validator = |path: &str| options::validate_orig_path(path); let validator = |path: &str| options::validate_orig_path(path);
let path = prompt::text("Tarball path or URL", "", Some(&validator))?; let path = prompter.text("Tarball path or URL", "", Some(&validator))?;
cli.orig_path = Some(path); cli.orig_path = Some(path);
} }
} }
// 5. Debian revision. // 5. Debian revision.
if cli.revision.is_none() { if cli.revision.is_none() {
let answer = ask_text("Debian revision", "1", validate_revision_answer)?; let answer = ask_text(prompter, "Debian revision", "1", validate_revision_answer)?;
cli.revision = answer.parse::<u32>().ok(); cli.revision = answer.parse::<u32>().ok();
} }
@@ -340,7 +340,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
.and_then(|p| p.description.clone()) .and_then(|p| p.description.clone())
.unwrap_or_default(); .unwrap_or_default();
let validate = required_answer("the description"); let validate = required_answer("the description");
let answer = ask_text("One-line description", &default, validate)?; let answer = ask_text(prompter, "One-line description", &default, validate)?;
cli.description = Some(answer); cli.description = Some(answer);
} }
@@ -357,7 +357,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
options::validate_homepage(url) options::validate_homepage(url)
} }
}; };
let answer = ask_text("Homepage (blank to skip)", &default, validate)?; let answer = ask_text(prompter, "Homepage (blank to skip)", &default, validate)?;
if !answer.is_empty() { if !answer.is_empty() {
cli.homepage = Some(answer); cli.homepage = Some(answer);
} }
@@ -373,12 +373,17 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
.or_else(|| detect::sniff_license(&detect_dir)); .or_else(|| detect::sniff_license(&detect_dir));
let (default, custom_default) = license_question_default(detected.as_deref()); let (default, custom_default) = license_question_default(detected.as_deref());
let options = license_menu(); let options = license_menu();
let answer = select_from(LICENSE_LABEL, &options, &default, |answer| { let answer = select_from(prompter, LICENSE_LABEL, &options, &default, |answer| {
options.contains(&answer.to_string()) options.contains(&answer.to_string())
})?; })?;
if answer == LICENSE_OTHER { if answer == LICENSE_OTHER {
let validate = required_answer("the license identifier"); let validate = required_answer("the license identifier");
let license = ask_text("License (SPDX identifier)", &custom_default, validate)?; let license = ask_text(
prompter,
"License (SPDX identifier)",
&custom_default,
validate,
)?;
cli.license = Some(license); cli.license = Some(license);
} else { } else {
cli.license = Some(answer); cli.license = Some(answer);
@@ -395,7 +400,12 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
.as_ref() .as_ref()
.and_then(|p| p.command.clone()) .and_then(|p| p.command.clone())
.unwrap_or_else(|| cli.name.clone().unwrap_or_default()); .unwrap_or_else(|| cli.name.clone().unwrap_or_default());
let command = ask_text("Command name", &default, options::validate_command)?; let command = ask_text(
prompter,
"Command name",
&default,
options::validate_command,
)?;
cli.command = Some(command); cli.command = Some(command);
} }
@@ -421,13 +431,13 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
); );
} }
let validate = |answer: &str| options::parse_maintainer(answer).map(|_| ()); let validate = |answer: &str| options::parse_maintainer(answer).map(|_| ());
cli.maintainer = Some(ask_text("Maintainer", &default, validate)?); cli.maintainer = Some(ask_text(prompter, "Maintainer", &default, validate)?);
} }
// 11. Target distribution. The menu derives from the distro data pkh // 11. Target distribution. The menu derives from the distro data pkh
// ships (sorted); ubuntu is moved to the front when present so it // ships (sorted); ubuntu is moved to the front when present so it
// stays the menu's first entry and fallback default as it has always // stays the menu's first entry and fallback default as it has always
// been — prompt::select positions on a default value, not an index. // been — the selector positions on a default value, not an index.
if cli.dist.is_none() { if cli.dist.is_none() {
let vendor = crate::build::env::current_vendor().to_lowercase(); let vendor = crate::build::env::current_vendor().to_lowercase();
let mut options = crate::distro_info::supported_dists(); let mut options = crate::distro_info::supported_dists();
@@ -442,7 +452,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
} else { } else {
"ubuntu".to_string() "ubuntu".to_string()
}; };
let answer = select_from(DIST_LABEL, &options, &default, |answer| { let answer = select_from(prompter, DIST_LABEL, &options, &default, |answer| {
options.contains(&answer.to_string()) options.contains(&answer.to_string())
})?; })?;
cli.dist = Some(answer); cli.dist = Some(answer);
@@ -456,7 +466,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
if cli.series.is_none() { if cli.series.is_none() {
match crate::distro_info::get_ordered_series_name(&dist).await { match crate::distro_info::get_ordered_series_name(&dist).await {
Ok(series) if !series.is_empty() => { Ok(series) if !series.is_empty() => {
let answer = prompt::select(SERIES_LABEL, &series, &series[0])?; let answer = prompter.select(SERIES_LABEL, &series, &series[0])?;
cli.series = Some(answer); cli.series = Some(answer);
} }
_ => { _ => {
@@ -472,6 +482,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
if template == TemplateId::EMPTY && cli.depends.is_empty() { if template == TemplateId::EMPTY && cli.depends.is_empty() {
let validate = |answer: &str| options::validate_depends(answer).map(|_| ()); let validate = |answer: &str| options::validate_depends(answer).map(|_| ());
let answer = ask_text( let answer = ask_text(
prompter,
"Depends (metapackage, comma-separated, blank for an empty base)", "Depends (metapackage, comma-separated, blank for an empty base)",
"", "",
validate, validate,
@@ -494,7 +505,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
// `--no-git` already declined, or there is nothing to initialize. // `--no-git` already declined, or there is nothing to initialize.
cli.git = false; cli.git = false;
} else { } else {
cli.git = prompt::confirm("Initialize a git repository?", cli.git)?; cli.git = prompter.confirm("Initialize a git repository?", cli.git)?;
} }
// Resolve through the same pipeline as the non-interactive path: one // Resolve through the same pipeline as the non-interactive path: one
@@ -505,7 +516,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
// build resolve libraries through pkg-config? The project files prefill // build resolve libraries through pkg-config? The project files prefill
// the default (dependency() / pkg_check_modules calls found). // the default (dependency() / pkg_check_modules calls found).
if matches!(template, TemplateId::MESON | TemplateId::CMAKE) if matches!(template, TemplateId::MESON | TemplateId::CMAKE)
&& prompt::confirm( && prompter.confirm(
"Does the build resolve libraries through pkg-config (add it to Build-Depends)?", "Does the build resolve libraries through pkg-config (add it to Build-Depends)?",
pkg_config_hint(&detect_dir, template), pkg_config_hint(&detect_dir, template),
)? )?
@@ -515,7 +526,7 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
// Wizard-only extras (default off). // Wizard-only extras (default off).
if template != TemplateId::EMPTY if template != TemplateId::EMPTY
&& prompt::confirm( && prompter.confirm(
"Add an autopkgtest smoke test (debian/tests/control)?", "Add an autopkgtest smoke test (debian/tests/control)?",
false, false,
)? )?
@@ -523,15 +534,15 @@ async fn run_wizard(mut cli: NewCli) -> Result<NewOptions, Box<dyn Error>> {
opts.autopkgtest = true; opts.autopkgtest = true;
} }
if let Some(watch) = watch_template(opts.homepage.as_deref()) if let Some(watch) = watch_template(opts.homepage.as_deref())
&& prompt::confirm("Add a debian/watch release watcher?", false)? && prompter.confirm("Add a debian/watch release watcher?", false)?
{ {
opts.watch = Some(watch); opts.watch = Some(watch);
} }
// Summary screen + final confirmation: Ctrl+C or 'n' abort with // Summary screen + final confirmation: Ctrl+C or 'n' abort with
// nothing written (generation is all-or-nothing later anyway). // nothing written (generation is all-or-nothing later anyway).
println!("{}", summary_text(&opts, toolchain_pin.as_deref())); prompter.present(&summary_text(&opts, toolchain_pin.as_deref()));
if !prompt::confirm("Generate?", true)? { if !prompter.confirm("Generate?", true)? {
return Err("Aborted: nothing was written to disk.".into()); return Err("Aborted: nothing was written to disk.".into());
} }
@@ -553,9 +564,10 @@ pub async fn offer_verification(
outcome: &ScaffoldOutcome, outcome: &ScaffoldOutcome,
multi: &MultiProgress, multi: &MultiProgress,
no_verify: bool, no_verify: bool,
prompter: &dyn Prompter,
) { ) {
let tree = opts.target_dir(&std::env::current_dir().unwrap_or_default()); let tree = opts.target_dir(&std::env::current_dir().unwrap_or_default());
let display = crate::ui::display_path(&tree); let display = crate::report::display_path(&tree);
let display = if display.is_empty() { let display = if display.is_empty() {
".".to_string() ".".to_string()
} else { } else {
@@ -565,7 +577,7 @@ pub async fn offer_verification(
if outcome.vendoring_failed { if outcome.vendoring_failed {
// Set apart from the surrounding success output by blank lines: a // Set apart from the surrounding success output by blank lines: a
// single warning between two success lines is easy to miss. // single warning between two success lines is easy to miss.
println!(); prompter.present("");
log::warn!( log::warn!(
"The Cargo dependencies could NOT be vendored: this package will \ "The Cargo dependencies could NOT be vendored: this package will \
not build until the vendoring is completed by hand:\n\ not build until the vendoring is completed by hand:\n\
@@ -573,10 +585,10 @@ pub async fn offer_verification(
\x20 2. add the printed source replacement to .cargo/config.toml, \ \x20 2. add the printed source replacement to .cargo/config.toml, \
plus `[net] offline = true`" plus `[net] offline = true`"
); );
println!(); prompter.present("");
} }
if no_verify || !is_interactive() { if no_verify || !prompter.interactive() {
return; return;
} }
@@ -585,7 +597,7 @@ pub async fn offer_verification(
} else { } else {
"Verify with `pkh build` now?" "Verify with `pkh build` now?"
}; };
let verify_source = match prompt::confirm(build_offer, !outcome.vendoring_failed) { let verify_source = match prompter.confirm(build_offer, !outcome.vendoring_failed) {
Ok(answer) => answer, Ok(answer) => answer,
Err(_) => return, Err(_) => return,
}; };
@@ -593,12 +605,13 @@ pub async fn offer_verification(
return; return;
} }
let ui = Some(std::sync::Arc::new(crate::ui::deb::DebUi::new(multi))); let ui = std::sync::Arc::new(crate::ui::deb::DebUi::new(multi));
if let Err(e) = crate::build::build_source_package( if let Err(e) = crate::build::build_source_package(crate::build::BuildSourceOptions {
Some(&tree), source: Some(tree.clone()),
crate::build::SourceBuildOptions::default(), options: crate::build::SourceBuildOptions::default(),
ui, view: &*ui,
) { prompter,
}) {
log::error!("Verification source build failed: {e}"); log::error!("Verification source build failed: {e}");
log::info!( log::info!(
"The scaffolded tree is intact. Inspect it, then retry with \ "The scaffolded tree is intact. Inspect it, then retry with \
@@ -612,7 +625,7 @@ pub async fn offer_verification(
return; return;
} }
let verify_deb = match prompt::confirm( let verify_deb = match prompter.confirm(
"Verify with `pkh deb` now? (needs network + build deps)", "Verify with `pkh deb` now? (needs network + build deps)",
false, false,
) { ) {
@@ -623,20 +636,13 @@ pub async fn offer_verification(
return; return;
} }
let ui = Some(std::sync::Arc::new(crate::ui::deb::DebUi::new(multi))); let view = crate::ui::deb::DebUi::new(multi);
if let Err(e) = crate::deb::build_binary_package( if let Err(e) = crate::deb::build_binary_package(crate::deb::DebBuildOptions {
None, series: Some(opts.series.clone()),
Some(&opts.series), cwd: Some(tree.clone()),
None, view: &view,
Some(&tree), ..Default::default()
false, })
None,
None,
None,
None,
ui,
None,
)
.await .await
{ {
log::error!("Verification binary build failed: {e}"); log::error!("Verification binary build failed: {e}");
@@ -803,9 +809,13 @@ fn license_question_default(probe_license: Option<&str>) -> (String, String) {
/// Ask the language question until a known template label (or CLI /// Ask the language question until a known template label (or CLI
/// identifier) is answered — the selector allows typing arbitrary text. /// identifier) is answered — the selector allows typing arbitrary text.
fn select_template(options: &[String], default: &str) -> Result<TemplateId, Box<dyn Error>> { fn select_template(
prompter: &dyn Prompter,
options: &[String],
default: &str,
) -> Result<TemplateId, Box<dyn Error>> {
loop { loop {
let answer = prompt::select(LANGUAGE_LABEL, options, default)?; let answer = prompter.select(LANGUAGE_LABEL, options, default)?;
match TemplateId::from_label(&answer) { match TemplateId::from_label(&answer) {
Some(id) => return Ok(id), Some(id) => return Ok(id),
None => log::warn!( None => log::warn!(
@@ -819,13 +829,14 @@ fn select_template(options: &[String], default: &str) -> Result<TemplateId, Box<
/// Ask a `select` question until `accept` holds for the answer (the /// Ask a `select` question until `accept` holds for the answer (the
/// selector allows typing arbitrary text, which callers may need to reject). /// selector allows typing arbitrary text, which callers may need to reject).
fn select_from( fn select_from(
prompter: &dyn Prompter,
label: &str, label: &str,
options: &[String], options: &[String],
default: &str, default: &str,
accept: impl Fn(&str) -> bool, accept: impl Fn(&str) -> bool,
) -> Result<String, Box<dyn Error>> { ) -> Result<String, Box<dyn Error>> {
loop { loop {
let answer = prompt::select(label, options, default)?; let answer = prompter.select(label, options, default)?;
if accept(&answer) { if accept(&answer) {
return Ok(answer); return Ok(answer);
} }
@@ -838,7 +849,7 @@ fn select_from(
/// upstream version like `1.0-2` carrying a Debian revision) is withheld — /// upstream version like `1.0-2` carrying a Debian revision) is withheld —
/// the question is asked without a default instead of offering one that /// the question is asked without a default instead of offering one that
/// Enter would accept verbatim. /// Enter would accept verbatim.
fn offered_default<'a>(default: &'a str, validate: &prompt::Validator) -> &'a str { fn offered_default<'a>(default: &'a str, validate: &Validator) -> &'a str {
if default.is_empty() || validate(default).is_ok() { if default.is_empty() || validate(default).is_ok() {
default default
} else { } else {
@@ -850,11 +861,7 @@ fn offered_default<'a>(default: &'a str, validate: &prompt::Validator) -> &'a st
/// `default`, and whatever answer is finally proposed — typed or the /// `default`, and whatever answer is finally proposed — typed or the
/// default — must pass `validate`. `Err` carries the validation error so the /// default — must pass `validate`. `Err` carries the validation error so the
/// caller re-asks with it. /// caller re-asks with it.
fn accept_answer( fn accept_answer(answer: &str, default: &str, validate: &Validator) -> Result<String, String> {
answer: &str,
default: &str,
validate: &prompt::Validator,
) -> Result<String, String> {
let answer = if answer.is_empty() { default } else { answer }; let answer = if answer.is_empty() { default } else { answer };
validate(answer).map(|_| answer.to_string()) validate(answer).map(|_| answer.to_string())
} }
@@ -868,6 +875,7 @@ fn accept_answer(
/// validation error) — probe data can never bypass validation and only blow /// validation error) — probe data can never bypass validation and only blow
/// up later in [`options::resolve`]. /// up later in [`options::resolve`].
fn ask_text( fn ask_text(
prompter: &dyn Prompter,
label: &str, label: &str,
default: &str, default: &str,
validate: impl Fn(&str) -> Result<(), String> + 'static, validate: impl Fn(&str) -> Result<(), String> + 'static,
@@ -889,7 +897,7 @@ fn ask_text(
} }
}; };
loop { loop {
let answer = prompt::text(label, default, Some(&accept_empty))?; let answer = prompter.text(label, default, Some(&accept_empty))?;
// Typed non-empty answers were already validated by the prompt; only // Typed non-empty answers were already validated by the prompt; only
// an empty one resolves to the default, pre-decided above. // an empty one resolves to the default, pre-decided above.
let answer = if answer.is_empty() { let answer = if answer.is_empty() {
@@ -1551,7 +1559,7 @@ mod tests {
#[test] #[test]
fn select_labels_carry_their_own_separator() { fn select_labels_carry_their_own_separator() {
// prompt::select renders `> <label><answer>` verbatim; a label // the selector renders `> <label><answer>` verbatim; a label
// without a trailing separator glues the answer to the prompt // without a trailing separator glues the answer to the prompt
// (regression: the wizard once rendered "> LicenseMIT"). // (regression: the wizard once rendered "> LicenseMIT").
for label in SELECT_LABELS { for label in SELECT_LABELS {
+33
View File
@@ -9,6 +9,22 @@ use crate::apt::release::{self, VerifiedRelease};
use crossterm::style::Stylize; use crossterm::style::Stylize;
use log::{debug, warn}; use log::{debug, warn};
/// Split a PPA reference into its `(user, name)` parts
///
/// A PPA is written `user/ppa_name` (e.g. `user/my-ppa`); anything else —
/// more segments, empty parts — is a format error carrying the canonical
/// message.
pub fn split_ppa(ppa: &str) -> Result<(&str, &str), String> {
let parts: Vec<&str> = ppa.split('/').collect();
if parts.len() == 2 && !parts[0].is_empty() && !parts[1].is_empty() {
Ok((parts[0], parts[1]))
} else {
Err(format!(
"Invalid PPA format: '{ppa}'. Expected: user/ppa_name"
))
}
}
/// Convert a PPA specification to a base URL /// Convert a PPA specification to a base URL
/// ///
/// # Arguments /// # Arguments
@@ -943,6 +959,23 @@ pub async fn lookup(
mod tests { mod tests {
use super::*; use super::*;
/// `user/ppa_name` splits into its two parts.
#[test]
fn split_ppa_parses_the_canonical_form() {
assert_eq!(split_ppa("user/my-ppa"), Ok(("user", "my-ppa")));
}
/// Anything but exactly two non-empty segments is rejected, with the
/// canonical message.
#[test]
fn split_ppa_rejects_malformed_references() {
for bad in ["", "user", "user/", "/ppa", "a/b/c"] {
let err = split_ppa(bad).unwrap_err();
assert!(err.contains("Invalid PPA format"), "{err}");
assert!(err.contains(bad), "{err}");
}
}
/// Serve canned byte responses on a local port, one per connection (the /// Serve canned byte responses on a local port, one per connection (the
/// last response repeats), and return the base URL /// last response repeats), and return the base URL
/// ///
+66 -42
View File
@@ -452,50 +452,57 @@ async fn fetch_orig_tarball(
Path::new(&info.stanza.package).to_path_buf() Path::new(&info.stanza.package).to_path_buf()
}; };
// Find the orig tarball in the file list // Upstream tarballs in the file list: the main orig tarball plus, for
// Usually ends with .orig.tar.gz or .orig.tar.xz // multi-orig ("3.0 (quilt)" extra-component) sources, one component
let orig_file = info // tarball per bundled module (`*.orig-<component>.tar.<ext>`). dpkg-source
// unpacks all of them side by side, so a git pull must fetch them all.
let orig_files: Vec<_> = info
.stanza .stanza
.files .files
.iter() .iter()
.find(|f| f.name.contains(".orig.tar.")) .filter(|f| crate::build::changes::is_orig_tarball(&f.name))
.ok_or_else(|| { .collect();
format!( if orig_files.is_empty() {
"Could not find orig tarball in file list for package '{}'. \ return Err(format!(
Available files: {:?}", "Could not find orig tarball in file list for package '{}'. \
info.stanza.package, Available files: {:?}",
info.stanza info.stanza.package,
.files info.stanza
.iter() .files
.map(|f| &f.name) .iter()
.collect::<Vec<_>>() .map(|f| &f.name)
) .collect::<Vec<_>>()
})?; )
let filename = &orig_file.name; .into());
}
// 1. Try executing pristine-tar // 1. Try executing pristine-tar
// Setup pristine-tar branch if needed (by tracking remote branch) // Setup pristine-tar branch if needed (by tracking remote branch)
let _ = setup_pristine_tar_branch(&package_dir, info.dist.as_str()); let _ = setup_pristine_tar_branch(&package_dir, info.dist.as_str());
if let Err(e) = checkout_pristine_tar(&package_dir, filename.as_str()) { for orig_file in orig_files {
debug!( let filename = &orig_file.name;
"pristine-tar failed: {}. Falling back to archive download.",
e
);
// 2. Fallback to archive download if let Err(e) = checkout_pristine_tar(&package_dir, filename.as_str()) {
// We download to the parent directory of the package repo (which is standard for build tools) debug!(
// or the current directory if cwd is None (which effectively is the parent of the package dir) "pristine-tar failed: {}. Falling back to archive download.",
let target_dir = cwd.unwrap_or_else(|| Path::new(".")); e
download_file_checksum( );
format!("{}/{}", info.archive_url, filename).as_str(),
&orig_file.checksum, // 2. Fallback to archive download
orig_file.checksum_algo, // We download to the parent directory of the package repo (which is standard for build tools)
target_dir, // or the current directory if cwd is None (which effectively is the parent of the package dir)
progress, let target_dir = cwd.unwrap_or_else(|| Path::new("."));
) download_file_checksum(
.await?; format!("{}/{}", info.archive_url, filename).as_str(),
&orig_file.checksum,
orig_file.checksum_algo,
target_dir,
progress,
)
.await?;
}
} }
Ok(()) Ok(())
@@ -844,23 +851,29 @@ mod tests {
} }
} }
// Check for orig tarball in package dir (only for non-native packages) // Check for the orig tarballs in the package dir (only for non-native
let mut found_tarball = false; // packages): every orig listed in the stanza must be present, including
// the component tarballs of multi-orig packages (dpkg-source needs them
// all to unpack the merged upstream tree)
let mut found_dsc = false; let mut found_dsc = false;
for entry in std::fs::read_dir(package_dir).unwrap() { for entry in std::fs::read_dir(&package_dir).unwrap() {
let entry = entry.unwrap(); let entry = entry.unwrap();
let name = entry.file_name().to_string_lossy().to_string(); let name = entry.file_name().to_string_lossy().to_string();
if name.contains(".orig.tar.") {
found_tarball = true;
}
if name.ends_with(".dsc") { if name.ends_with(".dsc") {
found_dsc = true; found_dsc = true;
} }
} }
// Only check for orig tarball if the package is not native
if !info.is_native() { if !info.is_native() {
assert!(found_tarball, "Orig tarball not found in package dir"); for file in &info.stanza.files {
if crate::build::changes::is_orig_tarball(&file.name) {
assert!(
package_dir.join(&file.name).exists(),
"Orig tarball '{}' not found in package dir",
file.name
);
}
}
} }
assert!(found_dsc, "DSC file not found in package dir"); assert!(found_dsc, "DSC file not found in package dir");
} }
@@ -914,6 +927,17 @@ mod tests {
test_pull_package_end_to_end("paraview", Some("noble"), None, None).await; test_pull_package_end_to_end("paraview", Some("noble"), None, None).await;
} }
/// Multi-orig ("3.0 (quilt)" extra component) regression test: node-jest
/// ships its bundled modules as separate `*.orig-<component>.tar.xz`
/// tarballs next to the main orig. The git pull path must fetch every
/// component, or the later `dpkg-source -b` quilt verification fails
/// with "can't find file to patch" on the first patch touching a
/// component directory.
#[tokio::test]
async fn test_pull_node_jest_debian_end_to_end() {
test_pull_package_end_to_end("node-jest", Some("trixie"), None, None).await;
}
/// Build a minimal uncompressed ustar archive from (name, data) entries. /// Build a minimal uncompressed ustar archive from (name, data) entries.
/// ///
/// Raw header blocks are crafted instead of using `tar::Builder` because /// Raw header blocks are crafted instead of using `tar::Builder` because
+456
View File
@@ -0,0 +1,456 @@
//! Anonymous FTP transport for the Launchpad PPA upload queue: the graceful
//! degradation of the SFTP transport when the SSH connection itself never
//! comes up (name resolution, TCP, banner or key exchange). dput-ng's plain
//! `ppa:user/ppa` profile pushes over this same queue — ppa.launchpad.net
//! over FTP, anonymous login, incoming `~user/ppa` — so it is the
//! interoperability-tested path.
//!
//! The client is [`suppaftp`]'s (plain-FTP, no TLS) blocking stream on the
//! same time-bounded sockets as the ssh2 transport: the TCP connect and
//! the control/data channel reads and writes all carry timeouts, so a
//! black-holed or stalled server fails the upload instead of hanging it
//! (suppaftp's defaults do not bound them). The upload order (payload
//! first, `.changes` last — the caller passes the files in that order) and
//! the best-effort cleanup of a failed upload (`DELE` of what was already
//! pushed, in reverse upload order) mirror the SFTP path exactly.
use std::fs::File;
use std::io::{Read, Write};
use std::net::{SocketAddr, TcpStream};
use std::path::{Path, PathBuf};
use std::time::Duration;
use suppaftp::types::FileType;
use suppaftp::{FtpError, FtpStream};
use super::ssh;
/// Bounds one TCP connection attempt, to the control channel or a passive
/// data port: `connect(2)` would otherwise block for minutes (or forever,
/// behind a silent firewall). Generous enough for slow links to Launchpad,
/// short enough that a dead target fails in seconds — the same value and
/// rationale as the SSH path's bound.
const TCP_CONNECT_TIMEOUT: Duration = Duration::from_secs(15);
/// Read/write timeout on the control channel: every reply is a few bytes,
/// so a stalled server has had its say by the time this expires.
const CONTROL_TIMEOUT: Duration = Duration::from_secs(60);
/// Write timeout of one data-transfer chunk: the clock restarts at every
/// write, so the wall-clock duration of a large upload is not bounded by
/// design — but a black-holed data connection fails one chunk after this
/// budget.
const DATA_TIMEOUT: Duration = Duration::from_secs(120);
/// Data transfers stream in chunks of this size.
const CHUNK_SIZE: usize = 32 * 1024;
/// Upload `files` to the `incoming` queue of `host:port` over anonymous FTP,
/// in the order given (payload first, `.changes` last — the queue processor
/// must never observe a `.changes` without its payload).
/// `on_progress(name, uploaded_bytes, total_bytes)` reports progress.
///
/// A failure mid-upload best-effort removes what was already pushed (`DELE`,
/// reverse upload order — a lingering payload in the write-only queue area
/// is only hygiene) before returning the original error, like the SFTP
/// path's [`super::cleanup_partial_upload`].
pub fn upload_queue(
host: &str,
port: u16,
incoming: &str,
files: &[(PathBuf, String)],
on_progress: &dyn Fn(&str, u64, u64),
) -> Result<(), Box<dyn std::error::Error>> {
let mut ftp = connect(host, port)?;
let who = std::env::var("USER").unwrap_or_else(|_| "anonymous".to_string());
ftp.login("anonymous".to_string(), format!("{who}@pkh.invalid"))
.map_err(|e| format!("the FTP queue rejected the anonymous login: {e}"))?;
ftp.cwd(incoming)
.map_err(|e| format!("cannot enter the upload queue '{incoming}': {e}"))?;
ftp.transfer_type(FileType::Binary)
.map_err(|e| format!("the FTP queue refused binary transfers: {e}"))?;
// Remote names pushed so far, in upload order, for the cleanup
let mut uploaded: Vec<String> = Vec::new();
for (path, name) in files {
if let Err(e) = store(&mut ftp, path, name, on_progress) {
// The failed file itself joins the cleanup: its `STOR` was
// accepted before the transfer failure, so a partial may be
// sitting in the queue
for leftover in super::cleanup_list(&uploaded, Some(name)) {
match ftp.rm(leftover.as_str()) {
Ok(()) => {
log::info!("Removed leftover {leftover} from the failed upload")
}
Err(e) => log::warn!(
"Could not remove the leftover {leftover} of the \
failed upload: {e}"
),
}
}
return Err(e);
}
uploaded.push(name.clone());
}
// Best effort: the queue keeps what it accepted, so a failure here must
// not fail a completed upload
if let Err(e) = ftp.quit() {
log::debug!("closing the FTP session: {e}");
}
Ok(())
}
/// Connect to the queue, read its banner and set every time bound and
/// workaround the plain suppaftp stream does not carry by itself.
fn connect(host: &str, port: u16) -> Result<FtpStream, Box<dyn std::error::Error>> {
let tcp = ssh::tcp_connect(host, port)?;
let mut ftp = FtpStream::connect_with_stream(tcp)?.passive_stream_builder(data_connect);
{
let control = ftp.get_ref();
control.set_read_timeout(Some(CONTROL_TIMEOUT))?;
control.set_write_timeout(Some(CONTROL_TIMEOUT))?;
}
// A `PASV` reply announcing an unroutable address (a server behind NAT
// that does not know its public IP) means the control connection's peer
ftp.set_passive_nat_workaround(true);
Ok(ftp)
}
/// The passive data-channel connect: bounded like every other network call,
/// where suppaftp's default builder is a plain, unbounded
/// `TcpStream::connect`.
fn data_connect(addr: SocketAddr) -> Result<TcpStream, FtpError> {
TcpStream::connect_timeout(&addr, TCP_CONNECT_TIMEOUT).map_err(FtpError::ConnectionError)
}
/// Upload `path` as `name` over a passive data connection, reporting
/// progress through `on_progress` per chunk. `finish` reads the transfer
/// completion reply — the only way to learn the server accepted the file.
fn store(
ftp: &mut FtpStream,
path: &Path,
name: &str,
on_progress: &dyn Fn(&str, u64, u64),
) -> Result<(), Box<dyn std::error::Error>> {
let mut upload = ftp
.put_with_stream(name)
.map_err(|e| format!("the FTP queue rejected '{name}': {e}"))?;
if let suppaftp::DataStream::Tcp(socket) = upload.get_mut() {
socket.set_write_timeout(Some(DATA_TIMEOUT))?;
}
let mut file =
File::open(path).map_err(|e| format!("cannot read '{}': {e}", path.display()))?;
let total = file.metadata().map(|m| m.len()).unwrap_or(0);
let mut buffer = vec![0u8; CHUNK_SIZE];
let mut uploaded: u64 = 0;
loop {
let read = file.read(&mut buffer)?;
if read == 0 {
break;
}
upload.write_all(&buffer[..read])?;
uploaded += read as u64;
on_progress(name, uploaded, total);
}
upload
.finish()
.map_err(|e| format!("upload of '{name}' failed: {e}"))?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
use std::net::TcpListener;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use parking_lot::Mutex;
/// In-process fake of the Launchpad FTP upload queue: one control
/// session speaking the protocol subset the client uses (banner,
/// USER/PASS, CWD, TYPE, PASV, STOR, DELE, QUIT). Records every
/// command, stores received bytes under the `STOR` name, and can be
/// told to reject one `STOR` (by index) to exercise the cleanup. Its
/// `PASV` replies announce `0.0.0.0`, the NAT form, so the
/// happy-path test proves the control-peer fallback too.
struct FakeQueue {
addr: SocketAddr,
commands: Arc<Mutex<Vec<String>>>,
files: Arc<Mutex<HashMap<String, Vec<u8>>>>,
}
impl FakeQueue {
/// Serve one session on a background thread, greeting it with
/// `banner` and rejecting the `STOR` number `reject_stor`, when
/// set.
fn start(banner: &str, reject_stor: Option<usize>) -> FakeQueue {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap();
let commands = Arc::new(Mutex::new(Vec::new()));
let files = Arc::new(Mutex::new(HashMap::new()));
let (c, f) = (Arc::clone(&commands), Arc::clone(&files));
let banner = banner.to_string();
std::thread::spawn(move || serve(listener, &banner, c, f, reject_stor));
FakeQueue {
addr,
commands,
files,
}
}
}
fn serve(
listener: TcpListener,
banner: &str,
commands: Arc<Mutex<Vec<String>>>,
files: Arc<Mutex<HashMap<String, Vec<u8>>>>,
reject_stor: Option<usize>,
) {
use std::io::BufRead;
let (stream, _) = listener.accept().unwrap();
let mut reader = std::io::BufReader::new(stream.try_clone().unwrap());
let mut writer = stream;
writer
.write_all(format!("{banner}\r\n").as_bytes())
.unwrap();
let mut pending_data: Option<TcpListener> = None;
let stor_index = AtomicUsize::new(0);
loop {
let mut line = String::new();
if reader.read_line(&mut line).unwrap_or(0) == 0 {
break;
}
let cmd = line.trim_end().to_string();
commands.lock().push(cmd.clone());
let (verb, arg) = cmd.split_once(' ').unwrap_or((cmd.as_str(), ""));
match verb {
"USER" | "PASS" => {
writer
.write_all(b"230 Anonymous login ok, access restrictions apply.\r\n")
.unwrap();
}
"CWD" => {
writer.write_all(b"250 Command successful\r\n").unwrap();
}
"TYPE" => {
writer.write_all(b"200 Type set to I\r\n").unwrap();
}
"PASV" => {
let data = TcpListener::bind("127.0.0.1:0").unwrap();
let port = data.local_addr().unwrap().port();
writer
.write_all(
format!(
"227 Entering Passive Mode (0,0,0,0,{},{})\r\n",
port / 256,
port % 256
)
.as_bytes(),
)
.unwrap();
pending_data = Some(data);
}
"STOR" => {
let index = stor_index.fetch_add(1, Ordering::SeqCst);
if Some(index) == reject_stor {
writer.write_all(b"550 Permission denied\r\n").unwrap();
continue;
}
writer.write_all(b"150 Ok to send data\r\n").unwrap();
let (mut data, _) = pending_data.take().unwrap().accept().unwrap();
let mut bytes = Vec::new();
data.read_to_end(&mut bytes).unwrap();
files.lock().insert(arg.to_string(), bytes);
writer.write_all(b"226 Transfer complete\r\n").unwrap();
}
"DELE" => {
if files.lock().remove(arg).is_some() {
writer.write_all(b"250 File deleted\r\n").unwrap();
} else {
writer.write_all(b"550 No such file\r\n").unwrap();
}
}
"QUIT" => {
writer.write_all(b"221 Bye\r\n").unwrap();
break;
}
other => {
writer
.write_all(format!("502 Command '{other}' not implemented\r\n").as_bytes())
.unwrap();
}
}
}
}
/// A temp file with `content`, to upload.
fn upload_file(dir: &Path, name: &str, content: &[u8]) -> (PathBuf, String) {
let path = dir.join(name);
std::fs::write(&path, content).unwrap();
(path, name.to_string())
}
/// The full push: anonymous login, the queue directory entered,
/// payload files uploaded in order before the `.changes`, their bytes
/// intact, and progress reported up to each file's size.
#[test]
fn upload_queue_pushes_payload_before_changes() {
let queue = FakeQueue::start("220 Launchpad upload server", None);
let dir = tempfile::tempdir().unwrap();
let files = vec![
upload_file(dir.path(), "pkg_1.0.orig.tar.xz", b"orig bytes"),
upload_file(dir.path(), "pkg_1.0-1.dsc", b"dsc bytes"),
upload_file(dir.path(), "pkg_1.0-1_source.changes", b"changes bytes"),
];
let seen = Arc::new(Mutex::new(Vec::new()));
let progress_log = Arc::clone(&seen);
upload_queue(
"127.0.0.1",
queue.addr.port(),
"~vhaudiquet/lp2167827",
&files,
&|name, uploaded, total| {
progress_log
.lock()
.push((name.to_string(), uploaded, total))
},
)
.unwrap();
let commands = queue.commands.lock().clone();
assert_eq!(
commands[0], "USER anonymous",
"the anonymous login comes first"
);
assert_eq!(
commands
.iter()
.filter(|c| c == &"CWD ~vhaudiquet/lp2167827")
.count(),
1
);
let cwd = commands.iter().position(|c| c.starts_with("CWD")).unwrap();
let first_stor = commands.iter().position(|c| c.starts_with("STOR")).unwrap();
assert!(cwd < first_stor, "the queue directory is entered first");
assert!(
commands.iter().any(|c| c == "TYPE I"),
"binary transfers are requested"
);
// Upload order: the payload first, the .changes last
let stors: Vec<&String> = commands.iter().filter(|c| c.starts_with("STOR")).collect();
assert_eq!(
stors.iter().map(|s| s.as_str()).collect::<Vec<_>>(),
vec![
"STOR pkg_1.0.orig.tar.xz",
"STOR pkg_1.0-1.dsc",
"STOR pkg_1.0-1_source.changes",
]
);
assert_eq!(*commands.last().unwrap(), "QUIT");
// The received bytes are intact
let stored = queue.files.lock();
assert_eq!(stored.get("pkg_1.0.orig.tar.xz").unwrap(), b"orig bytes");
assert_eq!(stored.get("pkg_1.0-1.dsc").unwrap(), b"dsc bytes");
assert_eq!(
stored.get("pkg_1.0-1_source.changes").unwrap(),
b"changes bytes"
);
// Progress reached each file's size
let progress = seen.lock();
for (path, name) in &files {
let size = path.metadata().unwrap().len();
let reached = progress
.iter()
.any(|(n, uploaded, total)| n == name && *uploaded == size && *total == size);
assert!(reached, "no progress report completed for '{name}'");
}
}
/// A `STOR` the queue rejects fails the upload, and the client removes
/// what it already pushed — the failed file first, then the earlier
/// payloads, in reverse upload order — before returning the error.
#[test]
fn upload_queue_cleans_up_after_a_rejected_stor() {
let queue = FakeQueue::start("220 Launchpad upload server", Some(1));
let dir = tempfile::tempdir().unwrap();
let files = vec![
upload_file(dir.path(), "one.dsc", b"one"),
upload_file(dir.path(), "two.tar.xz", b"two"),
upload_file(dir.path(), "three.changes", b"three"),
];
let err = upload_queue(
"127.0.0.1",
queue.addr.port(),
"~user/ppa",
&files,
&|_, _, _| {},
)
.unwrap_err();
assert!(
err.to_string().contains("rejected 'two.tar.xz'"),
"the error names the rejected file, got: {err}"
);
let commands = queue.commands.lock();
let deles: Vec<&String> = commands.iter().filter(|c| c.starts_with("DELE")).collect();
assert_eq!(
deles.iter().map(|d| d.as_str()).collect::<Vec<_>>(),
vec!["DELE two.tar.xz", "DELE one.dsc"],
"the failed file is removed first, then the earlier payloads"
);
let files = queue.files.lock();
assert!(
!files.contains_key("one.dsc"),
"the pushed payload is removed"
);
assert!(!files.contains_key("three.changes"), "never reached");
}
/// A multi-line banner (`220-first` closed by `220 last`) parses like
/// the single-line form.
#[test]
fn upload_queue_reads_multiline_replies() {
let queue = FakeQueue::start("220-Launchpad\r\n220 upload server", None);
let dir = tempfile::tempdir().unwrap();
let files = vec![upload_file(dir.path(), "pkg.dsc", b"bytes")];
upload_queue(
"127.0.0.1",
queue.addr.port(),
"~user/ppa",
&files,
&|_, _, _| {},
)
.unwrap();
let stored = queue.files.lock();
assert_eq!(stored.get("pkg.dsc").unwrap(), b"bytes");
}
/// Live control-channel handshake with the real Launchpad FTP queue:
/// banner, anonymous login and a `CWD` — nothing is uploaded, the
/// queue is left untouched. For deliberate ad-hoc runs
/// (`cargo test -- --ignored`), not the pre-commit pass: it hits the
/// network.
#[test]
#[ignore = "hits the network: the real Launchpad FTP queue"]
fn live_launchpad_control_channel() {
let (host, port) = crate::launchpad::ppa_ftp_queue();
assert_eq!((host.as_str(), port), ("ppa.launchpad.net", 21));
let mut ftp = connect(&host, port).unwrap();
ftp.login("anonymous", "pkh@invalid").unwrap();
ftp.cwd("/").unwrap();
ftp.quit().unwrap();
}
}
+131 -131
View File
@@ -1,24 +1,18 @@
//! Native upload of built source packages (`pkh put`): the dput //! PPA upload: the dput replacement. Resolves the upload target, discovers
//! replacement. Resolves the upload target, discovers and validates the //! and validates the `.changes` file and its artifacts, then pushes them
//! `.changes` file and its artifacts, then pushes them over SFTP with //! over SFTP with host-key verification and an upload record preventing
//! host-key verification and an upload record preventing accidental //! accidental duplicate uploads. When the SSH connection itself never
//! duplicate uploads. //! comes up, the upload degrades to the anonymous FTP queue — the
//! //! transport dput's plain `ppa:` profiles use — through [`ftp`].
//! Payload files are uploaded first and the `.changes` file last, like
//! dput does, so a partially uploaded set cannot be picked up by the
//! server-side queue processors. A run that fails mid-upload removes its
//! already-uploaded files from the incoming queue (best effort), so a
//! retried upload starts from a clean queue; a failed upload is never
//! recorded in the upload log, so a re-run replays every file.
pub mod changes; pub mod changes;
pub mod ftp;
pub mod ssh; pub mod ssh;
pub mod target; pub mod target;
use std::cmp::Ordering; use std::cmp::Ordering;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use indicatif::{MultiProgress, ProgressBar};
use log::info; use log::info;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@@ -27,10 +21,10 @@ use crate::debian::checksums::FileChecksums;
use crate::debian::control::ControlInfo; use crate::debian::control::ControlInfo;
use crate::debian::version::DebianVersion; use crate::debian::version::DebianVersion;
use crate::launchpad; use crate::launchpad;
use crate::ui; use crate::report::{BuildTarget, BuildView, Prompter};
/// Everything `put` needs to run. /// Everything `put` needs to run.
pub struct PutOptions { pub struct PutOptions<'a> {
/// PPA to upload to, `user/ppa_name` format. /// PPA to upload to, `user/ppa_name` format.
pub ppa: String, pub ppa: String,
/// Explicit `.changes` file to upload; when `None`, the one matching the /// Explicit `.changes` file to upload; when `None`, the one matching the
@@ -42,14 +36,25 @@ pub struct PutOptions {
pub force: bool, pub force: bool,
/// Source package directory (the one containing `debian/`). /// Source package directory (the one containing `debian/`).
pub cwd: PathBuf, pub cwd: PathBuf,
/// Where the upload progress (status messages, per-file byte counts) is
/// reported.
pub view: &'a dyn BuildView,
/// Who answers the host-key question on first contact with the target
/// server (fail-closed when nobody can be asked).
pub prompter: &'a dyn Prompter,
} }
/// Upload the package described by `opts` to its target through `multi`'s /// Upload the package described by `opts` to its target, reporting progress
/// progress bars. /// through the view and asking the prompter when the server is unknown.
pub async fn put( pub async fn put(opts: &PutOptions<'_>) -> Result<(), Box<dyn std::error::Error>> {
opts: &PutOptions, // The display is released on every path (success and early `?` bails)
multi: &MultiProgress, // before the outcome is logged, so no stale status line lingers above it
) -> Result<(), Box<dyn std::error::Error>> { let result = put_impl(opts).await;
opts.view.suspend();
result
}
async fn put_impl(opts: &PutOptions<'_>) -> Result<(), Box<dyn std::error::Error>> {
let target = launchpad::ppa_target(&opts.ppa)?; let target = launchpad::ppa_target(&opts.ppa)?;
let ssh_config = ssh::lookup_ssh_config(&target.fqdn); let ssh_config = ssh::lookup_ssh_config(&target.fqdn);
@@ -79,21 +84,20 @@ pub async fn put(
}; };
let changes = changes::parse(&changes_path)?; let changes = changes::parse(&changes_path)?;
// The summary line stays up for the whole flow: the completion message // The summary line stays up for the whole flow: rendered as the view's
// replaces it on success, and the `PutBars` guard clears it on every // persistent status, with the per-step messages below it
// early `?` bail so the error logged by main is not preceded by stale opts.view.target(BuildTarget {
// bars package: &changes.source,
let mut bars = PutBars::default(); version: &changes.version,
let summary = multi.add(ProgressBar::new(0)); target: &target.label,
// Style and prefix go in before the steady tick: otherwise the first display: format!(
// tick can render one frame with the default bar template "Uploading {} {} to {}",
summary.set_style(ui::spinner_style()); changes.source, changes.version, target.label
summary.set_prefix(format!( ),
"Uploading {} {} to {}", source_only: false,
changes.source, changes.version, target.label // An upload runs no subprocess: nothing to tee
)); tee_log: false,
summary.enable_steady_tick(TICK); });
bars.track(summary.clone());
changes::validate(&changes)?; changes::validate(&changes)?;
// Pre-flight checks for everything the upload queue only rejects after // Pre-flight checks for everything the upload queue only rejects after
@@ -109,14 +113,10 @@ pub async fn put(
), ),
} }
let checking = multi.add(ProgressBar::new(0)); opts.view
checking.set_style(ui::spinner_style()); .message(&format!("Checking {} on Launchpad...", target.label));
checking.set_prefix(format!("Checking {} on Launchpad...", target.label));
checking.enable_steady_tick(TICK);
bars.track(checking.clone());
launchpad::ppa_info(&opts.ppa).await?; launchpad::ppa_info(&opts.ppa).await?;
launchpad::check_ppa_series(&changes.distribution).await?; launchpad::check_ppa_series(&changes.distribution).await?;
clear_bar(&checking);
// Last pre-flight check: a version the PPA already publishes at or // Last pre-flight check: a version the PPA already publishes at or
// above the changes' one would supersede (or reject) this upload // above the changes' one would supersede (or reject) this upload
@@ -134,15 +134,6 @@ pub async fn put(
.into()); .into());
} }
let connecting = multi.add(ProgressBar::new(0));
connecting.set_style(ui::spinner_style());
connecting.set_prefix(format!("Connecting to {login}@{host}:{port}..."));
connecting.enable_steady_tick(TICK);
bars.track(connecting.clone());
let session = ssh::connect(&host, port, &login, &ssh_config)?;
let sftp = ssh::sftp(&session)?;
clear_bar(&connecting);
// Payload first, the .changes file last (like dput), so the server-side // Payload first, the .changes file last (like dput), so the server-side
// queue processor can never pick up an incomplete upload // queue processor can never pick up an incomplete upload
let dir = changes_path.parent().unwrap_or_else(|| Path::new(".")); let dir = changes_path.parent().unwrap_or_else(|| Path::new("."));
@@ -156,57 +147,52 @@ pub async fn put(
.and_then(|n| n.to_str()) .and_then(|n| n.to_str())
.ok_or_else(|| format!("invalid .changes path: {}", changes_path.display()))? .ok_or_else(|| format!("invalid .changes path: {}", changes_path.display()))?
.to_string(); .to_string();
uploads.push((changes_path.clone(), changes_name.clone())); uploads.push((changes_path.clone(), changes_name));
let incoming = target.incoming.trim_end_matches('/'); let incoming = target.incoming.trim_end_matches('/');
// Remote names of the files uploaded so far, in upload order (the // The upload queue is SFTP first, degrading to the anonymous FTP
// .changes last). A run failing mid-upload removes these from the // queue when the SSH connection itself never comes up (name
// write-only incoming queue before returning: the uploaded payloads // resolution, TCP, banner or key exchange): dput pushes PPAs over
// would otherwise linger in the queue area forever, and a .changes // that FTP queue by default, so it is the interoperability-tested
// truncated by a failed close could even be picked up by the scanner. // fallback. A server that answers but refuses the upload (host key
let mut uploaded: Vec<String> = Vec::new(); // not accepted, no matching key) stays an error: silently switching
// transport would bypass the refusal.
for (path, name) in &uploads { opts.view
let size = match path.metadata() { .message(&format!("Connecting to {login}@{host}:{port}..."));
Ok(metadata) => metadata.len(), let transfer = match ssh::connect(&host, port, &login, &ssh_config, opts.prompter) {
Err(e) => { Ok(session) => sftp_transfer(&session, &uploads, incoming, &host, opts.view),
// Nothing was attempted for this file: only what earlier Err(ssh::ConnectFailure::Transport(e)) => {
// iterations uploaded needs removing log::warn!("SSH transport to {host}:{port} failed: {e}");
let error: Box<dyn std::error::Error> = let (ftp_host, ftp_port) = launchpad::ppa_ftp_queue();
format!("cannot stat '{}': {}", path.display(), e).into(); opts.view.message(&format!(
cleanup_partial_upload(&sftp, incoming, &uploaded, None, &host); "Falling back to the anonymous FTP queue on {ftp_host}:{ftp_port} \
return Err(error); (dput's upload method)..."
} ));
}; ftp::upload_queue(
// Same transfer view as pull: prefix line, bar on its own line &ftp_host,
let bar = multi.add(ProgressBar::new(size)); ftp_port,
bar.enable_steady_tick(std::time::Duration::from_millis(50)); incoming,
bar.set_style(ui::transfer_style()); &uploads,
bar.set_prefix(format!("Uploading {name}...")); &|name, uploaded, total| {
opts.view.progress(
let remote = format!("{incoming}/{name}"); &format!("Uploading {name}"),
let result = ssh::upload_file(&sftp, path, &remote, &host, &bar); uploaded as usize,
bar.finish_and_clear(); total as usize,
if let Err(e) = result { );
// The failed file itself joins the cleanup: its remote `create` },
// may have succeeded before the failure, leaving a partial — or, )
// on a failed close, a truncated — file behind
cleanup_partial_upload(&sftp, incoming, &uploaded, Some(name), &host);
return Err(e);
} }
uploaded.push(name.clone()); Err(ssh::ConnectFailure::Refused(e)) => return Err(e),
} };
transfer?;
// Recorded only once the whole upload succeeded: the log backs the // Recorded only once the whole upload succeeded: the log backs the
// duplicate-upload guard, and a failed upload must not count as // duplicate-upload guard, and a failed upload must not count as
// uploaded (a re-run replays every file — `sftp.create` truncates, so // uploaded (a re-run replays every file — `sftp.create` truncates and
// replaying is safe). // the FTP `STOR` overwrites, so replaying is safe).
record_upload(&upload_log_path()?, &record)?; record_upload(&upload_log_path()?, &record)?;
// The completion lines replace the summary bar; the guard's own clear
// at scope exit is a no-op for the already-finished bars
clear_bar(&summary);
info!( info!(
"Upload of {} {} to {} complete.", "Upload of {} {} to {} complete.",
changes.source, changes.version, target.label changes.source, changes.version, target.label
@@ -218,6 +204,54 @@ pub async fn put(
Ok(()) Ok(())
} }
/// Push `uploads` over SFTP: payload files first, the `.changes` last
/// (like dput), so the server-side queue processor can never pick up an
/// incomplete upload. Best-effort removal of a partial upload, mirroring
/// the FTP transport's `DELE` cleanup ([`ftp::upload_queue`]).
fn sftp_transfer(
session: &ssh2::Session,
uploads: &[(PathBuf, String)],
incoming: &str,
host: &str,
view: &dyn BuildView,
) -> Result<(), Box<dyn std::error::Error>> {
let sftp = ssh::sftp(session)?;
// Remote names of the files uploaded so far, in upload order (the
// .changes last). A run failing mid-upload removes these from the
// write-only incoming queue before returning: the uploaded payloads
// would otherwise linger in the queue area forever, and a .changes
// truncated by a failed close could even be picked up by the scanner.
let mut uploaded: Vec<String> = Vec::new();
for (path, name) in uploads {
let size = match path.metadata() {
Ok(metadata) => metadata.len(),
Err(e) => {
// Nothing was attempted for this file: only what earlier
// iterations uploaded needs removing
let error: Box<dyn std::error::Error> =
format!("cannot stat '{}': {}", path.display(), e).into();
cleanup_partial_upload(&sftp, incoming, &uploaded, None, host);
return Err(error);
}
};
let remote = format!("{incoming}/{name}");
let label = format!("Uploading {name}");
let on_progress = |uploaded: u64| view.progress(&label, uploaded as usize, size as usize);
let result = ssh::upload_file(&sftp, path, &remote, host, &on_progress);
if let Err(e) = result {
// The failed file itself joins the cleanup: its remote `create`
// may have succeeded before the failure, leaving a partial — or,
// on a failed close, a truncated — file behind
cleanup_partial_upload(&sftp, incoming, &uploaded, Some(name), host);
return Err(e);
}
uploaded.push(name.clone());
}
Ok(())
}
/// The remote names to attempt removing after a failed upload: everything /// The remote names to attempt removing after a failed upload: everything
/// already uploaded plus, when set, `failed` (the file whose upload just /// already uploaded plus, when set, `failed` (the file whose upload just
/// failed — its remote `create` may have succeeded before the failure, /// failed — its remote `create` may have succeeded before the failure,
@@ -225,8 +259,9 @@ pub async fn put(
/// so a `.changes` is removed before the payloads it references and the /// so a `.changes` is removed before the payloads it references and the
/// queue scanner never observes the payload set shrinking under a /// queue scanner never observes the payload set shrinking under a
/// still-present `.changes`. Pure so the ordering decision is testable /// still-present `.changes`. Pure so the ordering decision is testable
/// without a server; the network side is [`cleanup_partial_upload`]. /// without a server; the network sides are [`cleanup_partial_upload`] and
fn cleanup_list(uploaded: &[String], failed: Option<&str>) -> Vec<String> { /// the FTP transport's `DELE` loop ([`ftp`]).
pub(crate) fn cleanup_list(uploaded: &[String], failed: Option<&str>) -> Vec<String> {
let mut names: Vec<String> = uploaded.to_vec(); let mut names: Vec<String> = uploaded.to_vec();
if let Some(failed) = failed { if let Some(failed) = failed {
names.push(failed.to_string()); names.push(failed.to_string());
@@ -260,41 +295,6 @@ fn cleanup_partial_upload(
} }
} }
/// Steady tick interval of every bar rendered by a `put` run
const TICK: std::time::Duration = std::time::Duration::from_millis(50);
/// Stop `bar`'s steady tick and clear it from the terminal. The tick is
/// disabled first: a tick firing right after the clear would redraw a stale
/// frame, the race [`crate::ui::deb::DebUi::suspend`] guards against too.
/// Clearing twice is harmless: finished bars stay finished.
fn clear_bar(bar: &ProgressBar) {
bar.disable_steady_tick();
bar.finish_and_clear();
}
/// The bars rendered by one `put` run, cleared when the guard drops. The
/// error paths bail out early through `?` and `main` logs the error
/// afterwards, so a bar left unfinished would linger on screen above it.
#[derive(Default)]
struct PutBars {
bars: Vec<ProgressBar>,
}
impl PutBars {
/// Track `bar` so it is cleared with the rest when the guard drops
fn track(&mut self, bar: ProgressBar) {
self.bars.push(bar);
}
}
impl Drop for PutBars {
fn drop(&mut self) {
for bar in &self.bars {
clear_bar(bar);
}
}
}
/// Validate the source package's `Section` (debian/control source stanza) /// Validate the source package's `Section` (debian/control source stanza)
/// against the distribution's valid sections: a bare section or a /// against the distribution's valid sections: a bare section or a
/// `section/subsection` is accepted. Archives reject uploads carrying an /// `section/subsection` is accepted. Archives reject uploads carrying an
@@ -521,7 +521,7 @@ fn discover_changes(cwd: &Path) -> Result<PathBuf, Box<dyn std::error::Error>> {
Pass the one to upload explicitly", Pass the one to upload explicitly",
entry.source, entry.source,
many.iter() many.iter()
.map(|p| ui::display_path(p)) .map(|p| crate::report::display_path(p))
.collect::<Vec<_>>() .collect::<Vec<_>>()
.join(", ") .join(", ")
) )
+54 -18
View File
@@ -20,14 +20,13 @@ use std::net::{SocketAddr, TcpStream, ToSocketAddrs};
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::time::Duration; use std::time::Duration;
use indicatif::ProgressBar;
use log::debug; use log::debug;
use serde::Deserialize; use serde::Deserialize;
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
use ssh2::{CheckResult, HostKeyType, KnownHostFileKind, KnownHosts, Session}; use ssh2::{CheckResult, HostKeyType, KnownHostFileKind, KnownHosts, Session};
use crate::data::embed_data; use crate::data::embed_data;
use crate::ui::prompt; use crate::report::Prompter;
/// Pinned SSH host key fingerprints, loaded from the bundled /// Pinned SSH host key fingerprints, loaded from the bundled
/// `host_keys.yml` data file (same pattern as `distro_info.yml`): data /// `host_keys.yml` data file (same pattern as `distro_info.yml`): data
@@ -254,7 +253,9 @@ fn duration_ms(timeout: Duration) -> u32 {
/// address in order (like `TcpStream::connect` does) with /// address in order (like `TcpStream::connect` does) with
/// [`TCP_CONNECT_TIMEOUT`] per attempt instead of blocking indefinitely. /// [`TCP_CONNECT_TIMEOUT`] per attempt instead of blocking indefinitely.
/// Fails with a message naming the target and every per-address error. /// Fails with a message naming the target and every per-address error.
fn tcp_connect(host: &str, port: u16) -> Result<TcpStream, String> { /// Also the TCP layer of the anonymous FTP fallback transport
/// ([`super::ftp`]), whose connection semantics are identical.
pub(crate) fn tcp_connect(host: &str, port: u16) -> Result<TcpStream, String> {
let addrs: Vec<SocketAddr> = (host, port) let addrs: Vec<SocketAddr> = (host, port)
.to_socket_addrs() .to_socket_addrs()
.map_err(|e| format!("cannot resolve {host}:{port}: {e}"))? .map_err(|e| format!("cannot resolve {host}:{port}: {e}"))?
@@ -293,6 +294,39 @@ fn connect_failed_message(
attempts.len() attempts.len()
) )
} }
/// Why establishing the SSH session failed.
///
/// [`Transport`] failures mean the connection never came up (name
/// resolution, TCP, banner or key exchange): the target may still be
/// reachable over another transport, so `pkh put` degrades to the
/// anonymous FTP queue — dput's default for PPAs ([`super::ftp`]).
/// [`Refused`] failures mean the server answered but rejected the
/// upload (host key not accepted, no matching authentication): silently
/// switching to anonymous FTP would bypass a refusal, so they stay
/// errors.
#[derive(Debug)]
pub enum ConnectFailure {
/// The connection itself never came up.
Transport(Box<dyn std::error::Error>),
/// The server answered but rejected the upload.
Refused(Box<dyn std::error::Error>),
}
impl std::fmt::Display for ConnectFailure {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
ConnectFailure::Transport(e) | ConnectFailure::Refused(e) => write!(f, "{e}"),
}
}
}
impl std::error::Error for ConnectFailure {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
match self {
ConnectFailure::Transport(e) | ConnectFailure::Refused(e) => Some(e.as_ref()),
}
}
}
/// Connect to `host:port`, verify the server host key and authenticate as /// Connect to `host:port`, verify the server host key and authenticate as
/// `login`: every ssh-agent identity first, then the configured and default /// `login`: every ssh-agent identity first, then the configured and default
@@ -303,10 +337,14 @@ pub fn connect(
port: u16, port: u16,
login: &str, login: &str,
config: &SshConfig, config: &SshConfig,
) -> Result<Session, Box<dyn std::error::Error>> { prompter: &dyn Prompter,
let tcp = tcp_connect(host, port)?; ) -> Result<Session, ConnectFailure> {
use ConnectFailure::*;
let mut session = Session::new()?; let tcp = tcp_connect(host, port).map_err(|e| Transport(e.into()))?;
let mut session = Session::new()
.map_err(|e| Transport(format!("cannot initialize the SSH session: {e}").into()))?;
// In blocking mode (the libssh2 default), a call that would block loops // In blocking mode (the libssh2 default), a call that would block loops
// in `_libssh2_wait_socket` (via the `BLOCK_ADJUST` macros of // in `_libssh2_wait_socket` (via the `BLOCK_ADJUST` macros of
// session.h in the vendored libssh2-sys sources), which bounds the // session.h in the vendored libssh2-sys sources), which bounds the
@@ -321,14 +359,14 @@ pub fn connect(
session.set_tcp_stream(tcp); session.set_tcp_stream(tcp);
session session
.handshake() .handshake()
.map_err(|e| format!("SSH handshake with {host} failed: {e}"))?; .map_err(|e| Transport(format!("SSH handshake with {host} failed: {e}").into()))?;
let (key, key_type) = session let (key, key_type) = session
.host_key() .host_key()
.ok_or_else(|| format!("{host} offered no host key"))?; .ok_or_else(|| Transport(format!("{host} offered no host key").into()))?;
verify_host_key(host, port, key, key_type)?; verify_host_key(host, port, key, key_type, prompter).map_err(Refused)?;
authenticate(&session, host, login, config)?; authenticate(&session, host, login, config).map_err(Refused)?;
// Only SFTP open/data calls remain on this session: switch from the // Only SFTP open/data calls remain on this session: switch from the
// connection-phase budget to the generous per-call transfer one // connection-phase budget to the generous per-call transfer one
@@ -353,6 +391,7 @@ fn verify_host_key(
port: u16, port: u16,
key: &[u8], key: &[u8],
key_type: HostKeyType, key_type: HostKeyType,
prompter: &dyn Prompter,
) -> Result<(), Box<dyn std::error::Error>> { ) -> Result<(), Box<dyn std::error::Error>> {
let fingerprint = fingerprint(key); let fingerprint = fingerprint(key);
@@ -387,12 +426,7 @@ fn verify_host_key(
format!("[{host}]:{port}") format!("[{host}]:{port}")
}; };
// The banner is plain output: the confirmation prompt itself if !prompter.accept_host_key(&display, key_type_desc, &fingerprint) {
// must stay a single line for its redraw logic
println!("The authenticity of host '{display}' can't be established.");
println!("{key_type_desc} key fingerprint is {fingerprint}.");
let accepted = prompt::confirm("Accept and store this host key?", false)?;
if !accepted {
return Err(format!("Host key for {display} rejected, aborting upload").into()); return Err(format!("Host key for {display} rejected, aborting upload").into());
} }
if let Some(name) = key_type_name(key_type) { if let Some(name) = key_type_name(key_type) {
@@ -592,7 +626,7 @@ pub fn upload_file(
local: &Path, local: &Path,
remote: &str, remote: &str,
host: &str, host: &str,
bar: &ProgressBar, on_progress: &dyn Fn(u64),
) -> Result<(), Box<dyn std::error::Error>> { ) -> Result<(), Box<dyn std::error::Error>> {
let mut local_file = let mut local_file =
fs::File::open(local).map_err(|e| format!("cannot open '{}': {}", local.display(), e))?; fs::File::open(local).map_err(|e| format!("cannot open '{}': {}", local.display(), e))?;
@@ -602,6 +636,7 @@ pub fn upload_file(
.map_err(|e| format!("cannot create remote file {remote} on {host}: {e}"))?; .map_err(|e| format!("cannot create remote file {remote} on {host}: {e}"))?;
let mut buf = [0u8; 32 * 1024]; let mut buf = [0u8; 32 * 1024];
let mut uploaded: u64 = 0;
loop { loop {
let n = local_file.read(&mut buf)?; let n = local_file.read(&mut buf)?;
if n == 0 { if n == 0 {
@@ -610,7 +645,8 @@ pub fn upload_file(
remote_file remote_file
.write_all(&buf[..n]) .write_all(&buf[..n])
.map_err(|e| format!("failed uploading to {remote} on {host}: {e}"))?; .map_err(|e| format!("failed uploading to {remote} on {host}: {e}"))?;
bar.inc(n as u64); uploaded += n as u64;
on_progress(uploaded);
} }
// Close explicitly: quota-exceeded and similar failures only surface in // Close explicitly: quota-exceeded and similar failures only surface in
+237 -32
View File
@@ -4,15 +4,46 @@
//! and apply them during pull and deb operations. //! and apply them during pull and deb operations.
use crate::data::embed_data; use crate::data::embed_data;
use crate::debian::deps::{Deps, ParseOpts, PkgRelation};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::collections::HashMap; use std::collections::HashMap;
/// Build-dependency resolution rules for a package
///
/// Applied after the declared Build-* fields are parsed and reduced,
/// before the resolver derives anything from them. Dependency strings
/// use the full dependency grammar: `name[:arch] [(op version)]
/// [arches] <restrictions>`.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
pub struct DependencyQuirks {
/// Declared dependency name -> dependency string to resolve in its
/// place. The replacement is parsed fresh and replaces the declared
/// dependency wholesale (qualifier, version, restrictions).
#[serde(default)]
pub replace: HashMap<String, String>,
/// Dependencies to resolve as if the control declared them.
#[serde(default)]
pub inject: Vec<String>,
/// Declared dependency names to ignore.
#[serde(default)]
pub drop: Vec<String>,
}
/// Quirks configuration for a specific operation (pull or deb) /// Quirks configuration for a specific operation (pull or deb)
#[derive(Debug, Clone, Deserialize, Serialize)] #[derive(Debug, Clone, Deserialize, Serialize)]
pub struct OperationQuirks { pub struct OperationQuirks {
/// Extra dependencies to install before the operation /// Series the entry applies to. An empty list applies to every
/// series; packaging workarounds should carry the series they were
/// verified against, so they can be dropped once the upstream
/// packaging catches up.
#[serde(default)] #[serde(default)]
pub extra_dependencies: Vec<String>, pub series: Vec<String>,
/// Build-dependency resolution rules.
#[serde(default)]
pub dependencies: Option<DependencyQuirks>,
/// Additional parameters for the operation /// Additional parameters for the operation
#[serde(default)] #[serde(default)]
@@ -26,15 +57,19 @@ pub struct OperationQuirks {
} }
/// Quirks for a specific package /// Quirks for a specific package
///
/// `pull` and `deb` hold one entry per scope: an operation can carry
/// several entries with different `series` lists; every matching entry
/// applies, in file order.
#[derive(Debug, Clone, Deserialize, Serialize)] #[derive(Debug, Clone, Deserialize, Serialize)]
pub struct PackageQuirks { pub struct PackageQuirks {
/// Quirks to apply during pull operation /// Quirks to apply during pull operation
#[serde(default)] #[serde(default)]
pub pull: Option<OperationQuirks>, pub pull: Vec<OperationQuirks>,
/// Quirks to apply during deb operation /// Quirks to apply during deb operation
#[serde(default)] #[serde(default)]
pub deb: Option<OperationQuirks>, pub deb: Vec<OperationQuirks>,
} }
/// Top-level quirks configuration /// Top-level quirks configuration
@@ -63,63 +98,233 @@ pub fn get_package_quirks<'a>(
config.quirks.get(package) config.quirks.get(package)
} }
/// Get deb-time extra dependencies for a package /// Whether a quirks entry applies to `series`: an empty series filter
/// matches every series, otherwise the series must be listed.
fn entry_applies_to_series(quirks: &OperationQuirks, series: &str) -> bool {
quirks.series.is_empty() || quirks.series.iter().any(|s| s == series)
}
/// Get the build-dependency resolution rules of a package for a series
/// ///
/// This function returns the list of extra dependencies that should be installed /// Every deb entry whose series list matches contributes its rules; the
/// before building a package, as defined in the quirks configuration. /// returned rules apply in file order.
/// ///
/// # Arguments /// # Arguments
/// * `package` - The package name /// * `package` - The package name
/// * `series` - The distribution series (e.g. "resolute")
/// ///
/// # Returns /// # Returns
/// * `Vec<String>` - List of extra dependencies, or empty vector if none /// * `Vec<DependencyQuirks>` - The matching rules, empty when the package
pub fn get_deb_extra_dependencies(package: &str) -> Vec<String> { /// has no deb entry or none applies to the series
if let Some(quirks) = get_package_quirks(&QUIRKS_DATA, package) pub fn get_deb_dependency_quirks(package: &str, series: &str) -> Vec<DependencyQuirks> {
&& let Some(deb_quirks) = &quirks.deb let Some(quirks) = get_package_quirks(&QUIRKS_DATA, package) else {
{ return Vec::new();
return deb_quirks.extra_dependencies.clone(); };
} quirks
.deb
Vec::new() .iter()
.filter(|deb| entry_applies_to_series(deb, series))
.filter_map(|deb| deb.dependencies.clone())
.collect()
} }
/// Get package directories from quirks configuration /// Get package directories from quirks configuration
/// ///
/// This function returns the list of custom package directories to try /// This function returns the list of custom package directories to try
/// when looking for the package source directory. /// when looking for the package source directory: every matching deb
/// entry contributes its directories, falling back to the pull entries
/// when no deb entry carries any.
/// ///
/// # Arguments /// # Arguments
/// * `package` - The package name /// * `package` - The package name
/// * `series` - The distribution series (e.g. "resolute")
/// ///
/// # Returns /// # Returns
/// * `Vec<String>` - List of package directories to try, or empty vector if none /// * `Vec<String>` - List of package directories to try, or empty vector if none
pub fn get_package_directories(package: &str) -> Vec<String> { pub fn get_package_directories(package: &str, series: &str) -> Vec<String> {
if let Some(quirks) = get_package_quirks(&QUIRKS_DATA, package) { let Some(quirks) = get_package_quirks(&QUIRKS_DATA, package) else {
// Check deb quirks first, then pull quirks return Vec::new();
if let Some(deb_quirks) = &quirks.deb };
&& !deb_quirks.package_directory.is_empty()
let mut directories = Vec::new();
for deb in quirks
.deb
.iter()
.filter(|q| entry_applies_to_series(q, series))
{
directories.extend(deb.package_directory.iter().cloned());
}
if directories.is_empty() {
for pull in quirks
.pull
.iter()
.filter(|q| entry_applies_to_series(q, series))
{ {
return deb_quirks.package_directory.clone(); directories.extend(pull.package_directory.iter().cloned());
} }
if let Some(pull_quirks) = &quirks.pull }
&& !pull_quirks.package_directory.is_empty() directories
{ }
return pull_quirks.package_directory.clone();
/// Apply the dependency quirks of `package` in `series` to parsed
/// build-dependency clauses
///
/// Rules apply in order — drop, replace, inject. `replace` matches by
/// declared name wherever the dependency appears; rule names that match
/// nothing are warned about, so stale quirks surface once the upstream
/// packaging is fixed.
pub fn apply_dependency_quirks(
package: &str,
series: &str,
clauses: &mut Vec<Vec<PkgRelation>>,
opts: &ParseOpts,
) -> Result<(), String> {
for deps in get_deb_dependency_quirks(package, series) {
apply_rules(clauses, &deps, opts)?;
}
Ok(())
}
/// Apply one set of dependency rules to parsed clauses.
fn apply_rules(
clauses: &mut Vec<Vec<PkgRelation>>,
deps: &DependencyQuirks,
opts: &ParseOpts,
) -> Result<(), String> {
for name in &deps.drop {
let hits = clauses
.iter()
.flatten()
.filter(|rel| &rel.package == name)
.count();
if hits == 0 {
log::warn!("dependency quirk: 'drop {name}' matched nothing");
}
}
if !deps.drop.is_empty() {
for clause in clauses.iter_mut() {
clause.retain(|rel| !deps.drop.iter().any(|name| name == &rel.package));
}
clauses.retain(|clause| !clause.is_empty());
}
for (declared, replacement) in &deps.replace {
let mut hits = 0;
for clause in clauses.iter_mut() {
for rel in clause.iter_mut() {
if rel.package == *declared {
*rel = crate::debian::deps::parse_simple(replacement, true)
.map_err(|e| format!("invalid replacement '{replacement}': {e}"))?;
hits += 1;
}
}
}
if hits == 0 {
log::warn!("dependency quirk: 'replace {declared}' matched nothing");
} }
} }
Vec::new() for injected in &deps.inject {
let parsed = Deps::parse(injected, opts)?;
clauses.extend(parsed.clauses().map(<[PkgRelation]>::to_vec));
}
Ok(())
} }
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
fn parse(s: &str) -> PkgRelation {
crate::debian::deps::parse_simple(s, true).unwrap()
}
fn opts() -> ParseOpts {
ParseOpts {
host_arch: "riscv64".into(),
build_arch: "amd64".into(),
build_profiles: vec!["cross".into()],
reduce_restrictions: true,
union: false,
build_dep: true,
}
}
#[test] #[test]
fn test_unknown_package_has_no_quirks() { fn test_unknown_package_has_no_quirks() {
// A package absent from quirks.yml (currently every package) has no // A package absent from quirks.yml has no dependency rules nor
// extra dependencies nor custom directories, and must not panic // custom directories, and must not panic
assert!(get_deb_extra_dependencies("not-in-quirks").is_empty()); assert!(get_deb_dependency_quirks("not-in-quirks", "resolute").is_empty());
assert!(get_package_directories("not-in-quirks").is_empty()); assert!(get_package_directories("not-in-quirks", "resolute").is_empty());
}
/// The linux dependency quirks are scoped to the series they were
/// verified against.
#[test]
fn linux_dependency_quirks_are_series_scoped() {
for package in ["linux", "linux-riscv"] {
let rules = get_deb_dependency_quirks(package, "resolute");
assert_eq!(rules.len(), 1, "the resolute entry applies");
assert_eq!(
rules[0].replace.get("llvm-21-dev").map(String::as_str),
Some("llvm-21-dev:native <!stage1>")
);
assert!(get_deb_dependency_quirks(package, "noble").is_empty());
}
}
/// `replace` rewrites exactly the dependencies whose declared name
/// matches, wholesale: the replacement carries its own qualifier and
/// restrictions.
#[test]
fn replace_rewrites_matching_names_only() {
let mut clauses = vec![vec![
parse("llvm-21-dev <!stage1>"),
parse("clang-21:native"),
]];
let deps = DependencyQuirks {
replace: HashMap::from([(
"llvm-21-dev".to_string(),
"llvm-21-dev:native <!stage1>".to_string(),
)]),
..Default::default()
};
apply_rules(&mut clauses, &deps, &opts()).unwrap();
let rewritten = &clauses[0][0];
assert_eq!(rewritten.arch_qualifier.as_deref(), Some("native"));
assert_eq!(rewritten.restrictions.len(), 1);
assert_eq!(clauses[0][1].arch_qualifier.as_deref(), Some("native"));
}
/// `drop` removes named dependencies (empty clauses disappear) and
/// `inject` appends dependencies resolved like declared ones.
#[test]
fn drop_and_inject() {
let mut clauses = vec![vec![parse("broken-dep"), parse("keep-me")]];
let deps = DependencyQuirks {
inject: vec!["injected-dep:any".to_string()],
drop: vec!["broken-dep".to_string()],
..Default::default()
};
apply_rules(&mut clauses, &deps, &opts()).unwrap();
let names: Vec<&str> = clauses
.iter()
.flatten()
.map(|rel| rel.package.as_str())
.collect();
assert_eq!(names, ["keep-me", "injected-dep"]);
}
/// A replacement that does not parse is a quirk configuration error,
/// not a silent no-op.
#[test]
fn invalid_replacement_is_an_error() {
let mut clauses = vec![vec![parse("llvm-21-dev")]];
let deps = DependencyQuirks {
replace: HashMap::from([("llvm-21-dev".to_string(), "not@@valid".to_string())]),
..Default::default()
};
assert!(apply_rules(&mut clauses, &deps, &opts()).is_err());
} }
} }
+207
View File
@@ -0,0 +1,207 @@
//! Environment-agnostic reporting ports.
//!
//! The core flows report progress and ask questions exclusively through the
//! traits in this module, so the same pipeline can drive a terminal live
//! view, a headless library consumer, or a remote frontend (e.g. a builder
//! server forwarding build events to a web UI over server-sent events): every
//! event carries plain data — strings, numbers, paths — with no terminal,
//! styling or locale assumptions. Adapters decide how events reach the user:
//! the terminal live view ([`crate::ui::deb::DebUi`]) renders them in place,
//! while another embedding maps each method onto its own wire format.
//!
//! Every [`BuildView`] method defaults to doing nothing, so implementations
//! only override the events they care about; [`Quiet`] provides the inert
//! implementations used by headless runs and tests. [`Prompter`] is
//! deliberately blocking: an implementation may round-trip each question to
//! a remote user, as long as it eventually answers (or takes the default).
//! Asking can fail (the user cancels, the connection drops); flows
//! propagate the error, which aborts them.
use std::error::Error;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use crate::context::LineSink;
use crate::logfmt::Classifier;
/// Answer validator of [`Prompter::text`]: accepts the answer, or explains
/// why it is rejected (the implementation re-asks with the explanation).
pub type Validator = dyn Fn(&str) -> Result<(), String>;
/// Identity of the build whose events follow, as announced through
/// [`BuildView::target`].
#[derive(Debug, Clone)]
pub struct BuildTarget<'a> {
/// Source package name (e.g. `hello`).
pub package: &'a str,
/// Full version being built (e.g. `2.10-3`).
pub version: &'a str,
/// What the build targets: a distribution series, optionally with an
/// architecture (`noble`, `sid`, `noble/arm64`), or the upload target
/// label.
pub target: &'a str,
/// Ready-to-render status line for display adapters, composed by the
/// flow (e.g. "Building source package hello (2.10-3) for unstable",
/// "Uploading hello (2.10-3) to ppa:user/ppa"): the wording is the
/// flow's, adapters render it verbatim.
pub display: String,
/// Whether this is a source-only build (producing a `.dsc`); names the
/// terminal adapter's tee log (`build-*.log` vs `deb-*.log`).
pub source_only: bool,
/// Whether the view should tee raw subprocess output to its log file.
/// Flows without subprocess output (uploads) pass `false` and create
/// no log file.
pub tee_log: bool,
}
/// Observer of a running build: target identification, phases, status
/// messages, progress and the final outcome.
///
/// Implement this to observe the [`crate::build`] and [`crate::deb`] flows
/// from any frontend. All events arrive in order from the build thread;
/// long-lived views are expected to be `Send + Sync` because builds may run
/// inside async tasks.
pub trait BuildView: Send + Sync {
/// The build target was identified; the events that follow belong to it
/// (including the earliest subprocess output, e.g. a chroot download).
fn target(&self, _target: BuildTarget) {}
/// A named phase started (e.g. "Applying patches"). `classifier`
/// rewrites the phase's raw subprocess lines (see [`crate::logfmt`])
/// into display actions and countable progress; views that do not
/// rewrite lines locally can ignore it and forward raw lines from
/// [`BuildView::sink`] instead.
fn phase(&self, _name: &str, _classifier: Box<dyn Classifier>) {}
/// A status message about in-process work that produces no subprocess
/// output (e.g. "Generating .changes").
fn message(&self, _text: &str) {}
/// Determinate progress within the current phase (e.g. artifact
/// retrieval); `pos` runs from 0 to `total`.
fn progress(&self, _label: &str, _pos: usize, _total: usize) {}
/// Sink receiving every raw subprocess line while a build command runs,
/// when this view consumes the lines itself (live rewriting, tee to a
/// log file, forwarding over the network). `None` lets the caller fall
/// back to its default handling (e.g. test capture).
fn sink(&self) -> Option<Arc<dyn LineSink>> {
None
}
/// The build succeeded; `artifacts` lists the produced files in
/// distribution order (dsc → tarballs → buildinfo → changes).
fn finish_success(&self, _artifacts: &[PathBuf]) {}
/// The build failed; the view should release the display and may report
/// diagnostics it collected through [`BuildView::sink`].
fn finish_failure(&self) {}
/// Release the display before writing directly to the shared terminal
/// (passthrough diagnostics, cleanup commands that inherit it). The
/// release is final for this build; later events may be dropped. A
/// no-op for views without a display.
fn suspend(&self) {}
/// Whether this view presents build results to the user by itself;
/// callers use this to fall back to plain-line rendering when it does
/// not (headless views, verbose mode).
fn is_enabled(&self) -> bool {
false
}
}
/// Answerer of the questions a core flow may ask mid-run.
///
/// Questions are blocking on purpose: a terminal implementation waits for
/// key presses, and a builder-server implementation may forward the question
/// to a web client and await the answer on a channel. Implementations that
/// cannot ask anyone answer with the question's default.
pub trait Prompter: Send + Sync {
/// Whether this prompter can interact with a user at all. Flows with an
/// interactive and a headless path use this to pick one: a headless run
/// takes the defaults or fails with the list of missing answers instead
/// of asking question by question.
fn interactive(&self) -> bool {
false
}
/// Ask a yes/no question. `default` is the answer to take when no user
/// can be reached; `Err` means the question was cancelled (Ctrl+C,
/// dropped connection) and the flow should abort.
fn confirm(&self, _question: &str, default: bool) -> Result<bool, Box<dyn Error>> {
Ok(default)
}
/// Ask a one-line selection among `options`, with `default`
/// preselected. Selector implementations may also accept typed
/// arbitrary text; callers validate the answer and re-ask through the
/// same method when they must reject one. `Err` cancels the flow.
fn select(
&self,
_label: &str,
_options: &[String],
default: &str,
) -> Result<String, Box<dyn Error>> {
Ok(default.to_string())
}
/// Ask a free-text answer. `validate` is applied by the implementation
/// so invalid input re-asks at the source (mid-input for a terminal,
/// round-trip for a server). `Err` cancels the flow.
fn text(
&self,
_label: &str,
default: &str,
_validate: Option<&Validator>,
) -> Result<String, Box<dyn Error>> {
Ok(default.to_string())
}
/// Ask whether to accept and store an unverified SSH host key (trust on
/// first use): `host` is the display form (`host` or `[host]:port`),
/// `key_type` the key type name ("ssh-ed25519", ...) and `fingerprint`
/// the human-readable digest. Fail-closed: implementations that cannot
/// ask anyone answer `false`, refusing the connection.
fn accept_host_key(&self, _host: &str, _key_type: &str, _fingerprint: &str) -> bool {
false
}
/// Present information to the user outside of any question: the
/// scaffold wizard's summary screen, prominent notices around a
/// warning. Terminal implementations print the text as-is (unstyled,
/// on stdout); server implementations forward it as a display event.
/// Implementations that cannot show anything drop it — flows only
/// present context that is also available structurally (returned data,
/// log records).
fn present(&self, _text: &str) {}
}
/// Inert view and prompter: drops every event and answers every question
/// with its default. The stand-in for headless library runs, verbose mode
/// and tests.
pub struct Quiet;
impl BuildView for Quiet {}
impl Prompter for Quiet {}
/// Render a path for terminal display: relative to the current working
/// directory when the target lives inside it or directly next to it
/// (`../name`, the usual layout of build artifacts), absolute otherwise.
///
/// Pure formatting shared by the flows that mention paths in their events
/// and messages; a remote frontend reproduces it (or not) on its side.
pub fn display_path(path: &Path) -> String {
let Ok(cwd) = std::env::current_dir() else {
return path.display().to_string();
};
if let Ok(rel) = path.strip_prefix(&cwd) {
return rel.display().to_string();
}
if let Some(parent) = cwd.parent()
&& let Ok(rel) = path.strip_prefix(parent)
{
return format!("../{}", rel.display());
}
path.display().to_string()
}
+4
View File
@@ -445,6 +445,10 @@ mod imp {
self.inner.exists(path) self.inner.exists(path)
} }
fn is_dir(&self, path: &Path) -> io::Result<bool> {
self.inner.is_dir(path)
}
fn cleanup(&self) -> io::Result<()> { fn cleanup(&self) -> io::Result<()> {
self.inner.cleanup() self.inner.cleanup()
} }
-21
View File
@@ -3,8 +3,6 @@
/// Live build view for `pkh deb` (status bar + rolling log pane) /// Live build view for `pkh deb` (status bar + rolling log pane)
pub mod deb; pub mod deb;
/// Line classifiers rewriting raw subprocess output for the live views
pub mod logfmt;
/// Interactive raw-mode prompts: free-text input, option selection and /// Interactive raw-mode prompts: free-text input, option selection and
/// yes/no confirmation /// yes/no confirmation
pub mod prompt; pub mod prompt;
@@ -12,27 +10,8 @@ pub mod prompt;
#[cfg(test)] #[cfg(test)]
use indicatif::ProgressDrawTarget; use indicatif::ProgressDrawTarget;
use indicatif::{MultiProgress, ProgressBar, ProgressStyle}; use indicatif::{MultiProgress, ProgressBar, ProgressStyle};
use std::path::Path;
use std::time::Duration; use std::time::Duration;
/// Render a path for terminal display: relative to the current working
/// directory when the target lives inside it or directly next to it
/// (`../name`, the usual layout of build artifacts), absolute otherwise.
pub fn display_path(path: &Path) -> String {
let Ok(cwd) = std::env::current_dir() else {
return path.display().to_string();
};
if let Ok(rel) = path.strip_prefix(&cwd) {
return rel.display().to_string();
}
if let Some(parent) = cwd.parent()
&& let Ok(rel) = path.strip_prefix(parent)
{
return format!("../{}", rel.display());
}
path.display().to_string()
}
/// Style of an unsized operation: spinner and prefix on one line /// Style of an unsized operation: spinner and prefix on one line
pub(crate) fn spinner_style() -> ProgressStyle { pub(crate) fn spinner_style() -> ProgressStyle {
ProgressStyle::default_bar() ProgressStyle::default_bar()
+447 -238
View File
@@ -3,7 +3,7 @@
//! ("a terminal in the terminal"). //! ("a terminal in the terminal").
//! //!
//! Subprocess output is captured through a [`LineSink`] implementation, //! Subprocess output is captured through a [`LineSink`] implementation,
//! rewritten by classifiers ([`crate::ui::logfmt`]) and rendered in place //! rewritten by classifiers ([`crate::logfmt`]) and rendered in place
//! with indicatif, so pkh's own log lines keep printing above the widget via //! with indicatif, so pkh's own log lines keep printing above the widget via
//! `indicatif-log-bridge`. Every raw captured line is also tee'd to a log //! `indicatif-log-bridge`. Every raw captured line is also tee'd to a log
//! file under the pkh cache directory. //! file under the pkh cache directory.
@@ -11,20 +11,19 @@
use std::collections::VecDeque; use std::collections::VecDeque;
use std::fs::{self, File}; use std::fs::{self, File};
use std::io::Write; use std::io::Write;
use std::path::{Path, PathBuf}; use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
use crossterm::{cursor, execute, style::Stylize, terminal::Clear, terminal::ClearType}; use crossterm::style::Stylize;
use directories::ProjectDirs; use directories::ProjectDirs;
use indicatif::{MultiProgress, ProgressBar, ProgressStyle}; use indicatif::{MultiProgress, ProgressBar, ProgressDrawTarget, ProgressStyle};
use unicode_width::{UnicodeWidthChar, UnicodeWidthStr};
use crate::context::{LineSink, Stream}; use crate::context::{LineSink, Stream};
use crate::ui::logfmt::{ use crate::logfmt::{Action, Classifier, GenericClassifier};
Action, AptInstallClassifier, AptUpdateClassifier, Classifier, GenericClassifier, use crate::report::BuildTarget;
MakeClassifier, MmdebstrapClassifier, QuiltClassifier,
};
/// Number of lines displayed in the rolling pane /// Number of lines displayed in the rolling pane
const PANE_LINES: usize = 10; const PANE_LINES: usize = 10;
@@ -32,69 +31,6 @@ const PANE_LINES: usize = 10;
/// Minimum interval between pane redraws /// Minimum interval between pane redraws
const REDRAW_INTERVAL: Duration = Duration::from_millis(50); const REDRAW_INTERVAL: Duration = Duration::from_millis(50);
/// Build phases of `pkh deb`, shown in the status bar
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Phase {
/// Downloading the chroot tarball (mmdebstrap)
PreparingChroot,
/// Extracting the chroot tarball
ExtractingChroot,
/// Device nodes, /proc bind mount, etc.
FinalizingChroot,
/// apt-get update
UpdatingPackageLists,
/// Installing build-essential & co
InstallingEssentials,
/// quilt push -a
ApplyingPatches,
/// --inject packages
InjectingPackages,
/// apt-get build-dep
InstallingBuildDeps,
/// debian/rules build
Building,
/// fakeroot debian/rules binary
ProducingBinaries,
/// Retrieving produced .deb files
RetrievingArtifacts,
}
impl Phase {
/// Human-readable label displayed in the status bar
pub fn label(&self) -> &'static str {
match self {
Phase::PreparingChroot => "Preparing chroot",
Phase::ExtractingChroot => "Extracting chroot",
Phase::FinalizingChroot => "Finalizing chroot",
Phase::UpdatingPackageLists => "Updating package lists",
Phase::InstallingEssentials => "Installing essential packages",
Phase::ApplyingPatches => "Applying patches",
Phase::InjectingPackages => "Injecting packages",
Phase::InstallingBuildDeps => "Installing build dependencies",
Phase::Building => "Building package",
Phase::ProducingBinaries => "Producing binary packages",
Phase::RetrievingArtifacts => "Retrieving artifacts",
}
}
}
/// Default classifier used for a given phase
fn default_classifier(phase: Phase) -> Box<dyn Classifier> {
match phase {
Phase::PreparingChroot => Box::new(MmdebstrapClassifier::new()),
Phase::ExtractingChroot | Phase::FinalizingChroot => Box::new(GenericClassifier::new()),
Phase::UpdatingPackageLists => Box::new(AptUpdateClassifier::new()),
Phase::InstallingEssentials => Box::new(AptInstallClassifier::new("Installing essentials")),
Phase::ApplyingPatches => Box::new(QuiltClassifier::new(0)),
Phase::InjectingPackages => Box::new(AptInstallClassifier::new("Injecting packages")),
Phase::InstallingBuildDeps => {
Box::new(AptInstallClassifier::new("Installing build dependencies"))
}
Phase::Building | Phase::ProducingBinaries => Box::new(MakeClassifier::new()),
Phase::RetrievingArtifacts => Box::new(GenericClassifier::new()),
}
}
/// Visual kind of a pane line, driving its color /// Visual kind of a pane line, driving its color
#[derive(Debug, Clone, Copy, PartialEq)] #[derive(Debug, Clone, Copy, PartialEq)]
enum Kind { enum Kind {
@@ -114,8 +50,11 @@ struct Pipeline {
/// State shared between [`DebUi`] and its sinks /// State shared between [`DebUi`] and its sinks
struct Shared { struct Shared {
multi: MultiProgress,
top: ProgressBar, top: ProgressBar,
pane: ProgressBar, /// The rolling pane bar, created on demand: flows without subprocess
/// output (e.g. `pkh put`) never show it at all
pane: Mutex<Option<ProgressBar>>,
state: Mutex<Pipeline>, state: Mutex<Pipeline>,
tee: Mutex<Option<File>>, tee: Mutex<Option<File>>,
log_path: Mutex<PathBuf>, log_path: Mutex<PathBuf>,
@@ -129,9 +68,9 @@ struct Shared {
/// Live build view for `pkh deb` / `pkh build` /// Live build view for `pkh deb` / `pkh build`
/// ///
/// Create one per build (disabled automatically when stdout is not a TTY or /// Create one per build (it disables itself automatically when stdout is not
/// when the user requests verbose output), pass it down as /// a TTY) and pass it down through the [`crate::report::BuildView`] port.
/// `Option<Arc<DebUi>>`, and feed subprocess output through [`DebUi::sink`]. /// Subprocess output reaches it through [`crate::report::BuildView::sink`].
pub struct DebUi { pub struct DebUi {
shared: Arc<Shared>, shared: Arc<Shared>,
} }
@@ -145,38 +84,33 @@ impl DebUi {
let enabled = is_stdout_tty(); let enabled = is_stdout_tty();
let top = if enabled { let top = if enabled {
// The tty renders a ^C keypress as the two visible characters
// "^C"; on a terminal where the cursor sits near the right edge
// that wraps to the next row, and the erase at teardown —
// anchored to where indicatif last drew — ends up one row off,
// leaving the first widget line on screen. Rendering control
// characters raw instead (ECHOCTL off) makes the echo an
// invisible byte that moves nothing; ECHO itself stays on, so
// terminals showing a padlock while input is hidden are not
// triggered. Restored by `suspend_shared`.
suppress_control_char_echo();
multi.set_draw_target(ProgressDrawTarget::stderr());
let pb = multi.add(ProgressBar::new(0)); let pb = multi.add(ProgressBar::new(0));
pb.enable_steady_tick(Duration::from_millis(80)); pb.enable_steady_tick(Duration::from_millis(80));
pb.set_style(spinner_style()); pb.set_style(spinner_style());
pb.set_prefix("Building package"); pb.set_prefix("Building package");
pb.set_message("(starting…)");
pb
} else {
ProgressBar::hidden()
};
let pane = if enabled {
let pb = multi.add(ProgressBar::new(0));
pb.enable_steady_tick(Duration::from_millis(150));
// No template margin: multi-line messages are only prefixed by
// the template on their first line, which would misalign the
// pane; each rendered line carries its own indent instead.
pb.set_style(
ProgressStyle::default_bar()
.template("{msg}")
.expect("valid template"),
);
pb.set_message(" │ (starting…)");
pb pb
} else { } else {
ProgressBar::hidden() ProgressBar::hidden()
}; };
let pane = Mutex::new(None);
let timestamp = chrono::Utc::now().format("%Y%m%dT%H%M%S").to_string(); let timestamp = chrono::Utc::now().format("%Y%m%dT%H%M%S").to_string();
let log_path = default_log_path(&timestamp); let log_path = default_log_path(&timestamp);
let ui = Self { let ui = Self {
shared: Arc::new(Shared { shared: Arc::new(Shared {
multi: multi.clone(),
top, top,
pane, pane,
state: Mutex::new(Pipeline { state: Mutex::new(Pipeline {
@@ -187,7 +121,7 @@ impl DebUi {
bar_total: 0, bar_total: 0,
}), }),
tee: Mutex::new(None), tee: Mutex::new(None),
log_path: Mutex::new(log_path.clone()), log_path: Mutex::new(log_path),
timestamp, timestamp,
enabled, enabled,
suspended: AtomicBool::new(false), suspended: AtomicBool::new(false),
@@ -195,8 +129,13 @@ impl DebUi {
}), }),
}; };
// Ctrl+C: the signal wiring lives in the CLI; this view only
// registers with `crate::interrupt` how to clear itself and where
// the full log lives. The log path is read from the shared state at
// interrupt time, so the rename in `open_log` stays visible to the
// reporter.
if ui.shared.enabled { if ui.shared.enabled {
install_sigint_hook(&log_path); set_interrupt_reporter(ui.shared.clone());
} }
ui ui
@@ -205,7 +144,7 @@ impl DebUi {
/// Identify the binary package being built; names the log file and the /// Identify the binary package being built; names the log file and the
/// status bar /// status bar
pub fn set_target(&self, package: &str, version: &str, series: &str, arch: &str) { pub fn set_target(&self, package: &str, version: &str, series: &str, arch: &str) {
if self.shared.enabled { if self.active() {
self.shared.top.set_prefix(format!( self.shared.top.set_prefix(format!(
"Building {package} ({version}) for {series}/{arch}" "Building {package} ({version}) for {series}/{arch}"
)); ));
@@ -213,17 +152,6 @@ impl DebUi {
self.open_log("deb", package, version, &format!("for {series}/{arch}")); self.open_log("deb", package, version, &format!("for {series}/{arch}"));
} }
/// Identify the source package being built; names the log file
/// (`build-<package>-<version>-<timestamp>.log`) and the status bar
pub fn set_build_target(&self, package: &str, version: &str, distribution: &str) {
if self.shared.enabled {
self.shared.top.set_prefix(format!(
"Building source package {package} ({version}) for {distribution}"
));
}
self.open_log("build", package, version, &format!("for {distribution}"));
}
/// Rename the placeholder log file to include the build identity /// Rename the placeholder log file to include the build identity
/// (best-effort), then open it so subsequent captured lines are tee'd /// (best-effort), then open it so subsequent captured lines are tee'd
fn open_log(&self, kind: &str, package: &str, version: &str, detail: &str) { fn open_log(&self, kind: &str, package: &str, version: &str, detail: &str) {
@@ -237,7 +165,6 @@ impl DebUi {
}; };
let _ = fs::rename(&old_path, &log_path); let _ = fs::rename(&old_path, &log_path);
*self.shared.log_path.lock().unwrap() = log_path.clone(); *self.shared.log_path.lock().unwrap() = log_path.clone();
update_sigint_log_path(&log_path);
if let Some(dir) = log_path.parent() { if let Some(dir) = log_path.parent() {
let _ = fs::create_dir_all(dir); let _ = fs::create_dir_all(dir);
@@ -261,20 +188,8 @@ impl DebUi {
} }
} }
/// Switch to a phase, installing its default classifier /// Switch to an arbitrary status label with a custom classifier
pub fn phase(&self, phase: Phase) { fn phase_custom(&self, label: &str, classifier: Box<dyn Classifier>) {
self.phase_custom(phase.label(), default_classifier(phase));
}
/// Switch to a phase with a custom classifier (e.g. quilt with a known
/// patch count)
pub fn phase_with(&self, phase: Phase, classifier: Box<dyn Classifier>) {
self.phase_custom(phase.label(), classifier);
}
/// Switch to an arbitrary status label with a custom classifier; used by
/// flows whose phases are not part of [`Phase`] (e.g. source builds)
pub fn phase_custom(&self, label: &str, classifier: Box<dyn Classifier>) {
{ {
let mut st = self.shared.state.lock().unwrap(); let mut st = self.shared.state.lock().unwrap();
st.classifier = classifier; st.classifier = classifier;
@@ -282,89 +197,50 @@ impl DebUi {
st.bar_total = 0; st.bar_total = 0;
st.last_draw = Instant::now(); st.last_draw = Instant::now();
} }
if self.shared.enabled { if self.active() {
self.shared.top.set_style(spinner_style()); self.shared.top.set_style(spinner_style());
self.shared.top.set_message(label.to_string()); self.shared.top.set_message(label.to_string());
self.shared.pane.set_message(""); drop_pane(&self.shared);
} }
} }
/// Update the status bar message directly (for in-process work such as
/// tarball extraction that has no subprocess output)
pub fn progress_message(&self, msg: &str) {
if self.active() {
self.shared.top.set_message(msg.to_string());
}
}
/// Drive the determinate progress bar directly (e.g. artifact retrieval)
pub fn count_progress(&self, label: &str, pos: usize, total: usize) {
if !self.active() || total == 0 {
return;
}
apply_progress(
&self.shared.top,
&mut self.shared.state.lock().unwrap(),
pos as u64,
total as u64,
);
self.shared.top.set_message(label.to_string());
}
/// Whether the widget is enabled and still drawn /// Whether the widget is enabled and still drawn
fn active(&self) -> bool { fn active(&self) -> bool {
self.shared.enabled && !self.shared.suspended.load(Ordering::SeqCst) self.shared.enabled && !self.shared.suspended.load(Ordering::SeqCst)
} }
/// Whether the widget renders at all (false on non-TTY stdout); callers /// Release the widget from the terminal (e.g. before printing
/// use this to fall back to plain-line summaries /// passthrough diagnostics or letting child cleanup commands write to
pub fn is_enabled(&self) -> bool { /// the terminal); idempotent
self.shared.enabled fn suspend(&self) {
suspend_shared(&self.shared);
} }
/// Obtain a sink feeding this view; pass it to `ContextCommand::capture` /// Success outcome body: clear the widget and print the artifacts,
pub fn sink(self: &Arc<Self>) -> Arc<dyn LineSink> {
Arc::new(Sink {
shared: self.shared.clone(),
})
}
/// Remove the widget from the terminal (e.g. before printing passthrough
/// diagnostics or letting child cleanup commands write to the terminal);
/// idempotent
///
/// Steady ticks are disabled first: otherwise a tick can redraw a frame
/// right after the clear, leaving stale copies of the widget on screen.
pub fn suspend(&self) {
if !self.shared.enabled {
return;
}
if self.shared.suspended.swap(true, Ordering::SeqCst) {
return;
}
self.shared.top.disable_steady_tick();
self.shared.pane.disable_steady_tick();
self.shared.top.finish_and_clear();
self.shared.pane.finish_and_clear();
}
/// Clear the widget and print a success summary with the artifacts,
/// rendered relative to the working directory when possible /// rendered relative to the working directory when possible
pub fn finish_success(&self, artifacts: &[PathBuf], elapsed: Duration) { fn success_summary(&self, artifacts: &[PathBuf], elapsed: Duration) {
self.suspend(); self.suspend();
if self.shared.enabled && !artifacts.is_empty() { if self.shared.enabled && !artifacts.is_empty() {
println!("Built in {}s:", elapsed.as_secs()); println!("Built in {}s:", elapsed.as_secs());
for artifact in artifacts { for artifact in artifacts {
println!(" {}", crate::ui::display_path(artifact)); println!(" {}", crate::report::display_path(artifact));
} }
} }
} }
/// Clear the widget and print a failure summary (recent captured errors /// Failure outcome body: clear the widget and print a summary (recent
/// and the path to the full log) /// captured errors and the path to the full log)
pub fn finish_failure(&self) { ///
/// On a Ctrl+C the interrupt watchdog owns the reporting — its captured
/// errors are just the killed children's death throes, and the watchdog
/// already points at the full log — so this prints nothing.
fn failure_summary(&self) {
self.suspend(); self.suspend();
if crate::interrupt::interrupted() {
return;
}
let st = self.shared.state.lock().unwrap(); let st = self.shared.state.lock().unwrap();
if self.shared.enabled && !st.errors.is_empty() { if self.shared.enabled && !st.errors.is_empty() {
eprintln!("Last captured errors:"); eprintln!("Last captured errors:");
@@ -385,15 +261,70 @@ impl DebUi {
eprintln!("Full log: {}", log_path.display()); eprintln!("Full log: {}", log_path.display());
} }
} }
}
/// Time elapsed since the view was created /// [`crate::report::BuildView`] port: forwards build events to the live
pub fn elapsed(&self) -> Duration { /// widget, so core flows drive the view without knowing it is a terminal
self.shared.started.elapsed() /// widget.
impl crate::report::BuildView for DebUi {
fn target(&self, target: BuildTarget<'_>) {
if self.active() {
self.shared.top.set_prefix(target.display.clone());
}
if target.tee_log {
let kind = if target.source_only { "build" } else { "deb" };
self.open_log(
kind,
target.package,
target.version,
&format!("for {}", target.target),
);
}
} }
/// Path of the full build log file fn phase(&self, name: &str, classifier: Box<dyn Classifier>) {
pub fn log_path(&self) -> PathBuf { self.phase_custom(name, classifier);
self.shared.log_path.lock().unwrap().clone() }
fn message(&self, text: &str) {
if self.active() {
self.shared.top.set_message(text.to_string());
}
}
fn progress(&self, label: &str, pos: usize, total: usize) {
if !self.active() || total == 0 {
return;
}
apply_progress(
&self.shared.top,
&mut self.shared.state.lock().unwrap(),
pos as u64,
total as u64,
);
self.shared.top.set_message(label.to_string());
}
fn sink(&self) -> Option<Arc<dyn LineSink>> {
Some(Arc::new(Sink {
shared: self.shared.clone(),
}))
}
fn finish_success(&self, artifacts: &[PathBuf]) {
self.success_summary(artifacts, self.shared.started.elapsed());
}
fn finish_failure(&self) {
self.failure_summary();
}
fn suspend(&self) {
DebUi::suspend(self);
}
fn is_enabled(&self) -> bool {
self.shared.enabled
} }
} }
@@ -475,23 +406,106 @@ fn push_line(shared: &Shared, st: &mut Pipeline, kind: Kind, text: String) {
let now = Instant::now(); let now = Instant::now();
if now.duration_since(st.last_draw) >= REDRAW_INTERVAL { if now.duration_since(st.last_draw) >= REDRAW_INTERVAL {
st.last_draw = now; st.last_draw = now;
shared.pane.set_message(render_pane(&st.lines)); if let Some(pane) = ensure_pane(shared) {
pane.set_message(render_pane(&st.lines));
}
} }
} }
/// Render the pane content with per-kind colors /// The pane bar, added to the terminal on the first call and reused after
///
/// Returns `None` once the widget is suspended: a line racing the suspend
/// must not re-add a bar the cleanup just cleared.
fn ensure_pane(shared: &Shared) -> Option<ProgressBar> {
let mut pane = shared.pane.lock().unwrap();
if let Some(pb) = pane.as_ref() {
return Some(pb.clone());
}
if shared.suspended.load(Ordering::SeqCst) {
return None;
}
let pb = shared.multi.add(ProgressBar::new(0));
pb.enable_steady_tick(Duration::from_millis(150));
// No template margin: multi-line messages are only prefixed by the
// template on their first line, which would misalign the pane; each
// rendered line carries its own indent instead.
pb.set_style(
ProgressStyle::default_bar()
.template("{msg}")
.expect("valid template"),
);
*pane = Some(pb.clone());
Some(pb)
}
/// Take the pane bar off the terminal; the next pushed line re-creates it
fn drop_pane(shared: &Shared) {
if let Some(pb) = shared.pane.lock().unwrap().take() {
pb.disable_steady_tick();
pb.finish_and_clear();
shared.multi.remove(&pb);
}
}
/// Render the pane content with per-kind colors, ellipsizing lines that are
/// wider than the terminal so they do not overflow onto a wrapped line
fn render_pane(lines: &VecDeque<(Kind, String)>) -> String { fn render_pane(lines: &VecDeque<(Kind, String)>) -> String {
let max_width = terminal_width().map(|w| w.saturating_sub(PANE_PREFIX_WIDTH));
render_pane_with_width(lines, max_width)
}
/// [`render_pane`] with the available pane width injected (in display
/// columns); `None` means the terminal size is unknown and lines are kept whole
fn render_pane_with_width(lines: &VecDeque<(Kind, String)>, max_width: Option<usize>) -> String {
lines lines
.iter() .iter()
.map(|(kind, text)| match kind { .map(|(kind, text)| {
Kind::Normal => format!(" │ {text}"), let text = match max_width {
Kind::Warning => format!(" │ {}", text.as_str().yellow()), Some(width) => ellipsize(text, width),
Kind::Error => format!(" │ {}", text.as_str().red()), None => text.clone(),
};
match kind {
Kind::Normal => format!(" │ {text}"),
Kind::Warning => format!(" │ {}", text.as_str().yellow()),
Kind::Error => format!(" │ {}", text.as_str().red()),
}
}) })
.collect::<Vec<_>>() .collect::<Vec<_>>()
.join("\n") .join("\n")
} }
/// Display width of the ` │ ` prefix rendered before each pane line
const PANE_PREFIX_WIDTH: usize = 4;
/// Width of the terminal in columns, or `None` when it cannot be determined
fn terminal_width() -> Option<usize> {
crossterm::terminal::size()
.ok()
.map(|(cols, _)| cols as usize)
}
/// Ellipsize `text` to at most `max_width` display columns, keeping its head
/// and appending `…` when it does not fit
fn ellipsize(text: &str, max_width: usize) -> String {
if UnicodeWidthStr::width(text) <= max_width {
return text.to_string();
}
// Reserve one column for the ellipsis itself
let budget = max_width.saturating_sub(1);
let mut out = String::new();
let mut width = 0;
for ch in text.chars() {
let w = UnicodeWidthChar::width(ch).unwrap_or(0);
if width + w > budget {
break;
}
out.push(ch);
width += w;
}
out.push('…');
out
}
/// Status bar style while no determinate progress is known /// Status bar style while no determinate progress is known
/// ///
/// The target lives on the first line and the current phase/message on its own /// The target lives on the first line and the current phase/message on its own
@@ -527,60 +541,255 @@ fn default_log_path(timestamp: &str) -> PathBuf {
dir.join(format!("pkh-{timestamp}.log")) dir.join(format!("pkh-{timestamp}.log"))
} }
static SIGINT_LOG_PATH: Mutex<Option<PathBuf>> = Mutex::new(None); /// Register the interrupt reporter: release the widget from the terminal
static SIGINT_INSTALLED: AtomicBool = AtomicBool::new(false); /// and return the log-file hint to print below the notice
///
/// The CLI watchdog runs this as the first step of the interrupt shutdown,
/// so the widget disappears the moment Ctrl+C is hit. Registered only for
/// enabled views: in `--verbose` mode or with piped output there is no
/// widget and nothing to report.
fn set_interrupt_reporter(shared: Arc<Shared>) {
crate::interrupt::set_reporter(Box::new(move || interrupt_report(&shared)));
}
/// Install a best-effort Ctrl+C handler clearing the widget and pointing at /// [`suspend_shared`] plus the log-file hint, in teardown order
/// the log file before exiting ///
fn install_sigint_hook(log_path: &Path) { /// Testable end to end: the reporter closure is private to the interrupt
update_sigint_log_path(log_path); /// watchdog, but the drawing behavior is not.
if SIGINT_INSTALLED.swap(true, Ordering::SeqCst) { fn interrupt_report(shared: &Shared) -> Option<String> {
suspend_shared(shared);
let log_path = shared.log_path.lock().unwrap().clone();
if log_path.exists() {
Some(format!("Full log: {}", log_path.display()))
} else {
None
}
}
/// [`DebUi::suspend`] body, shared with the interrupt reporter
///
/// Steady ticks are disabled first: otherwise a tick can redraw a frame
/// right after the clear, leaving stale copies of the widget on screen.
/// The tty echo suppressed at view start is restored here, before the
/// erases: an echo from a keypress landing mid-teardown could not shift the
/// cursor anymore. The draw target is finally killed: the interrupted flow
/// keeps emitting log records while the cleanup hooks run, and every one of
/// them would otherwise make the log bridge repaint the cleared bars from
/// their cached frames.
fn suspend_shared(shared: &Shared) {
if !shared.enabled {
return; return;
} }
if shared.suspended.swap(true, Ordering::SeqCst) {
return;
}
restore_tty_echo();
shared.top.disable_steady_tick();
drop_pane(shared);
shared.top.finish_and_clear();
shared.multi.set_draw_target(ProgressDrawTarget::hidden());
}
// SAFETY: installing a signal handler; the handler itself is best-effort /// Termios snapshot taken when the echo is suppressed; `Some` only while the
// (it performs non async-signal-safe operations, acceptable here because /// live view is on screen
// it immediately exits afterwards). static SAVED_TTY_TERMIOS: Mutex<Option<libc::termios>> = Mutex::new(None);
/// Stop the tty from rendering control-character input (^C would show as a
/// visible two-character "^C") while the live view is up: rendered echoes
/// move the cursor without indicatif knowing, and the teardown erase ends
/// up aimed past the widget
///
/// ECHO itself stays on — turning it off would trigger terminals'
/// hidden-input padlock — so the only visible difference is that a ^C
/// keypress echoes as a raw, cursor-invisible control byte. No-op without a
/// tty on stdin.
fn suppress_control_char_echo() {
// SAFETY: tcgetattr on stdin with a valid, zero-initialized buffer
let mut termios: libc::termios = unsafe { std::mem::zeroed() };
// SAFETY: reading the current attributes of stdin
if unsafe { libc::tcgetattr(libc::STDIN_FILENO, &mut termios) } != 0 {
return;
}
*SAVED_TTY_TERMIOS.lock().unwrap() = Some(termios);
termios.c_lflag &= !libc::ECHOCTL;
// SAFETY: applying the modified attributes to stdin
unsafe { unsafe {
libc::signal(libc::SIGINT, on_sigint as *const () as usize); libc::tcsetattr(libc::STDIN_FILENO, libc::TCSANOW, &termios);
} }
} }
/// Point the sigint handler at the current log file location /// Restore the tty attributes saved by [`suppress_tty_echo`]
fn update_sigint_log_path(log_path: &Path) { fn restore_tty_echo() {
*SIGINT_LOG_PATH.lock().unwrap() = Some(log_path.to_path_buf()); if let Some(termios) = SAVED_TTY_TERMIOS.lock().unwrap().take() {
// SAFETY: re-applying the snapshot taken at view start
unsafe {
libc::tcsetattr(libc::STDIN_FILENO, libc::TCSANOW, &termios);
}
}
} }
extern "C" fn on_sigint(_sig: libc::c_int) { #[cfg(test)]
// Best-effort cleanup: clear leftover widget lines and show the cursor mod tests {
let _ = execute!( use super::*;
std::io::stdout(),
Clear(ClearType::FromCursorDown), #[test]
cursor::Show fn ellipsize_keeps_short_lines() {
); assert_eq!(ellipsize("short", 10), "short");
if let Ok(guard) = SIGINT_LOG_PATH.try_lock() assert_eq!(ellipsize("exactly10!", 10), "exactly10!");
&& let Some(path) = guard.as_ref()
{
eprintln!("\nInterrupted — full log: {}", path.display());
} }
// Run the registered cleanup hooks (currently: unmount and remove the
// ephemeral build chroot, see `deb::ephemeral::sigint_cleanup_chroot`), #[test]
// then exit with the conventional 130 status. fn ellipsize_truncates_long_lines_to_the_width_budget() {
// let out = ellipsize("a very long build line that overflows", 20);
// Like the terminal restoration above, this is NOT strictly assert_eq!(UnicodeWidthStr::width(out.as_str()), 20);
// async-signal-safe: it locks a mutex, spawns subprocesses and does I/O. assert!(out.ends_with('…'));
// That is a deliberate tradeoff, no worse than the rest of this handler: assert!(out.starts_with("a very long build"));
// exiting immediately would skip all destructors and leak the chroot }
// together with its bind-mounted /proc and overlay mounts. The hooks are
// self-contained (they only touch stored paths and spawn umount/rm #[test]
// directly), so they cannot deadlock on a lock the interrupted thread fn ellipsize_never_exceeds_the_budget_with_wide_characters() {
// might have held; the hook registry itself is only ever taken with let out = ellipsize("wíth émojis 🎉 and 文字 mixing", 12);
// try_lock plus a bounded retry for the same reason. Note that SIGINT assert!(UnicodeWidthStr::width(out.as_str()) <= 12);
// stays blocked for the duration of the handler, so a second Ctrl-C will assert!(out.ends_with('…'));
// not interrupt a slow cleanup — send SIGTERM/SIGKILL if it ever hangs. }
crate::deb::ephemeral::run_cleanup_hooks();
// SAFETY: raw exit bypassing destructors, intended in a signal handler #[test]
unsafe { fn ellipsize_degenerate_width_still_terminates() {
libc::_exit(130); assert_eq!(ellipsize("overflowing", 0), "…");
assert_eq!(ellipsize("overflowing", 1), "…");
}
#[test]
fn pane_lines_are_ellipsized_but_keep_their_prefix_and_color() {
let mut lines = VecDeque::new();
lines.push_back((
Kind::Normal,
"gcc -DHAVE_CONFIG_H -I. -I.. -g -O2 -c hello.c".to_string(),
));
lines.push_back((
Kind::Error,
"an error much too long for the pane".to_string(),
));
let rendered = render_pane_with_width(&lines, Some(20));
let rendered = rendered.lines().collect::<Vec<_>>();
assert_eq!(rendered.len(), 2);
// The injected budget is the text width; every rendered line stays
// within the simulated terminal width (prefix + budget)
for line in &rendered {
let plain = strip_ansi(line);
assert!(
UnicodeWidthStr::width(plain.as_str()) <= 20 + PANE_PREFIX_WIDTH,
"{plain}"
);
}
assert!(strip_ansi(rendered[0]).starts_with(" │ gcc -DHAVE_CONFIG_H"));
// The error keeps its color wrapping around the ellipsized text
assert!(rendered[1].contains('\x1b'), "{rendered:?}");
assert!(strip_ansi(rendered[1]).starts_with(" │ an error much too l…"));
}
#[test]
fn pane_lines_are_kept_whole_without_a_known_terminal_size() {
let mut lines = VecDeque::new();
lines.push_back((
Kind::Normal,
"a line that would overflow a narrow pane".to_string(),
));
let rendered = render_pane_with_width(&lines, None);
assert!(rendered.contains("a line that would overflow a narrow pane"));
}
/// A fresh view has no pane at all: flows without subprocess output
/// (`pkh put`) must not render anything until a line arrives, and a
/// dropped or suspended pane stays gone
#[test]
fn pane_is_created_lazily_and_dropped_cleanly() {
let multi = MultiProgress::new();
let shared = Shared {
multi: multi.clone(),
top: multi.add(ProgressBar::new(0)),
pane: Mutex::new(None),
state: Mutex::new(Pipeline {
classifier: Box::new(GenericClassifier::new()),
lines: VecDeque::new(),
errors: Vec::new(),
last_draw: Instant::now(),
bar_total: 0,
}),
tee: Mutex::new(None),
log_path: Mutex::new(std::env::temp_dir().join("pkh-pane-test.log")),
timestamp: String::new(),
enabled: true,
suspended: AtomicBool::new(false),
started: Instant::now(),
};
assert!(shared.pane.lock().unwrap().is_none());
ensure_pane(&shared).unwrap();
assert!(shared.pane.lock().unwrap().is_some());
// Later lines hit the stored bar instead of stacking another one
ensure_pane(&shared).unwrap();
assert!(shared.pane.lock().unwrap().is_some());
drop_pane(&shared);
assert!(shared.pane.lock().unwrap().is_none());
// A line racing the suspend must not re-add the cleared bar
shared.suspended.store(true, Ordering::SeqCst);
assert!(ensure_pane(&shared).is_none());
}
/// The interrupt report kills the shared draw target: during the cleanup
/// hooks the interrupted flow keeps emitting log records, and every one
/// of them would otherwise make the log bridge repaint the bars from
/// their cached frames — resurrecting the widget that was just cleared.
#[test]
fn interrupt_report_disables_further_redraws() {
let multi = MultiProgress::new();
let shared = Shared {
multi: multi.clone(),
top: multi.add(ProgressBar::new(0)),
pane: Mutex::new(None),
state: Mutex::new(Pipeline {
classifier: Box::new(GenericClassifier::new()),
lines: VecDeque::new(),
errors: Vec::new(),
last_draw: Instant::now(),
bar_total: 0,
}),
tee: Mutex::new(None),
log_path: Mutex::new(std::env::temp_dir().join("pkh-interrupt-report-test.log")),
timestamp: String::new(),
enabled: true,
suspended: AtomicBool::new(false),
started: Instant::now(),
};
interrupt_report(&shared);
assert!(shared.suspended.load(Ordering::SeqCst));
assert!(shared.multi.is_hidden());
}
/// Best-effort ANSI escape stripper, enough for the assertions above
fn strip_ansi(line: &str) -> String {
let mut out = String::new();
let mut chars = line.chars();
while let Some(ch) = chars.next() {
if ch == '\x1b' {
for esc in chars.by_ref() {
if esc.is_ascii_alphabetic() {
break;
}
}
} else {
out.push(ch);
}
}
out
} }
} }
+50 -4
View File
@@ -12,7 +12,7 @@ use crossterm::{
style::{self, Color, Print, SetForegroundColor}, style::{self, Color, Print, SetForegroundColor},
terminal, terminal,
}; };
use std::io::{self, Write}; use std::io::{self, IsTerminal, Write};
use std::time::Duration; use std::time::Duration;
/// Why a prompt could not run interactively /// Why a prompt could not run interactively
@@ -32,9 +32,8 @@ impl std::fmt::Display for PromptError {
impl std::error::Error for PromptError {} impl std::error::Error for PromptError {}
/// Answer validator of [`text`]: accepts the answer, or explains why it is /// Answer validator of [`text`] (re-exported from [`crate::report`])
/// rejected pub use crate::report::Validator;
pub type Validator = dyn Fn(&str) -> Result<(), String>;
/// What a prompt's event loop asks its drawing helper to render; keeping all /// What a prompt's event loop asks its drawing helper to render; keeping all
/// terminal access behind this callback makes the loops pure logic that unit /// terminal access behind this callback makes the loops pure logic that unit
@@ -127,6 +126,53 @@ pub fn confirm(label: &str, default: bool) -> Result<bool, Box<dyn std::error::E
} }
} }
/// [`crate::report::Prompter`] answered by the interactive terminal: each
/// question runs the matching raw-mode prompt. Without an interactive
/// terminal (CI, piped input) [`Prompter::interactive`] is false — flows
/// then take their headless path instead of asking — and a cancel (Ctrl+C)
/// propagates as `Err` so flows can abort.
pub struct TerminalPrompter;
impl crate::report::Prompter for TerminalPrompter {
fn interactive(&self) -> bool {
std::io::stdin().is_terminal() && std::io::stdout().is_terminal()
}
fn confirm(&self, question: &str, default: bool) -> Result<bool, Box<dyn std::error::Error>> {
confirm(question, default)
}
fn select(
&self,
label: &str,
options: &[String],
default: &str,
) -> Result<String, Box<dyn std::error::Error>> {
select(label, options, default)
}
fn text(
&self,
label: &str,
default: &str,
validate: Option<&Validator>,
) -> Result<String, Box<dyn std::error::Error>> {
text(label, default, validate)
}
fn accept_host_key(&self, host: &str, key_type: &str, fingerprint: &str) -> bool {
// The banner is plain output: the confirmation prompt itself must
// stay a single line for its redraw logic
println!("The authenticity of host '{host}' can't be established.");
println!("{key_type} key fingerprint is {fingerprint}.");
confirm("Accept and store this host key?", false).unwrap_or(false)
}
fn present(&self, text: &str) {
println!("{text}");
}
}
/// Run `prompt` with the terminal in raw mode, always restoring it /// Run `prompt` with the terminal in raw mode, always restoring it
/// afterwards. Fails with [`PromptError::NotATty`] when raw mode cannot be /// afterwards. Fails with [`PromptError::NotATty`] when raw mode cannot be
/// enabled. /// enabled.